CN103617268B - Method and device for processing big data - Google Patents
Method and device for processing big data Download PDFInfo
- Publication number
- CN103617268B CN103617268B CN201310646544.0A CN201310646544A CN103617268B CN 103617268 B CN103617268 B CN 103617268B CN 201310646544 A CN201310646544 A CN 201310646544A CN 103617268 B CN103617268 B CN 103617268B
- Authority
- CN
- China
- Prior art keywords
- hadoop
- task
- internal
- data
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000012545 processing Methods 0.000 title abstract description 6
- 238000005194 fractionation Methods 0.000 claims description 35
- 230000002159 abnormal effect Effects 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 10
- 238000012217 deletion Methods 0.000 claims description 7
- 230000037430 deletion Effects 0.000 claims description 7
- 238000000151 deposition Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2219—Large Object storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/282—Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
Abstract
The invention discloses a method and device for processing big data. The method includes the steps that in the external environment of a hadoop distributed system infrastructure, a preset division strategy is used for dividing a hadoop total query task; hadoop internal query is conducted respectively in a hadoop distributed storage system according to the divided hadoop query task; all internal query results are stored to the hadoop external environment according to a preset storage strategy; statistics are made on storage information, and the storage information is displayed in a hadoop external query mode. The invention further discloses the device corresponding to the method. According to the method and device for processing the big data, the total query task is divided in the hadoop external environment, after the division query task is executed, the internal query results are correspondingly stored in a hierarchical mode according to the task division, paging browse of hadoop big data access is achieved, and data statistics are made on the premise that system performance is not affected.
Description
Technical field
The application is related to big data process field, and espespecially a kind of realization based on distributed system architecture hadoop is big
The method and device of data processing.
Background technology
The continuous development of digital living, creates the mass data of volume surge.The big data of many industries adopts at present
Distributed system architecture (hadoop), to be processed, and is shared to the integration of data with realization.Although hadoop has
Very strong extensibility, but because its high performance design also makes the application of hadoop be restricted.To surf the Net, discharge record is looked into
As a example inquiry, when data query is carried out to the big data being stored in hadoop, because hadoop only supports distributed file system
(hdfs) stream-oriented file reads, and can only carry out order from start to end for Query Result and read, this with hadoop
External environment condition when carrying out data query, it is possible to achieve larger gap has been browsed to the cross-page of Query Result.In addition hadoop pair
When bar number statistics made in surfing flow record, the stream-oriented file due to hdfs limits, and Statistical Speed is very slow and increased the negative of system
Load.Additionally, excessively high-frequency inquiry, also increase overhead.
From above analyzing examples, pass through hadoop internal environment at present, that is, in hadoop system architecture, to several greatly
During according to being inquired about, the hdfs under hadoop internal environment only supports that streaming fashion reads file, and that is, application program exists every time
The mode reading file in hdfs is from beginning order to ending, and this makes needing effect when reading inquiry file medium content
Rate step-down.In addition, if carrying out inquiring about data statistics, speed is very slow and increased system burden.If enquiry frequency is too high, and
Overhead can be increased.In a word, the processing method to data query and display for the hdfs of current hadoop, have impact on user couple
The use of data access, the method simultaneously adopting at present have impact on the service behaviour of hadoop.
Content of the invention
In order to solve the above problems, the present invention provides the method that a kind of big data of hadoop is processed, and is capable of to big
Data query result show cross-page browse, and under not affecting big data architecture system performance improve query statistic efficiency.
In order to reach the purpose of the present invention, the application provides a kind of method realizing big data process, comprising:
In distributed system architecture hadoop external environment condition, split hadoop using the fractionation strategy pre-setting total
Query task；
According to the hadoop query task splitting, carry out respectively inside hadoop in the distributed memory system of hadoop
Inquiry；
Each internal queries result is stored in hadoop external environment condition according to the storage strategy pre-setting；
Storage information is counted and is shown by hadoop external inquiry mode.
Further, split strategy to include: according to Annual distribution and/or, inquiry customer flow usage type is inquired about
Split.
Further, the method being counted includes:
Query statistic is carried out to every one page of the internal queries result of each fractionation task, and by every for each fractionation task one page
The query statistic information of portion's Query Result merges, and is accordingly split the statistical information of task；
The corresponding statistical information splitting task is merged, obtains the statistics of total query task.
Further, it is stored in hadoop external environment condition according to storage strategy to include:
Internal queries result is divided into according to storage strategy: normality data and abnormal data；
Carry out classification storage according to normality data and abnormal data,
When internal queries result is normality data, when normality data volume is more than the memory size of hadoop external environment condition,
According to memory size, first, the query page once accessing is saved in internal memory, then internal queries result part is stored in interior
Deposit；Remaining normality data is stored in the hard disk of hadoop external environment condition；When normality data volume is less than in hadoop external environment condition
When depositing capacity, all internal queries results are stored in internal memory；
When internal queries result is abnormal data, internal queries result is stored entirely in the hard disk of external environment condition.
Further, the method also includes: according to enquiry frequency, sets the deletion cycle to delete in internal queries result
Normality data.
On the other hand, the present invention provides a kind of device realizing big data process, is arranged at hadoop external environment condition, bag
Include: control unit, task split cells, classification memory cell, statistics display unit；Wherein,
Control unit, for when receiving query task, control task split cells carries out task fractionation；Control classification
Memory cell stores to internal Query Result；Control statistics display unit to storage internal queries result counted and
Display；
Task split cells, the fractionation strategy for pre-setting splits the total query task of hadoop, is sent in hadoop
Portion carries out data query；
Classification memory cell, receives hadoop internal queries result for setting, is stored in hadoop according to storage strategy
External environment condition；
Statistics display unit, for being counted to classification storage information by query statement and being shown.
Further, fractionation strategy includes: according to Annual distribution and/or, inquiry customer flow usage type is looked into
Ask and split.
Further, carry out statistics to include:
Query statistic is carried out to every one page of the internal queries result of each fractionation task, and every for each fractionation task one page is looked into
Ask statistical information to merge, accordingly split the statistical information of task；
The corresponding statistical information splitting task is merged, obtains the statistics of total query task.
Further, it is stored in hadoop external environment condition according to storage strategy to include:
Internal queries result is divided into according to storage strategy: normality data and abnormal data；
Carry out classification storage according to normality data and abnormal data,
When internal queries result is normality data, when normality data volume is more than the memory size of hadoop external environment condition,
According to memory size, first, the query page once accessing is saved in internal memory, then internal queries result part is stored in interior
Deposit；Remaining normality data is stored in the hard disk of hadoop external environment condition；When normality data volume is less than in hadoop external environment condition
When depositing capacity, all internal queries results are stored in internal memory；
When internal queries result is abnormal data, internal queries result is stored entirely in the hard disk of external environment condition.
Further, this device also includes deleting unit, for according to enquiry frequency, arranging the deletion cycle to delete inside
Normality data in Query Result.
The present invention proposes a kind of method realizing big data process, comprising: outside distributed system architecture hadoop
Portion's environment, splits the total query task of hadoop using the fractionation strategy pre-setting；According to split hadoop query task,
Carry out hadoop internal queries respectively in the distributed memory system of hadoop；By each internal queries result according to pre-setting
Storage strategy is stored in hadoop external environment condition；Storage information is counted and is shown by hadoop external inquiry mode.
Total query task is split in hadoop external environment condition by the inventive method, after having executed fractionation query task, will
Internal queries result according to task split be classified storage accordingly, realize hadoop big data access Tabbed browsing and
Do not affect to carry out data statistics under systematic function.
Brief description
Accompanying drawing is used for providing technical scheme is further understood, and constitutes a part for specification, with this
The embodiment of application is used for explaining the technical scheme of the application together, does not constitute the restriction to technical scheme.
Fig. 1 is the flow chart of the method that the present invention realizes big data process；
Fig. 2 is the structured flowchart of the device that the present invention realizes big data process.
Specific embodiment
Purpose, technical scheme and advantage for making the application become more apparent, below in conjunction with accompanying drawing to the application
Embodiment be described in detail.It should be noted that in the case of not conflicting, in embodiment in the application and embodiment
Feature can mutually be combined.
Fig. 1 is the flow chart of the method that the present invention realizes big data process, as shown in figure 1, with the big number based on hadoop
As a example processing, comprising:
Step 100, in hadoop external environment condition, split the total query task of hadoop using the fractionation strategy pre-setting.
In this step, split strategy and include: according to Annual distribution and/or, inquiry customer flow usage type is inquired about
Split.
So that the total query task of hadoop is for inquiry customer traffic as a example, when carrying out query task and splitting it is assumed that passing through to look into
Ask customer flow usage type and carry out inquiry fractionation, then made a distinction by the existing statistics to customer flow use:
Hypothesis is divided into: flow intelligent, the flow golden mean of the Confucian school and flow silence.Then class can be used according to the inquiry customer flow of above-mentioned differentiation
Type carries out inquiring about fractionation it is assumed that the flow use ratio of three of the above client is 9:3:1, then three kinds of clients are carrying out customer traffic
During inquiry, its data on flows is proportional with flow use ratio.Therefore, flow intelligent in the data of hadoop storage
Data is most, and the amount of user data of the flow golden mean of the Confucian school is placed in the middle, and the amount of user data of flow silence is minimum.When data volume is many,
Query task after fractionation is more.Assume that the thread resources that the task that each splits takies are identical, then in order to keep data to look into
That askes is efficient, and the query task after the shared fractionation of the user more than data volume is many；And the few user data of data volume, then split appoints
Business is relatively few.If carrying out task fractionation, according to variety classes by inquiring about the distribution of customer flow usage type binding time
The number of tasks that split of client with to split number with customer flow usage type identical.Divided by the time using with reference to flow
Cloth, the time period used in flow set, due to producing more data on flows, when therefore carrying out query task fractionation, flow
Time period in data set, the corresponding number of tasks splitting is more；And flow is using the time period do not concentrated, query task fractionation
Number of tasks less.Assume for the query task of flow intelligent in total query task to be split as 9, and 8 days beginning of the month and the 8 days the end of month
Use time of concentration for flow, middle of the month flow uses less, then, when carrying out task and splitting, the data at 8 days beginning of the month and the 8 days the end of month is looked into
Inquiry can respectively be split as 3 query tasks, and 14 days middle of the month (assume one month 30 days) although the time longer, due to data
Amount is little, also only splits into 3 query tasks.When the present invention carries out splitting strategy design, can be with reference to the statistics of prior art
Carry out, after query task of the present invention statistics, also can be adjusted correspondingly according to the data that itself obtains, it splits strategy simultaneously
Non- one layer constant, and above citing is intended merely to the clearer explanation present invention, in the protection being not meant to limit the present invention
Hold.By query task to be split as the query task number being directly proportional to ratio data relation according to query strategy, look into for each
Inquiry task distributes corresponding thread resources, realizes the reasonable of total query task fractionation, effectively improves the operating efficiency of query task.
Step 101, according to split hadoop query task, carry out respectively in the distributed memory system of hadoop
Hadoop internal queries.
It should be noted that internal queries here refer to the inquiry carrying out in the distributed memory system of hadoop.
Step 102, each internal queries result is stored in hadoop external environment condition according to the storage strategy pre-setting.
In this step, it is stored in hadoop external environment condition according to storage strategy and includes:
Internal queries result is divided into according to storage strategy: normality data and abnormal data；
Carry out classification storage according to normality data and abnormal data,
When internal queries result is normality data, when normality data volume is more than the memory size of hadoop external environment condition,
According to memory size, first, the query page once accessing is saved in internal memory, then internal queries result part is stored in interior
Deposit；Remaining normality data is stored in the hard disk of hadoop external environment condition；When normality data volume is less than in hadoop external environment condition
When depositing capacity, all internal queries results are stored in internal memory.It should be noted that the data being saved in internal memory is according to interior
The working condition deposited, typically with the maximum capacity of 80% about work storage normality data of hardware memory capacity.
When internal queries result is abnormal data, internal queries result is stored entirely in the hard disk of external environment condition.
It should be noted that normality data is the user of regular complaint in business system, and look in systems
Ask the data of overwriting；Abnormal data is the data of the user infrequently complaining in business system.And once accessed
Query page, refer in the inquiry log of system record carrying out inquiry content, this partly can pass through prior art
In system journal obtaining.The content that this part exists for prior art, will not be described here.
Because the external environment condition of storage, its internal queries result is by the inquiry with prior art hadoop external environment condition
Display is identical, is embodied in the form of multipage.
Step 103, storage information is counted and is shown by hadoop external inquiry mode.
Query statistic is carried out to every one page of the internal queries result of each fractionation task, and by every for each fractionation task one page
The query statistic information of portion's Query Result merges, and is accordingly split the statistical information of task；
The corresponding statistical information splitting task is merged, obtains the statistics of total query task.
The inventive method also includes: according to enquiry frequency, sets the deletion cycle to delete the normality in internal queries result
Data.
It should be noted that the normality data in internal queries result in internal memory and hard disk can be with certain deletion cycle
Deleted, general data in EMS memory is with 7 days for a deletion cycle.And the data in hard disk is with 15 days for deleting the cycle.
Certainly according to actual conditions, enquiry frequency is higher, illustrates to need the cycle of query statistic information updating shorter, and therefore it deletes week
Phase may be shorter.
Fig. 2 is a kind of big data processing meanss structured flowchart of hadoop of the present invention, as shown in Fig. 2 a kind of distributed system
The device that system architecture hadoop big data is processed, is arranged at hadoop external environment condition, comprising: control unit, task split
Unit, classification memory cell, statistics display unit；Wherein,
Control unit, for when receiving query task, control task split cells carries out task fractionation；Control classification
Memory cell stores to internal Query Result；Control statistics display unit to storage internal queries result counted and
Display；
Task split cells, the fractionation strategy for pre-setting splits the total query task of hadoop, is sent in hadoop
Portion carries out data query.Split strategy to include: according to Annual distribution and/or, inquiry customer flow usage type carries out inquiry and tears open
Point.
Classification memory cell, receives hadoop internal queries result for setting, is stored in hadoop according to storage strategy
External environment condition.
Further, it is stored in hadoop external environment condition according to storage strategy to include:
Internal queries result is divided into according to storage strategy: normality data and abnormal data；
Carry out classification storage according to normality data and abnormal data,
When internal queries result is normality data, when normality data volume is more than the memory size of hadoop external environment condition,
According to memory size, first, the query page once accessing is saved in internal memory, then internal queries result part is stored in interior
Deposit；Remaining normality data is stored in the hard disk of hadoop external environment condition；When normality data volume is less than in hadoop external environment condition
When depositing capacity, all internal queries results are stored in internal memory；
When internal queries result is abnormal data, internal queries result is stored entirely in the hard disk of external environment condition.
Statistics display unit, for being counted to classification storage information by query statement and being shown.
Further, statistics includes: query statistic is carried out to every one page of the internal queries result of each fractionation task, and will
The every one page query statistic information of each fractionation task merges, and is accordingly split the statistical information of task；
The corresponding statistical information splitting task is merged, obtains the statistics of total query task.
Apparatus of the present invention also include deleting unit, for according to enquiry frequency, arranging the deletion cycle to delete internal queries
Normality data in result.
Although the embodiment disclosed by the application is as above, described content only readily appreciates that the application adopts
Embodiment, is not limited to the application.Technical staff in any the application art, is being taken off without departing from the application
On the premise of the spirit and scope of dew, any modification and change can be carried out in the form implemented and details, but the application
Scope of patent protection, still must be defined by the scope of which is defined in the appended claims.
Claims (8)
1. a kind of method realizing big data process is it is characterised in that include:
In distributed system architecture hadoop external environment condition, split hadoop using the fractionation strategy pre-setting and always inquire about
Task；
According to the hadoop query task splitting, carry out respectively looking into inside hadoop in the distributed memory system of hadoop
Ask；
Each internal queries result is stored in hadoop external environment condition according to the storage strategy pre-setting；
Storage information is counted and is shown by hadoop external inquiry mode；
Wherein, the described method being counted includes:
Query statistic is carried out to every one page of the internal queries result of each fractionation task, and looks into inside every for each fractionation task one page
The query statistic information asking result merges, and is accordingly split the statistical information of task；
The corresponding statistical information splitting task is merged, obtains the statistics of total query task.
2. method according to claim 1 is it is characterised in that described fractionation strategy includes: according to Annual distribution and/or look into
Ask customer flow usage type and carry out inquiry fractionation.
3. method according to claim 1 is it is characterised in that described be stored in hadoop external environment condition according to storage strategy
Including:
Internal queries result is divided into according to storage strategy: normality data and abnormal data；
Carry out classification storage according to normality data and abnormal data,
When internal queries result is normality data, when normality data volume is more than the memory size of hadoop external environment condition, according to
Memory size, first, the query page once accessing is saved in internal memory, then internal queries result part is stored in internal memory；
Remaining normality data is stored in the hard disk of hadoop external environment condition；When the internal memory that normality data volume is less than hadoop external environment condition holds
During amount, all internal queries results are stored in internal memory；
When internal queries result is abnormal data, internal queries result is stored entirely in the hard disk of external environment condition.
4. method according to claim 3 is it is characterised in that the method also includes: according to enquiry frequency, sets and deletes week
Phase is to delete the normality data in internal queries result.
5. a kind of device realizing big data process is it is characterised in that be arranged at hadoop external environment condition, comprising: control unit,
Task split cells, classification memory cell, statistics display unit；Wherein,
Control unit, for when receiving query task, control task split cells carries out task fractionation；Control classification storage
Unit stores to internal Query Result；Statistics display unit is controlled the internal queries result of storage to be counted and is shown
Show；
Task split cells, for splitting the total query task of hadoop using the fractionation strategy pre-setting, is sent in hadoop
Portion carries out data query；
Classification memory cell, receives hadoop internal queries result for setting, is stored in outside hadoop according to storage strategy
Environment；
Statistics display unit, for being counted to classification storage information by query statement and being shown；The described statistics that carries out is wrapped
Include:
Query statistic is carried out to every one page of the internal queries result of each fractionation task, and every for each fractionation task one page is inquired about system
Meter information merges, and is accordingly split the statistical information of task；
The corresponding statistical information splitting task is merged, obtains the statistics of total query task.
6. device according to claim 5 is it is characterised in that described fractionation strategy includes: according to Annual distribution and/or
Inquiry customer flow usage type carries out inquiry and splits.
7. device according to claim 5 is it is characterised in that described be stored in hadoop external environment condition according to storage strategy
Including:
Internal queries result is divided into according to storage strategy: normality data and abnormal data；
Carry out classification storage according to normality data and abnormal data,
When internal queries result is normality data, when normality data volume is more than the memory size of hadoop external environment condition, according to
Memory size, first, the query page once accessing is saved in internal memory, then internal queries result part is stored in internal memory；
Remaining normality data is stored in the hard disk of hadoop external environment condition；When the internal memory that normality data volume is less than hadoop external environment condition holds
During amount, all internal queries results are stored in internal memory；
When internal queries result is abnormal data, internal queries result is stored entirely in the hard disk of external environment condition.
8. device according to claim 7 is it is characterised in that this device also includes deleting unit, for according to inquiry frequency
Rate, the setting deletion cycle is to delete the normality data in internal queries result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310646544.0A CN103617268B (en) | 2013-12-04 | 2013-12-04 | Method and device for processing big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310646544.0A CN103617268B (en) | 2013-12-04 | 2013-12-04 | Method and device for processing big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103617268A CN103617268A (en) | 2014-03-05 |
| CN103617268B true CN103617268B (en) | 2017-01-18 |
Family
ID=50167971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310646544.0A Active CN103617268B (en) | 2013-12-04 | 2013-12-04 | Method and device for processing big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103617268B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677650B (en) * | 2014-11-18 | 2019-06-25 | 中国移动通信集团公司 | A kind of data cached storage method and device |
| CN109118365A (en) * | 2017-06-26 | 2019-01-01 | 平安科技（深圳）有限公司 | Income calculation method, apparatus and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1588358A (en) * | 2004-08-26 | 2005-03-02 | 陈红 | Treating method and system for MDX multidimensional data search statement |
| CN102779183A (en) * | 2012-07-02 | 2012-11-14 | 华为技术有限公司 | Data inquiry method, equipment and system |
| CN102799622A (en) * | 2012-06-19 | 2012-11-28 | 北京大学 | Distributed structured query language (SQL) query method based on MapReduce expansion framework |
| CN103235835A (en) * | 2013-05-22 | 2013-08-07 | 曙光信息产业(北京)有限公司 | Inquiry implementation method for database cluster and device |
-
2013
- 2013-12-04 CN CN201310646544.0A patent/CN103617268B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1588358A (en) * | 2004-08-26 | 2005-03-02 | 陈红 | Treating method and system for MDX multidimensional data search statement |
| CN102799622A (en) * | 2012-06-19 | 2012-11-28 | 北京大学 | Distributed structured query language (SQL) query method based on MapReduce expansion framework |
| CN102779183A (en) * | 2012-07-02 | 2012-11-14 | 华为技术有限公司 | Data inquiry method, equipment and system |
| CN103235835A (en) * | 2013-05-22 | 2013-08-07 | 曙光信息产业(北京)有限公司 | Inquiry implementation method for database cluster and device |
Non-Patent Citations (3)
| Title |
|---|
| Hadoop整合Cassandra处理海量数据;苏翔宇 等;《电脑知识与技术》;20130305;第9卷(第7期);1491-1493页 * |
| 基于Hadoop及关系型数据库的海量数据分析研究;程莹 等;《电信科学》;20101115;第26卷(第11期);47-50页 * |
| 大数据分析——RDBMS与MapReduce的竞争与共生;覃雄派 等;《软件学报》;20120131;第23卷(第01期);33-45页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103617268A (en) | 2014-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101795211B (en) | Data storage method and system | |
| US20200089624A1 (en) | Apparatus and method for managing storage of data blocks | |
| CN103902467B (en) | Compressed memory access control method, device and system | |
| CN105868286B (en) | The parallel method of adding and system merged based on distributed file system small documents | |
| CN103617287A (en) | Log management method and device in distributed environment | |
| CN105378716B (en) | A kind of conversion method and device of data memory format | |
| CN104407879B (en) | A kind of power network sequential big data loaded in parallel method | |
| CN104580437A (en) | Cloud storage client and high-efficiency data access method thereof | |
| CN103488685B (en) | Fragmented-file storage method based on distributed storage system | |
| CN102307234A (en) | Resource retrieval method based on mobile terminal | |
| US11625412B2 (en) | Storing data items and identifying stored data items | |
| CN103365971A (en) | Mass data access processing system based on cloud computing | |
| CN104270412A (en) | Three-level caching method based on Hadoop distributed file system | |
| US20150112934A1 (en) | Parallel scanners for log based replication | |
| CN104572505A (en) | System and method for ensuring eventual consistency of mass data caches | |
| CN104915902A (en) | Cloud platform based implementation method for take-out order online delivery | |
| CN105162622B (en) | A kind of storage method and system | |
| CN104660442B (en) | Service opening method based on MongoDB and system | |
| CN103617268B (en) | Method and device for processing big data | |
| CN109885642A (en) | Classification storage method and device towards full-text search | |
| CN110768829A (en) | Method for realizing linear increase of traffic analysis service performance based on DPDK | |
| CN103645873B (en) | A kind of method realizing efficient data caching in trend curve system | |
| CN107423321B (en) | Method and device suitable for cloud storage of large-batch small files | |
| CN104717251A (en) | Scheduling method and system for Cell nodes through OpenStack cloud computing management platform | |
| CN103685544A (en) | Performance pre-evaluation based client cache distributing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |