CN104778235A - Tree traversal searching method based on MapReduce cloud calculation model - Google Patents
Tree traversal searching method based on MapReduce cloud calculation model Download PDFInfo
- Publication number
- CN104778235A CN104778235A CN201510153681.XA CN201510153681A CN104778235A CN 104778235 A CN104778235 A CN 104778235A CN 201510153681 A CN201510153681 A CN 201510153681A CN 104778235 A CN104778235 A CN 104778235A
- Authority
- CN
- China
- Prior art keywords
- tree
- traversal
- mapreduce
- node
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a tree traversal searching method based on a MapReduce cloud calculation model. The tree traversal searching method comprises the specific steps of abstracting a data source to be traversed into a marked tree model, marking the marked tree model in a coordinate mode according to the tree depth and the tree breadth, searching data, carrying out tree traversal on a generated data marked tree, processing a task by using the MapReduce model, resolving the task into a plurality of sub tasks, carrying parallel execution on the sub tasks, marking results of the sub tasks through flags, carrying out logic or operation on the results of all sub tasks to obtain a searching result, judging whether tree knots are the same according to the returned result in a tree traversal process and further obtaining whether the knots overlap. Compared with the prior art, the tree traversal searching method based on the MapReduce cloud calculation model can quickly finish the task for processing mass data on an oversized cluster, can expand and utilize a cloud platform and a network technique, enables operation to achieve task distribution processing on different computers in the internet and greatly improves the data processing efficiency.
Description
Technical field
The present invention relates to computer data traversal search field, specifically a kind of practical, based on the tree traversal search method of MapReduce cloud computing model.
Background technology
Along with the fast development of infotech, the data volume exponentially that enterprise is faced increases, and how processing, process and represent so huge data, is the matter of utmost importance that current enterprise will be considered.Based on this, now propose a kind of tree traversal search method based on MapReduce cloud computing model.The method carries out traversal search by utilizing MapReduce cloud computing model to data cell, fast query can go out the information useful to us, thus obtain the result of our needs, ensure that accuracy and the promptness of search inquiry, increase work efficiency, bring more benefit to enterprise.
Summary of the invention
Technical assignment of the present invention is for above weak point, provide a kind of practical, based on the tree traversal search method of MapReduce cloud computing model.
Based on a tree traversal search method for MapReduce cloud computing model, its specific implementation process is:
The data source that one, will travel through is abstract is labelled tree model, according to the depth & wideth of tree, gets up with the formal notation of coordinate;
Two, data are searched for: traversal of tree is carried out to the data markers tree produced, Map-Reduce model is used to process task, resolve into a lot of subtask and carry out executed in parallel, each subtask result is identified by a flag, all subtasks result is done logical OR computing, obtains Search Results; Whether, according to the result returned in traversal of tree process, whether decision tree node is identical, and then had node to overlap.
The coordinate of described labelled tree model is expressed as (ai, bi), and wherein ai represents that data source A marks node coordinate in tree, and bi represents the flag node coordinate of data source B in tree.
The detailed process of described step 2 is:
1) first a subtask is traveled through:
Perform Map process: certain node in A is called each node of map function traversal B; Then certain node in B is called each node of map function traversal B;
Perform Reduce process: according to identical key value, whether search result by the rule judgment of arranging, then return results;
Call traversal program, complete traversal;
According to key value stipulations, judge whether ai and bi has coincidence;
Input MapReduce-Traversal program, transfers to cloud computing platform to perform;
In cloud platform, MapReduce-Traversal programme distribution is given available free server by controller;
Computer group first performs Map program to process the small block data cut, and then carries out sequence by reduce program and converges integration also;
2) ergodic process to other subtask of tree is completed according to above-mentioned steps by traversal function recurrence.
A kind of tree traversal search method based on MapReduce cloud computing model of the present invention, has the following advantages:
A kind of tree traversal search method based on MapReduce cloud computing model that the present invention proposes, utilize the advantage of this technology in parallel computation and distributed treatment, can mass data processing task on quick solution super large cluster, and easily extensible utilizes cloud platform and network technology, make algorithm strange land computing machine on the internet realizes task distribution process, the treatment effeciency of very big raising data, practical; Based on MapReduce programming model, the traversal search task of mass data can be processed; Operate on computer cluster, multitask distribution performs, and is independent of each other, greatly improves execution efficiency, is easy to promote.
Accompanying drawing explanation
Accompanying drawing 1 is labelled tree coordinated manner figure.
Accompanying drawing 2 is search procedure figure of the present invention.
Embodiment
Below in conjunction with the drawings and specific embodiments, the invention will be further described.
The invention provides a kind of tree traversal search method based on MapReduce cloud computing model, first according to different data cells, build Tree model; When carrying out search spread, coordinatograph conversion is carried out to the data in Tree; The Map-Reduce mechanism decomposition of each traversal search process performs.
As shown in accompanying drawing 1, Fig. 2, its specific implementation process is,
The data source that will travel through is abstract is labelled tree model, and according to the depth & wideth of tree, whether get up with the formal notation of coordinate, in traversal of tree process, according to the result returned, whether decision tree node is identical, and then had node to overlap.
As shown in Figure 1, wherein ai represents that data source A marks node coordinate in tree to labelled tree form, and bi represents the flag node coordinate of data source B in tree.
In above-mentioned steps, labelled tree represents the data by a certain unit or region, with its structure of tree model representation, according to the depth & wideth of tree, with the formal notation back end of coordinate; Ai: represent that dataA marks node coordinate in Tree; Bi: represent the flag node coordinate of dataB in Tree.
In the process that data are searched for, first the data tree produced is carried out or traversal of tree, Map-Reduce model is used to process task, resolve into a lot of subtask and carry out executed in parallel, each subtask result is identified by a flag, all subtasks result is done logical OR computing, obtains Search Results.
The algorithmic procedure of traversal search process specifically describes as follows:
Input: Two DATAs A,B
Output: Collision Detection Result
Bool MapReduce-Traversal(RA,RB)
{
To create two trees TreeA and TreeB; // set up the labelled tree TreeA of DATAs A and DATAs B, TreeB;
Set nodes TreeA, TreeB to QUEUESTa, QUEUESTb; // by TreeA, TreeB as root node, put into queue;
// perform Map process
{
Call Map (leafnode ai, B); In //A, certain node calls each node of map function traversal B
Call Map (leafnode ai, A); In //B, certain node calls each node of map function traversal B
}
// perform Reduce process
{
call Reduce(key, list<key, value>);
Whether // reduce (), according to identical key value, searches result by the rule judgment of arranging,
}
Return Result; // return results.
}
Map (leafnode m,N)
{
// remove all nodes in traversal search DATAs N with the node of in DATAs M;
// call traversal program Traversal (), complete traversal;
}
reduce (key, list<key, value>)
{
// according to key value stipulations value, judge whether ai and bi has coincidence;
}
Bool queryResult (DATAs A,B)
{
Input MapReduce-Traversal program; // input MapReduce-Traversal program, transfers to cloud computing platform to perform;
In // cloud platform, MapReduce-Traversal programme distribution is given available free server by master controller;
// computer group first performs Map program to process the small block data cut, and then carries out sequence by reduce program and converges integration also;
return Result;
}
Traversal (leafnode m,N)
{
stack<Tree> stack;
while(m || !stack.empty()){
……
……
}
// tree recursive traversal functional based method body;
}。
Method of the present invention uses in the collision detection research topic of national new and high technology virtual reality, for the mass data processing task of complex scene, this kind of method can make full use of multiprocessor or the calculating of uniprocessor multithreading, test figure shows the time can saving nearly 70%, substantially increases executing efficiency.
Above-mentioned embodiment is only concrete case of the present invention; scope of patent protection of the present invention includes but not limited to above-mentioned embodiment; claims of any a kind of tree traversal search method based on MapReduce cloud computing model according to the invention and the those of ordinary skill of any described technical field to its suitable change done or replacement, all should fall into scope of patent protection of the present invention.
Claims (3)
1. based on a tree traversal search method for MapReduce cloud computing model, it is characterized in that, its specific implementation process is:
The data source that one, will travel through is abstract is labelled tree model, according to the depth & wideth of tree, gets up with the formal notation of coordinate;
Two, data are searched for: traversal of tree is carried out to the data markers tree produced, Map-Reduce model is used to process task, resolve into a lot of subtask and carry out executed in parallel, each subtask result is identified by a flag, all subtasks result is done logical OR computing, obtains Search Results; Whether, according to the result returned in traversal of tree process, whether decision tree node is identical, and then had node to overlap.
2. a kind of tree traversal search method based on MapReduce cloud computing model according to claim 1, it is characterized in that, the coordinate of described labelled tree model is expressed as (ai, bi), wherein ai represents that data source A marks node coordinate in tree, and bi represents the flag node coordinate of data source B in tree.
3. a kind of tree traversal search method based on MapReduce cloud computing model according to claim 2, it is characterized in that, the detailed process of described step 2 is:
1) first a subtask is traveled through:
Perform Map process: certain node in A is called each node of map function traversal B; Then certain node in B is called each node of map function traversal B;
Perform Reduce process: according to identical key value, whether search result by the rule judgment of arranging, then return results;
Call traversal program, complete traversal;
According to key value stipulations, judge whether ai and bi has coincidence;
Input MapReduce-Traversal program, transfers to cloud computing platform to perform;
In cloud platform, MapReduce-Traversal programme distribution is given available free server by controller;
Computer group first performs Map program to process the small block data cut, and then carries out sequence by reduce program and converges integration also;
2) ergodic process to other subtask of tree is completed according to above-mentioned steps by traversal function recurrence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510153681.XA CN104778235A (en) | 2015-04-02 | 2015-04-02 | Tree traversal searching method based on MapReduce cloud calculation model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510153681.XA CN104778235A (en) | 2015-04-02 | 2015-04-02 | Tree traversal searching method based on MapReduce cloud calculation model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104778235A true CN104778235A (en) | 2015-07-15 |
Family
ID=53619699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510153681.XA Pending CN104778235A (en) | 2015-04-02 | 2015-04-02 | Tree traversal searching method based on MapReduce cloud calculation model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104778235A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106096159A (en) * | 2016-06-20 | 2016-11-09 | 华北电力大学（保定） | Distributed system behavior simulation under a kind of cloud platform analyzes the implementation method of system |
| CN106294686A (en) * | 2016-08-05 | 2017-01-04 | 董涛 | A kind of method quickly updating tree node position coordinates in mind map |
| CN110134340A (en) * | 2019-05-23 | 2019-08-16 | 苏州浪潮智能科技有限公司 | A kind of method, apparatus of metadata updates, equipment and storage medium |
| CN110377601A (en) * | 2019-06-27 | 2019-10-25 | 河南省交通规划设计研究院股份有限公司 | A kind of MapReduce calculating process optimization method based on B-tree data structure |
| CN113672924A (en) * | 2021-08-24 | 2021-11-19 | 李宇佳 | Data intrusion detection method and device of distributed cloud computing system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8190610B2 (en) * | 2006-10-05 | 2012-05-29 | Yahoo! Inc. | MapReduce for distributed database processing |
| CN103116625A (en) * | 2013-01-31 | 2013-05-22 | 重庆大学 | Volume radio direction finde (RDF) data distribution type query processing method based on Hadoop |
-
2015
- 2015-04-02 CN CN201510153681.XA patent/CN104778235A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8190610B2 (en) * | 2006-10-05 | 2012-05-29 | Yahoo! Inc. | MapReduce for distributed database processing |
| CN103116625A (en) * | 2013-01-31 | 2013-05-22 | 重庆大学 | Volume radio direction finde (RDF) data distribution type query processing method based on Hadoop |
Non-Patent Citations (1)
| Title |
|---|
| 赵伟等: "基于MapReduce云计算模型的碰撞检测算法", 《系统仿真技术及其应用》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106096159A (en) * | 2016-06-20 | 2016-11-09 | 华北电力大学（保定） | Distributed system behavior simulation under a kind of cloud platform analyzes the implementation method of system |
| CN106096159B (en) * | 2016-06-20 | 2019-05-21 | 华北电力大学（保定） | A kind of implementation method of distributed system behavior simulation analysis system under cloud platform |
| CN106294686A (en) * | 2016-08-05 | 2017-01-04 | 董涛 | A kind of method quickly updating tree node position coordinates in mind map |
| CN106294686B (en) * | 2016-08-05 | 2021-04-02 | 董涛 | Method for rapidly updating tree node position coordinates in thought guide graph applied to computer technical field |
| CN110134340A (en) * | 2019-05-23 | 2019-08-16 | 苏州浪潮智能科技有限公司 | A kind of method, apparatus of metadata updates, equipment and storage medium |
| CN110134340B (en) * | 2019-05-23 | 2020-03-06 | 苏州浪潮智能科技有限公司 | Method, device, equipment and storage medium for updating metadata |
| CN110377601A (en) * | 2019-06-27 | 2019-10-25 | 河南省交通规划设计研究院股份有限公司 | A kind of MapReduce calculating process optimization method based on B-tree data structure |
| CN110377601B (en) * | 2019-06-27 | 2022-04-12 | 河南省交通规划设计研究院股份有限公司 | B-tree data structure-based MapReduce calculation process optimization method |
| CN113672924A (en) * | 2021-08-24 | 2021-11-19 | 李宇佳 | Data intrusion detection method and device of distributed cloud computing system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9053067B2 (en) | Distributed data scalable adaptive map-reduce framework | |
| US10515118B2 (en) | Processing a data flow graph of a hybrid flow | |
| CN104778235A (en) | Tree traversal searching method based on MapReduce cloud calculation model | |
| Chen et al. | Distributed modeling in a MapReduce framework for data-driven traffic flow forecasting | |
| CN105930479A (en) | Data skew processing method and apparatus | |
| Lu et al. | Parallel secondo: A practical system for large-scale processing of moving objects | |
| Gunarathne et al. | Portable parallel programming on cloud and hpc: Scientific applications of twister4azure | |
| CN110689174A (en) | Personnel route planning method and device based on public transport | |
| CN102902739B (en) | Towards the workflow view building method in uncertain data source under cloud computing environment | |
| CN116108764B (en) | Optical intelligent optimization method, device, equipment and medium | |
| Kumar et al. | Graphsteal: Dynamic re-partitioning for efficient graph processing in heterogeneous clusters | |
| Ying et al. | Towards fault tolerance optimization based on checkpoints of in-memory framework spark | |
| CN111444007A (en) | Remote sensing big data automatic processing method based on cloud computing | |
| CN115857918A (en) | Data processing method and device, electronic equipment and storage medium | |
| Liu et al. | BSPCloud: A hybrid distributed-memory and shared-memory programming model | |
| Perwej et al. | An extensive investigate the mapreduce technology | |
| Zhang et al. | Parallel option pricing with BSDEs method on MapReduce | |
| Jin et al. | A data-locality-aware task scheduler for distributed social graph queries | |
| Jamadagni et al. | GoDB: From batch processing to distributed querying over property graphs | |
| Aridhi et al. | Shortest path resolution using hadoop | |
| Hamad et al. | Improving performance of distributed data mining | |
| Singhal et al. | Modified mapreduce framework for enhancing performance of graph based algorithms by fast convergence in distributed environment | |
| Fontolan | Modularity based community detection on the GPU | |
| CN104991912A (en) | Large scale map data clustering algorithm based on MapReduce architecture | |
| Zhang et al. | Large-scale Data Mining Method based on Clustering Algorithm Combined with MAPREDUCE |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150715 |
|
| WD01 | Invention patent application deemed withdrawn after publication |