CN110347886B - Distributed parent-child relationship element searching method - Google Patents
Distributed parent-child relationship element searching method Download PDFInfo
- Publication number
- CN110347886B CN110347886B CN201910543604.3A CN201910543604A CN110347886B CN 110347886 B CN110347886 B CN 110347886B CN 201910543604 A CN201910543604 A CN 201910543604A CN 110347886 B CN110347886 B CN 110347886B
- Authority
- CN
- China
- Prior art keywords
- parent
- elements
- child relationship
- distributed
- distributed cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9027—Trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Multi Processors (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for searching distributed parent-child relationship elements, which comprises the following steps: establishing a distributed cache redis; splitting an element set into a plurality of element subsets, merging and sending the element subsets to different computers; the computer calculates the parent element of each element in the received element subset and stores the element and the corresponding parent element into a distributed cache redis; searching out elements meeting parent-child relation functions from the received element subset through a distributed cache redis; and summarizing the search results of the plurality of computers for comprehensive judgment. The provided method for searching the elements in the distributed parent-child relationship divides the element set into a plurality of element subsets which are sent to different computers for processing, stores the operation results of the computers by using the distributed cache redis, and greatly reduces the time for searching the elements in the element set which meet the parent-child relationship according to the key-value mapping characteristic of the distributed cache redis.
Description
Technical Field
The invention relates to a distributed parent-child relationship element searching method.
Background
In a set of elements, for example, for a set of elements { a, B, C }, the elements in the set of elements satisfy f (a) ═ B, and f (B) ═ C, we consider that each element in the set of elements satisfies the parent-child relationship function f (x), and the element A, B, C satisfies the parent-child relationship. Where element A is the parent of element B, element B is the child of element A, element B is the parent of element C, and element C is the child of element B. And the parent element of element a is not included in the set of elements, we call element a the root, and the child element of element C is not included in the set of elements, then element C is the child node.
In practical applications, it is often necessary to verify whether each element in an element set satisfies a parent-child relationship, and the existing method is to determine each element as follows: and verifying whether the child element or the parent element of each element exists in the element set, and if the parent element or the child element of the element exists in the element set, the element meets the parent-child relationship.
In the existing method, for verification of each element in the element set, the element needs to be substituted into a parent-child relationship function f (x) to calculate to obtain a child element, and the child element is compared with the remaining elements in the element set to determine whether the remaining elements include elements equal to the child element.
Specifically, taking the element set { a, B, C } as an example, if it is desired to verify whether the elements in the element set satisfy the parent-child relationship function f (x), the following steps are required:
and (3) verifying the element A: verifying whether f (a) equals B and f (a) equals B, requiring at most two loop calculations, verifying element B: verifying whether f (b) equals a and f (b) equals C, requiring at most two loop calculations, verifying element C: verifying whether f (C) is equal to A and f (C) is equal to B, circularly calculating twice at most, judging whether the element A, the element B and the element C meet a parent-child relationship function f (x) through the steps, verifying an element set with the element number of 3, and performing 6 comparison search operations with the worst possibility, wherein when the element number is n, the worst possibility is that n (n-1) comparison search operations are required, and assuming that the element number in the element set is large, the operation amount is increased rapidly, the calculation resources of a computer are greatly consumed, and meanwhile, the calculation time is very long.
Disclosure of Invention
In order to solve the defects of the prior art, the invention provides a distributed parent-child relationship element searching method which can solve the problems.
In order to achieve the above object, the present invention adopts the following technical solutions:
a distributed parent-child relationship element searching method is used for searching elements meeting a certain parent-child relationship function from an element set, and comprises the following steps:
establishing a distributed cache redis;
splitting the set of elements into a plurality of subsets of elements and sending the subsets of elements to different computers;
each computer computing its received parent of each of the elements in the subset of elements;
each computer stores the element and the parent element corresponding to the element into the distributed cache redis;
each computer finds out the elements meeting the parent-child relationship function from the element subset received by the computer through the distributed cache redis;
summarizing the search results of a plurality of computers and comprehensively judging the elements meeting the parent-child relationship function in the element set.
Further, the element set is divided equally into a plurality of element subsets.
Further, the specific method for each computer to calculate the parent element of each element in the element subset received by the computer is as follows:
calculating through the parent-child relationship function to obtain an inverse function of the parent-child relationship function as a parent-child relationship inverse function;
and substituting the element into the parent-child relationship inverse function obtained through calculation to obtain the parent element corresponding to the element.
Further, a specific method for each computer to store the element and the parent element corresponding to the element into the distributed cache redis is as follows:
and taking the element as a key value, taking the parent element corresponding to the element as a value, and storing a key-value pair formed by the element and the parent element into the distributed cache redis.
Further, a specific method for each computer to find out the elements satisfying the parent-child relationship function from the element subset received by the computer through the distributed cache redis is as follows:
taking the element as a key value to obtain a value corresponding to the element from the distributed cache redis;
and taking the obtained value as a new key value to obtain the value corresponding to the new key value from the distributed cache redis.
Further, when the obtained value can be obtained as a new key value from the distributed cache redis, the parent element of the element is included in the element set.
Further, when the obtained value cannot be obtained from the distributed cache redis as a new key value, the parent element of the element is not included in the element set.
Further, the search results of a plurality of computers are collected and comprehensively judged to obtain the elements of which the child elements or parent elements exist in the element set, and the elements are extracted to form a parent-child relationship element set.
Further, comprehensively judging the search results of a plurality of computers to obtain the elements of which the child elements and the parent elements are not in the element set, and extracting the elements to form a parent-child relationship element set.
The distributed parent-child relationship element searching method has the advantages that one element set is divided into a plurality of element child sets to be sent to different computers for distributed operation, meanwhile, the operation results of the computers are stored by the aid of the distributed cache redis, and time for searching elements meeting parent-child relationship in the element set is greatly shortened according to key-value mapping characteristics of the distributed cache redis.
Drawings
FIG. 1 is a flow chart of a distributed parent-child relationship element lookup method of the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and the embodiments.
Fig. 1 is a flowchart of a distributed parent-child relationship element searching method according to the present invention, which is used for searching an element satisfying a parent-child relationship function f (x) from an element set, and includes the following steps: s1, establishing a distributed cache redis. S2, splitting the element set into a plurality of element subsets and sending the element subsets to different computers. S3, each computer computes the parent of each element in its received subset of elements. S4, each computer stores the element and its corresponding parent element in a distributed cache redis. S5, each computer finds out the elements satisfying the parent-child relationship function f (x) from the element subset received by the computer through the distributed cache redis. And S6, summarizing the search results of the plurality of computers, and comprehensively judging the elements meeting the parent-child relationship function f (x) in the element set.
For S1, a distributed cache redis is established.
Firstly, a distributed cache redis capable of meeting storage requirements is established according to the number of elements in an element set needing to be searched.
For S2, the set of elements is split into a plurality of subsets of elements and the subsets of elements are sent to different computers.
And splitting the element set needing to be searched into a plurality of subsets. Preferably, the element set is split in a uniform manner, and the specific method comprises the following steps: the number of the elements in the element set is obtained, and the elements in the element set are equally divided according to the number of the element subsets needing to be divided, it can be understood that the number of the elements in the element set is not necessarily evenly divisible by the number of the element subsets, and the meaning of equally dividing at this time means that the difference of the number of the elements in each element subset is not more than 1. For example, assuming that the number of elements in the element set is 1002, the number of element subsets to be split is 100, the number of elements in the split element subsets is 101,100,100,100,100,100,100,100,100,101, and so on. And after the splitting is finished, each element subset is sent to different computers for distributed operation.
For S3, each computer computes the parent element of each element in its received subset of elements.
Specifically, the specific method for each computer to compute the parent element of each element in the element subset it receives is as follows: calculating by using a parent-child relationship function f (x) to obtain an inverse function of the parent-child relationship function f (x) as a parent-child relationship inverse function f (x)-1Substituting the elements into the inverse function f (x) of the parent-child relationship obtained by calculation-1The parent element corresponding to the element is obtained.
For S4, each computer stores the element and its corresponding parent element in a distributed cache redis.
Specifically, the element is used as a key value, a parent element corresponding to the element is used as a value, and the element and the parent element are combined into a key-value pair to be stored in the distributed cache redis. That is, the element and the parent element are combined into a key-value pair by a put method and stored in the distributed cache redis for subsequent lookup.
For S5, each computer finds the elements satisfying the parent-child relationship function f (x) from the subset of elements it receives through the distributed cache redis.
Specifically, each computer validates each element in the subset of elements that need validation by: taking the element a as an example, the element a is used as a key value to obtain a value corresponding to the element a in the distributed cache redis, that is, by a get (x) method, the element a is substituted into get (x) to perform operation and search on the element a from the distributed cache redis to obtain a value corresponding to the element a, and the obtained value is used as a new key value and then is obtained from the distributed cache redis by a get (x) method. The obtained value is assumed to be a new key value, and the value corresponding to the value is obtained from the distributed cache redis through a get (x) method, which indicates that the parent element of the element a in the element child set exists in the element set, that is, the element a satisfies the parent-child relationship. On the contrary, when the obtained value is taken as a new key value and the corresponding value cannot be obtained from the distributed cache redis by the get (x) method, it indicates that the parent element of the element a in the element child set does not exist in the element set, and it can be understood that the parent element of the element a does not exist in the element set and it cannot be directly obtained that the element a does not satisfy the parent-child relationship because the element a may also be a root, and it indicates that the element a does not satisfy the parent-child relationship only when it is determined that the element a is not the parent element of other elements in the element set after all verification steps, and on the contrary, if the element a is the parent element of other elements in the element set, the element a also satisfies the parent-child relationship.
Each computer may not complete the elements obtained by searching the element subset received by the computer, and it is assumed that one element a in the element subset 1 received by the computer 1 is a root, but the child element B of the element a is in the element subset 2 received by the computer 2, because the computations of each computer are mutually independent, when the computer 1 performs search verification on the element subset 1, because the search result of the computer 2 cannot be known, the judgment result of the computer 1 considers that the element a does not satisfy the parent-child relationship, which obviously violates the actual situation.
And S6, summarizing the search results of the plurality of computers, and comprehensively judging the elements meeting the parent-child relationship function f (x) in the element set.
In step S5, after each computer performs search verification on the received element subset, it is further required to summarize the search results of each computer, and then perform comprehensive judgment according to the summarized results to obtain all elements that satisfy the parent-child relationship. According to the assumed situation, since the search verification results of each computer are finally summarized, although the computer 1 shows that the element a has neither parent element nor child element as the search verification result of the element a, the computer 2 shows that the parent element of the element B is the element a as the search verification result of the element subset 2, and therefore, after the search verification results of a plurality of computers are combined for comprehensive judgment, the error result of excluding the element a can be avoided.
By the parent-child relationship element searching method, the element set is assumed to contain n elements, and each element is substituted into a parent-child relationship inverse function f (x)-1If the parent element is obtained by calculation and the mapping relation is stored in the distributed cache redis, the step needing operation is 2n, and when the element is searched, each element only needs to be searched twice by get (x) method from the distributed cache redis, the searching frequency is 2n, the total operation step is 4n, and meanwhile, the time of final operation is greatly shortened due to the adoption of the distributed operation. Compared with the worst possible operation step n (n-1) times in the prior art, the method for finding the parent-child relationship elements has the advantage that the operation amount is remarkably reduced.
It can be understood that, dividing the element set and sending the divided element subsets to different computers also generates some operations, but when the number of the element set is very large, the operation amount of dividing the element set and sending the element subsets is almost negligible compared with the whole operation process.
Optionally, the search results of the multiple computers are collected and comprehensively judged to obtain elements of which the child elements or parent elements exist in the element set, and the elements are extracted to form a parent-child relationship element set. And comprehensively judging the search results of the plurality of computers to obtain elements of which the child elements and the parent elements do not exist in the element set, and extracting the elements to form a parent-child relationship element set.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It should be understood by those skilled in the art that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by using equivalent alternatives or equivalent variations fall within the scope of the present invention.
Claims (5)
1. A distributed parent-child relationship element searching method is used for searching elements meeting a certain parent-child relationship function from an element set, and is characterized by comprising the following steps:
establishing a distributed cache redis;
splitting the set of elements into a plurality of subsets of elements and sending the subsets of elements to different computers;
each computer computing its received parent of each of the elements in the subset of elements;
each computer stores the element and the parent element corresponding to the element into the distributed cache redis;
each computer finds out the elements meeting the parent-child relationship function from the element subset received by the computer through the distributed cache redis;
summarizing search results of a plurality of computers and comprehensively judging the elements meeting the parent-child relationship function in the element set;
the specific method for each computer to store the element and the parent element corresponding to the element into the distributed cache redis is as follows:
taking the element as a key value and the parent element corresponding to the element as a value, and storing a key-value pair formed by the element and the parent element into the distributed cache redis;
the specific method for each computer to find out the elements meeting the parent-child relationship function from the element subset received by the computer through the distributed cache redis is as follows:
taking the element as a key value to obtain a value corresponding to the element from the distributed cache redis;
taking the obtained value as a new key value to obtain a corresponding value from the distributed cache redis;
when the obtained value can be obtained from the distributed cache redis as a new key value, the parent element of the element is contained in the element set;
when the obtained value cannot be obtained from the distributed cache redis as a new key value, the parent element of the element is not included in the element set.
2. The distributed parent-child relationship element lookup method according to claim 1,
and equally dividing the element set into a plurality of element subsets.
3. The distributed parent-child relationship element lookup method according to claim 1,
the specific method for each computer to calculate the parent element of each element in the element subset received by the computer is as follows:
calculating through the parent-child relationship function to obtain an inverse function of the parent-child relationship function as a parent-child relationship inverse function;
and substituting the element into the parent-child relationship inverse function obtained through calculation to obtain the parent element corresponding to the element.
4. The distributed parent-child relationship element lookup method according to claim 1,
and summarizing the search results of a plurality of computers and comprehensively judging to obtain the elements of which the child elements or parent elements exist in the element set, and extracting the elements to form a parent-child relationship element set.
5. The distributed parent-child relationship element lookup method according to claim 4,
and comprehensively judging the search results of a plurality of computers to obtain the elements of which the child elements and the parent elements are not in the element set, and extracting the elements to form a parent-child relationship element set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910543604.3A CN110347886B (en) | 2019-06-21 | 2019-06-21 | Distributed parent-child relationship element searching method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910543604.3A CN110347886B (en) | 2019-06-21 | 2019-06-21 | Distributed parent-child relationship element searching method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110347886A CN110347886A (en) | 2019-10-18 |
| CN110347886B true CN110347886B (en) | 2022-03-25 |
Family
ID=68182680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910543604.3A Active CN110347886B (en) | 2019-06-21 | 2019-06-21 | Distributed parent-child relationship element searching method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347886B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110351079B (en) * | 2019-07-09 | 2022-03-01 | 政采云有限公司 | Three-party decryption method based on parent-child relationship verification |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104156380A (en) * | 2014-03-04 | 2014-11-19 | 深圳信息职业技术学院 | Distributed memory Hash indexing method and system |
| CN104834751A (en) * | 2015-05-28 | 2015-08-12 | 成都艺辰德迅科技有限公司 | Data analysis method based on Internet of things |
| CN104834557A (en) * | 2015-05-18 | 2015-08-12 | 成都博元科技有限公司 | Data analysis method based on Hadoop |
| CN109726252A (en) * | 2018-12-29 | 2019-05-07 | 税友软件集团股份有限公司 | A kind of data managing method and relevant apparatus of distributed data collection |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9082127B2 (en) * | 2010-03-31 | 2015-07-14 | Cloudera, Inc. | Collecting and aggregating datasets for analysis |
-
2019
- 2019-06-21 CN CN201910543604.3A patent/CN110347886B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104156380A (en) * | 2014-03-04 | 2014-11-19 | 深圳信息职业技术学院 | Distributed memory Hash indexing method and system |
| CN104834557A (en) * | 2015-05-18 | 2015-08-12 | 成都博元科技有限公司 | Data analysis method based on Hadoop |
| CN104834751A (en) * | 2015-05-28 | 2015-08-12 | 成都艺辰德迅科技有限公司 | Data analysis method based on Internet of things |
| CN109726252A (en) * | 2018-12-29 | 2019-05-07 | 税友软件集团股份有限公司 | A kind of data managing method and relevant apparatus of distributed data collection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110347886A (en) | 2019-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107784044B (en) | Table data query method and device | |
| US9268796B2 (en) | Systems and methods for quantile estimation in a distributed data system | |
| EP3432157B1 (en) | Data table joining mode processing method and apparatus | |
| CN110147407B (en) | Data processing method and device and database management server | |
| CN111512283B (en) | Radix estimation in a database | |
| CN105550225A (en) | Index construction method and query method and apparatus | |
| CN105989015B (en) | Database capacity expansion method and device and method and device for accessing database | |
| US10466936B2 (en) | Scalable, multi-dimensional search for optimal configuration | |
| CN107016115B (en) | Data export method and device, computer readable storage medium and electronic equipment | |
| CN109753573B (en) | Processing method and device for constructing preset model based on graph database | |
| CN108304460B (en) | Improved database positioning method and system | |
| CN110347886B (en) | Distributed parent-child relationship element searching method | |
| CN112699134A (en) | Distributed graph database storage and query method based on graph subdivision | |
| CN115880508A (en) | Image data processing method, device, equipment and storage medium | |
| CN112559547B (en) | Method and device for determining consistency among multiple storage object copies | |
| CN103678545A (en) | Network resource clustering method and device | |
| CN108717444A (en) | A kind of big data clustering method and device based on distributed frame | |
| CN106294096B (en) | Information processing method and device | |
| CN114547384A (en) | Resource object processing method and device and computer equipment | |
| CN105468603A (en) | Data selection method and apparatus | |
| CN113282616A (en) | Incremental time sequence data conflict detection method and device and storage medium | |
| WO2021012211A1 (en) | Method and apparatus for establishing index for data | |
| CN110377642B (en) | Device for rapidly acquiring ordered sequence data | |
| CN112148706A (en) | Variable grouping method and related equipment | |
| CN111639099A (en) | Full-text indexing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |