CN112685511A - Method and device for commercial intelligent warehouse high-performance routing - Google Patents
Method and device for commercial intelligent warehouse high-performance routing Download PDFInfo
- Publication number
- CN112685511A CN112685511A CN202011637690.3A CN202011637690A CN112685511A CN 112685511 A CN112685511 A CN 112685511A CN 202011637690 A CN202011637690 A CN 202011637690A CN 112685511 A CN112685511 A CN 112685511A
- Authority
- CN
- China
- Prior art keywords
- data
- bin
- role
- information
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000004458 analytical method Methods 0.000 claims abstract description 40
- 238000012216 screening Methods 0.000 claims description 8
- 101000822425 Arthrobacter sp. (strain KUJ 8602) Guanidinobutyrase Proteins 0.000 claims description 7
- 238000012550 audit Methods 0.000 claims description 7
- 238000007689 inspection Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000013507 mapping Methods 0.000 description 3
- RWSOTUBLDIXVET-UHFFFAOYSA-N Dihydrogen sulfide Chemical compound S RWSOTUBLDIXVET-UHFFFAOYSA-N 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Abstract
The embodiment of the application discloses a business intelligent warehouse high-performance routing method, which comprises the steps of receiving a business intelligent analysis request of a business system, wherein the request comprises data information, data types and role information, determining a corresponding warehouse according to the role information, and obtaining a role warehouse set, so that the warehouse which can be accessed by a user role in the request is known. And determining corresponding bins according to the data type and the data information to obtain a data bin set, so that the bins corresponding to the data type and the data information in the request can be known. And obtaining a final number bin set by taking the intersection of the role number bin set and the data number bin set, namely the number bin set which can be accessed by the user role in the request and has the data information and the data type in the request, so that the request can be disassembled into the number bins according to the final number bin set to be executed in parallel. Therefore, high-performance routing of the commercial intelligent data warehouse is realized, data information, data types and role information do not need to traverse the matched data warehouse one by one, and the execution efficiency is higher.
Description
Technical Field
The application relates to the field of finance, in particular to a method and a device for commercial intelligent multi-warehouse high-performance routing.
Background
At present, a commercial intelligent integrated warehouse is composed of a plurality of heterogeneous warehouses, and data information, data types and authority rules contained in each warehouse have large difference, so that the warehouse is required to be matched one by traversing the data information, the data types and the role information before task distribution, and the efficiency is low.
How to improve the matching efficiency and shorten the matching time so as to improve the overall performance of the system is an urgent technical problem to be solved in the field.
Disclosure of Invention
In order to solve the technical problems, the application provides a method and a device for high-performance routing of a commercial intelligent warehouse, and by constructing various warehouse collections, the warehouse which finally accords with data information, data types and role information in a commercial intelligent request can be quickly obtained, the data information, the data types and the role information do not need to be traversed and matched one by one, and the execution efficiency is higher.
The embodiment of the application discloses the following technical scheme:
in a first aspect, an embodiment of the present application provides a method for high-performance routing of a business intelligence warehouse, including:
receiving a business intelligent analysis request of a business system, wherein the request comprises data information, data types and role information;
determining a corresponding number bin according to the role information to obtain a role number bin set;
determining a corresponding data bin according to the data type and the data information to obtain a data bin set;
obtaining an intersection of the role number bin set and the data number bin set to obtain a final number bin set;
and sending the final silo set to a task distribution unit so that the task distribution unit disassembles the request to each silo in the final silo set for parallel execution.
Optionally, the method further includes:
storing results obtained by parallel execution of the several bins;
and merging and returning the stored results.
Optionally, the number of bins includes:
a data bin of a Presto Preleistost high-performance engine, a solid-state hard disk high-performance version GBase structured data bin, a Kylin data bin and an Orale oracle Oracle Highun data bin are built in a Hadoop Hadupu unstructured universal data bin and Hadoop data environment.
Optionally, the data types include:
supervision and inspection type data, financial audit type data, customer screening type data, product marketing type data, operation index type data and/or analysis report type data.
Optionally, the method further includes:
and if the parallel execution is abnormally interrupted, automatically executing again.
In a second aspect, an embodiment of the present application provides an apparatus for commercial intelligent multi-bin high-performance routing, including:
the business system business intelligent analysis system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving business system business intelligent analysis requests which comprise data information, data types and role information;
the role information acquisition unit is used for acquiring role information of a role in the role;
the data counting bin set determining unit is used for determining a corresponding counting bin according to the data type and the data information to obtain a data counting bin set;
a final number bin set determining unit, configured to obtain an intersection of the role number bin set and the data number bin set to obtain a final number bin set;
and the sending unit is used for sending the final number bin set to the task distribution unit so that the task distribution unit disassembles the request to each number bin in the final number bin set to execute in parallel.
Optionally, the apparatus further comprises:
the storage unit is used for storing results obtained by parallel execution of the number bins;
and the return unit is used for merging and returning the stored results.
Optionally, the number of bins includes:
a data bin of a Presto Preleistost high-performance engine, a solid-state hard disk high-performance version GBase structured data bin, a Kylin data bin and an Orale oracle Oracle Highun data bin are built in a Hadoop Hadupu unstructured universal data bin and Hadoop data environment.
Optionally, the data types include:
supervision and inspection type data, financial audit type data, customer screening type data, product marketing type data, operation index type data and/or analysis report type data.
Optionally, the apparatus further comprises:
and the re-execution unit is used for automatically re-executing when the parallel execution is abnormally interrupted.
According to the technical scheme, compared with the prior art, the embodiment of the application has the following advantages:
according to the technical scheme, the business intelligent warehouse high-performance routing method and device can receive business intelligent analysis requests of a business system, the requests comprise data information, data types and role information, corresponding warehouse is determined according to the role information, and a role warehouse set is obtained, so that the warehouse which can be accessed by the user role in the request correspondingly can be known. And determining corresponding bins according to the data type and the data information to obtain a data bin set, so that the bins corresponding to the data type and the data information in the request can be known. And obtaining a final number bin set by taking the intersection of the role number bin set and the data number bin set, namely the number bin set which can be accessed by the user role in the request and has the data information and the data type in the request, so that the request can be disassembled into the number bins according to the final number bin set to be executed in parallel. Therefore, high-performance routing of the commercial intelligent data warehouse is realized, data information, data types and role information do not need to traverse the matched data warehouse one by one, and the execution efficiency is higher.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a method for intelligent commercial multi-warehouse high-performance routing according to an embodiment of the present application;
FIG. 2 is a schematic diagram of data classification and corresponding bin selection according to an embodiment of the present disclosure;
fig. 3 is a model diagram of a routing configuration based on a routing configuration template according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating a high performance routing operation according to an embodiment of the present application;
FIG. 5 is a schematic view of a full flow chart of a combination of various counting chambers provided in the embodiments of the present application;
fig. 6 is a schematic diagram of an apparatus for commercial intelligent multi-warehouse high-performance routing according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, a commercial intelligent integrated warehouse is composed of a plurality of heterogeneous warehouses, and data information, data types and authority rules contained in each warehouse have large difference, so that the warehouse is required to be matched one by traversing the data information, the data types and the role information before task distribution, and the efficiency is low.
How to improve the matching efficiency and shorten the matching time so as to improve the overall performance of the system is an urgent technical problem to be solved in the field.
In order to solve the foregoing technical problem, embodiments of the present application provide a method and an apparatus for high-performance routing of business intelligent bins, which can receive a business intelligent analysis request of a business system, where the request includes data information, a data type, and role information, and determine a corresponding bin according to the role information to obtain a role bin set, so as to know that a user role in the request corresponds to an accessible bin. And determining corresponding bins according to the data type and the data information to obtain a data bin set, so that the bins corresponding to the data type and the data information in the request can be known. And obtaining a final number bin set by taking the intersection of the role number bin set and the data number bin set, namely the number bin set which can be accessed by the user role in the request and has the data information and the data type in the request, so that the request can be disassembled into the number bins according to the final number bin set to be executed in parallel. Therefore, high-performance routing of the commercial intelligent data warehouse is realized, data information, data types and role information do not need to traverse the matched data warehouse one by one, and the execution efficiency is higher.
Various non-limiting embodiments of the present invention are described in detail below with reference to the accompanying drawings.
Exemplary method
Referring to fig. 1, which is a flowchart of a method for high-performance routing of a business intelligence warehouse according to an embodiment of the present application, the method may include:
s101, receiving a business system business intelligent analysis request, wherein the request comprises data information, data types and role information.
In the embodiment of the present application, referring to fig. 2, based on the business intelligence analysis scenario, the data types may be cold detail data, hot detail data, and summary data.
The cold detail data is data facing to historical data retrieval and analysis scenes such as supervision and inspection, financial audit and the like. The data has large total amount, few updates and low use frequency, so the commercial intelligent analysis has low complexity, and is suitable for adopting a general type data warehouse with low cost, large space and relatively general response speed. According to the embodiment of the application, the fact that the actual supervision audit usually contains various data requirements such as running water and documents is considered, and the composition structure of the data is relatively complex, so that a Hadoop unstructured universal data warehouse scheme is adopted.
The hot detail data is data oriented to client screening, product marketing and other inventory screening and ad hoc query scenes. The total amount of the partial data is relatively small, the updating is frequent, the use frequency is high, so that the complexity of the commercial intelligent analysis is high, and the method is suitable for adopting a high-performance special warehouse with quick response and small scale. The embodiment of the application adopts a double-bin scheme based on the data application type: for the list screening application, as the data volume is usually large and the query condition is relatively fixed, a warehouse counting scheme of a Presto high-performance engine is established by adopting a Hadoop data environment; for the ad hoc query application, because the query condition is very flexible and the requirement on data response time is higher, the logic condition is complex, and a GBase (SSD solid state disk high performance version) structured high-performance data warehouse is adopted.
The summarized data is data oriented to summary statistical scenes such as business indexes and analysis reports. The partial data is obtained by a large amount of calculation such as accumulation, grouping and the like of detail data, and is difficult to complete in real time during analysis, so a special data warehouse scheme for data pre-calculation is needed to be adopted, the calculation of common data is carried out in advance, and the data calculation amount during analysis is reduced. The embodiment of the application adopts a double-bin scheme based on the data application type: for the application of flexible data grouping conditions such as analysis reports and the like, a Kylin scheme pre-summarized according to dimensionality is adopted, and data summarization calculation is performed in advance according to common groups at non-service time such as night every day and the like; for the application of fixed grouping conditions such as index classes, an Oralce scheme is adopted, and after index data preprocessing is finished, the index data is stored in a line type for direct query in subsequent analysis.
It should be noted that Business Intelligence (BI), also called Business Intelligence or Business Intelligence, refers to a method for enhancing Business decision level by using modern data warehouse, online analysis, data mining and presentation technologies to perform Business value exploration.
A bin, known collectively as a Data WareHouse (Data WareHouse), is a theme-oriented, integrated, time-varying collection of Data, but the information itself is relatively stable.
Hadoop, an open-source software framework that supports data intensive distributed applications and is promulgated under the Apache 2.0 license agreement. Supporting applications running on large clusters built of commodity hardware.
Presto, an open-source distributed SQL query engine, is suitable for interactive analytical query, the data volume supports GB to PB bytes, and a Presto query can merge data of multiple data sources, and can span the entire organization for analysis.
GBase is a column type storage database supporting rapid analysis of mass data, and supports analysis characteristics such as transparent adaptive compression, intelligent indexing and bidirectional parallel.
Kylin, an open-source, distributed analytical data warehouse, provides SQL query interface and multidimensional analysis capability on Hadoop/Spark to support super-large scale data, and can query huge tables in sub-second.
Oracle, a set of software products developed by oracle corporation with a distributed database as the core.
The data information may include specific data such as loan data, deposit data, and the like. The role information includes the identity of the user, and in a banking system, the role information can be a teller, a customer manager, a financial manager, a supervisor, a leader, and the like.
In addition, the business system business intelligence analysis request can be received by the specific year and the specific client, for example, the business intelligence analysis request can be specifically used by a bank manager to view loan data of 2018 years of A client stored with cold detail data.
And S102, determining a corresponding number bin according to the role information to obtain a role number bin set.
In the embodiment of the application, the number bins corresponding to the role information can be configured according to a standard routing template in the non-service time of the bank system, the routing configuration template is shown in fig. 3, the number bins corresponding to the role information and having access authority can be obtained according to a configured routing number bin set, the time for the role information and having access authority to access the corresponding number bins can also be known, different times can be allocated according to different role information, for example, the access time setting of the role of the captain can be effective for a long time, and the access time setting of the role of the teller needing to transact the corresponding service can be effective for the current day of transacting the service.
The verification information and the corresponding multi-bin routing configuration information are stored according to a KEY (KEY) -VALUE (VALUE) type mapping relation, wherein the verification information is used as a KEY part, the corresponding multi-bin routing configuration information is used as a VALUE part, and each piece of KEY-VALUE data represents an available and routable multi-bin resource corresponding to the check condition.
The key-value type mapping of the verification information and the corresponding routing configuration information of the plurality of bins adopts a standardized routing configuration template so as to realize the unification of rules of partial data of the value and facilitate the comparison of the partial data of the value. If the data of the values corresponding to the two keys are the same, the routing rule corresponding to the value can simultaneously meet the verification conditions corresponding to the two keys, and the corresponding multi-bin resources are available and routable.
For example, the role information a and the role information b as two "keys" both correspond to the same "value" -a number bin, which indicates that both a and b have the right to access the a number bin.
Similarly, if the data of the "values" corresponding to the two "keys" are different, it indicates that the routing rule corresponding to the "value" cannot simultaneously satisfy the verification conditions corresponding to the two "keys", and the corresponding multi-bin resources are unavailable and non-routable.
Meanwhile, if the keys corresponding to the two values are the same, the verification condition corresponding to the key can simultaneously satisfy the routing rules corresponding to the two values.
For example, the "key" corresponding to the two "values" of the bins a and B can be the role information a, which indicates that a has the right to access both the bin a and the bin B.
Similarly, if the "key" corresponding to the two "values" are different, it indicates that the verification condition corresponding to the "key" cannot simultaneously satisfy the routing rules corresponding to the two "values".
S103, determining a corresponding data bin according to the data type and the data information to obtain a data bin set.
In the embodiment of the application, the data type and the data information corresponding to the data bin can be configured according to the standard routing template at the non-service time of the bank system, the routing configuration template refers to fig. 3, and the data bin with the data type and the data information can be obtained according to the configured data bin set. Furthermore, it is also known what type of operations the bin can perform, such as summing of details, extreming of details, etc., and what data is open to the outside in the bin.
For example, bin a opens the coldness detail loan data out, and bin a may sum the coldness detail loan data of the first customer stored therein.
And, when configuring the data type and the corresponding bin of the data information, it is also adopted to store the verification information and the corresponding bin routing configuration information according to a "KEY (KEY) -VALUE (VALUE)" type mapping relationship, where the verification information is used as a "KEY" part, the corresponding bin routing configuration information is used as a "VALUE" part, and each piece of "KEY-VALUE" data represents the available and routable bin resource corresponding to the check condition.
At this time, the data type and data information are referred to as corresponding "keys", and the bins are referred to as corresponding "values".
And S104, taking the intersection of the role number bin set and the data number bin set to obtain a final number bin set.
In the embodiment of the present application, referring to fig. 4, a role bin set refers to a bin corresponding to role information, that is, a bin to which the role information has an access right. A data bin set refers to a bin having the data type and data information contained in a user access request. And performing set operation in a cache unit Redis, wherein the cache unit adopts a high-performance cache Redis system, namely a remote dictionary service system. The intersection of the two refers to the number bin which has the corresponding data type and data information and can be accessed by the role information, and the number bin is used as a final number bin set.
For example, the set of character bins is { a1, a2, A3, a4, a5}, the set of data bins is { a1, a2, A3, A6, a7, A8}, and the final set of intersection bins of the two is { a1, a2, A3 }.
S105, sending the final bin set to a task distribution unit, so that the task distribution unit disassembles the request to each bin in the final bin set to execute in parallel.
In this embodiment, referring to fig. 5, the final bin set may be sent to the task distribution unit, so that the task distribution unit may disassemble the data analysis request into the analysis tasks corresponding to the bins according to the information in the final bin set, and submit the analysis tasks to the bins for parallel execution.
And after the parallel execution of the multiple bins is finished, the analysis results can be exported to a public cache unit for storage, the problem of inconsistent task completion among the multiple bins is solved, then the analysis results of the multiple bins are combined into a final analysis result, and the final analysis result is delivered to each application through a unified interface so as to be called.
The embodiment of the application provides a high-performance routing method for business intelligent warehouse, which can receive business intelligent analysis requests of a business system, wherein the requests comprise data information, data types and role information, and corresponding warehouse is determined according to the role information to obtain a role warehouse set, so that the warehouse which can be accessed by a user role in the request is known. And determining corresponding bins according to the data type and the data information to obtain a data bin set, so that the bins corresponding to the data type and the data information in the request can be known. And obtaining a final number bin set by taking the intersection of the role number bin set and the data number bin set, namely the number bin set which can be accessed by the user role in the request and has the data information and the data type in the request, so that the request can be disassembled into the number bins according to the final number bin set to be executed in parallel. Therefore, high-performance routing of the commercial intelligent data warehouse is realized, data information, data types and role information do not need to traverse the matched data warehouse one by one, and the execution efficiency is higher.
Exemplary device
Referring to fig. 6, a schematic diagram of an apparatus for high-performance routing of a commercial intelligent digital warehouse provided by an embodiment of the present application may include:
a receiving unit 201, configured to receive a business system business intelligence analysis request, where the request includes data information, a data type, and role information;
a role number bin set determining unit 202, configured to determine a corresponding number bin according to the role information, so as to obtain a role number bin set;
a data counting bin set determining unit 203, configured to determine a corresponding data counting bin according to the data type and the data information, so as to obtain a data counting bin set;
a final number bin set determining unit 204, configured to obtain an intersection of the role number bin set and the data number bin set to obtain a final number bin set;
a sending unit 205, configured to send the final bin set to a task distributing unit, so that the task distributing unit disassembles the request to each bin in the final bin set and executes the request in parallel.
Optionally, the apparatus further comprises:
the storage unit is used for storing results obtained by parallel execution of the number bins;
and the return unit is used for merging and returning the stored results.
Optionally, the number of bins includes:
a data bin of a Presto Preleistost high-performance engine, a solid-state hard disk high-performance version GBase structured data bin, a Kylin data bin and an Orale oracle Oracle Highun data bin are built in a Hadoop Hadupu unstructured universal data bin and Hadoop data environment.
Optionally, the data types include:
supervision and inspection type data, financial audit type data, customer screening type data, product marketing type data, operation index type data and/or analysis report type data.
Optionally, the apparatus further comprises:
and the re-execution unit is used for automatically re-executing when the parallel execution is abnormally interrupted.
The setting of each unit or module of the apparatus of the present application can be implemented by referring to the method shown in fig. 1, and is not described herein again.
The embodiment of the application provides a device for high-performance routing of business intelligent bins, which can receive business intelligent analysis requests of a business system, wherein the requests comprise data information, data types and role information, and corresponding bins are determined according to the role information to obtain a role bin set, so that the bins which can be accessed by user roles in the requests are known correspondingly. And determining corresponding bins according to the data type and the data information to obtain a data bin set, so that the bins corresponding to the data type and the data information in the request can be known. And obtaining a final number bin set by taking the intersection of the role number bin set and the data number bin set, namely the number bin set which can be accessed by the user role in the request and has the data information and the data type in the request, so that the request can be disassembled into the number bins according to the final number bin set to be executed in parallel. Therefore, high-performance routing of the commercial intelligent data warehouse is realized, data information, data types and role information do not need to traverse the matched data warehouse one by one, and the execution efficiency is higher.
It should be noted that, in the present specification, all the embodiments are described in a progressive manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only one specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of business intelligence binning high performance routing, the method comprising:
receiving a business intelligent analysis request of a business system, wherein the request comprises data information, data types and role information;
determining a corresponding number bin according to the role information to obtain a role number bin set;
determining a corresponding data bin according to the data type and the data information to obtain a data bin set;
obtaining an intersection of the role number bin set and the data number bin set to obtain a final number bin set;
and sending the final silo set to a task distribution unit so that the task distribution unit disassembles the request to each silo in the final silo set for parallel execution.
2. The method of claim 1, further comprising:
storing results obtained by parallel execution of the several bins;
and merging and returning the stored results.
3. The method of claim 1, wherein the binning comprises:
the method comprises the steps of establishing a Puresto Presto high-performance engine counting bin, a solid-state hard disk high-performance version GBase structured counting bin, an Kylin Kylin counting bin and an Oralce counting bin in Hadoop unstructured universal counting bin and Hadoop data environment.
4. The method of claim 1, wherein the data types comprise:
supervision and inspection type data, financial audit type data, customer screening type data, product marketing type data, operation index type data and/or analysis report type data.
5. The method of claim 1, further comprising:
and if the parallel execution is abnormally interrupted, automatically executing again.
6. An apparatus for commercial intelligent multi-bin high performance routing, the apparatus comprising:
the business system business intelligent analysis system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving business system business intelligent analysis requests which comprise data information, data types and role information;
the role information acquisition unit is used for acquiring role information of a role in the role;
the data counting bin set determining unit is used for determining a corresponding counting bin according to the data type and the data information to obtain a data counting bin set;
a final number bin set determining unit, configured to obtain an intersection of the role number bin set and the data number bin set to obtain a final number bin set;
and the sending unit is used for sending the final number bin set to the task distribution unit so that the task distribution unit disassembles the request to each number bin in the final number bin set to execute in parallel.
7. The apparatus of claim 6, further comprising:
the storage unit is used for storing results obtained by parallel execution of the number bins;
and the return unit is used for merging and returning the stored results.
8. The apparatus of claim 6, wherein the bins comprise:
a data bin of a Presto Preleistost high-performance engine, a solid-state hard disk high-performance version GBase structured data bin, a Kylin data bin and an Orale oracle Oracle Highun data bin are built in a Hadoop Hadupu unstructured universal data bin and Hadoop data environment.
9. The apparatus of claim 6, wherein the data types comprise:
supervision and inspection type data, financial audit type data, customer screening type data, product marketing type data, operation index type data and/or analysis report type data.
10. The apparatus of claim 6, further comprising:
and the re-execution unit is used for automatically re-executing when the parallel execution is abnormally interrupted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011637690.3A CN112685511A (en) | 2020-12-31 | 2020-12-31 | Method and device for commercial intelligent warehouse high-performance routing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011637690.3A CN112685511A (en) | 2020-12-31 | 2020-12-31 | Method and device for commercial intelligent warehouse high-performance routing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112685511A true CN112685511A (en) | 2021-04-20 |
Family
ID=75456567
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011637690.3A Pending CN112685511A (en) | 2020-12-31 | 2020-12-31 | Method and device for commercial intelligent warehouse high-performance routing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112685511A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105187365A (en) * | 2015-06-04 | 2015-12-23 | 北京邮电大学 | Method and device for access control based on roles and data items |
| CN109460645A (en) * | 2018-11-19 | 2019-03-12 | 湖南御家科技有限公司 | Distributed architecture-based permission service method, device and system |
| CN110414257A (en) * | 2018-04-26 | 2019-11-05 | 中移(苏州)软件技术有限公司 | A kind of data access method and server |
| CN111475841A (en) * | 2020-04-07 | 2020-07-31 | 腾讯科技（深圳）有限公司 | Access control method, related device, equipment, system and storage medium |
| CN111783042A (en) * | 2020-06-30 | 2020-10-16 | 北京金山云网络技术有限公司 | Database access control method and device, database main system and electronic equipment |
-
2020
- 2020-12-31 CN CN202011637690.3A patent/CN112685511A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105187365A (en) * | 2015-06-04 | 2015-12-23 | 北京邮电大学 | Method and device for access control based on roles and data items |
| CN110414257A (en) * | 2018-04-26 | 2019-11-05 | 中移(苏州)软件技术有限公司 | A kind of data access method and server |
| CN109460645A (en) * | 2018-11-19 | 2019-03-12 | 湖南御家科技有限公司 | Distributed architecture-based permission service method, device and system |
| CN111475841A (en) * | 2020-04-07 | 2020-07-31 | 腾讯科技（深圳）有限公司 | Access control method, related device, equipment, system and storage medium |
| CN111783042A (en) * | 2020-06-30 | 2020-10-16 | 北京金山云网络技术有限公司 | Database access control method and device, database main system and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10715598B1 (en) | Implementation of a web-scale data fabric | |
| US10691646B2 (en) | Split elimination in mapreduce systems | |
| El-Seoud et al. | Big Data and Cloud Computing: Trends and Challenges. | |
| US11238045B2 (en) | Data arrangement management in a distributed data cluster environment of a shared pool of configurable computing resources | |
| CN103177061B (en) | Unique value estimation in partition table | |
| US9916354B2 (en) | Generating multiple query access plans for multiple computing environments | |
| US10216782B2 (en) | Processing of updates in a database system using different scenarios | |
| US20140101093A1 (en) | Distributed, real-time online analytical processing (olap) | |
| US10007718B2 (en) | Managing data within a temporal relational database management system | |
| US11422881B2 (en) | System and method for automatic root cause analysis and automatic generation of key metrics in a multidimensional database environment | |
| US10289707B2 (en) | Data skipping and compression through partitioning of data | |
| US11200223B2 (en) | System and method for dependency analysis in a multidimensional database environment | |
| US10108665B2 (en) | Generating multiple query access plans for multiple computing environments | |
| CN114297173A (en) | Knowledge graph construction method and system for large-scale mass data | |
| Khattak et al. | Empirical analysis of recent advances, characteristics and challenges of big data | |
| Martin et al. | Multi-temperate logical data warehouse design for large-scale healthcare data | |
| CN112685511A (en) | Method and device for commercial intelligent warehouse high-performance routing | |
| Zhu et al. | Building Big Data and Analytics Solutions in the Cloud | |
| US11023485B2 (en) | Cube construction for an OLAP system | |
| Khatiwada | Architectural issues in real-time business intelligence | |
| US20240045878A1 (en) | Building and using a sparse time series database (tsdb) | |
| Wong et al. | Everything a Data Scientist Should Know About Data Management | |
| Alobaidi et al. | Benchmarking criteria for a cloud data warehouse | |
| Braun-Löhrer | Confidentiality and Performance for Cloud Databases | |
| Nagireddy | Job recommendation system with NoSQL databases: Neo4j, MongoDB, DynamoDB, Cassandra and their critical comparison |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |