CN110532261A - The method and apparatus that a kind of pair of Hive data warehouse carries out visual control - Google Patents
The method and apparatus that a kind of pair of Hive data warehouse carries out visual control Download PDFInfo
- Publication number
- CN110532261A CN110532261A CN201910672433.4A CN201910672433A CN110532261A CN 110532261 A CN110532261 A CN 110532261A CN 201910672433 A CN201910672433 A CN 201910672433A CN 110532261 A CN110532261 A CN 110532261A
- Authority
- CN
- China
- Prior art keywords
- information
- data warehouse
- buffer
- hive
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses the method and apparatus that a kind of pair of Hive data warehouse carries out visual control, which comprises the specific information and task routine message of each table of Hive data warehouse and subregion are stored by buffer；When routine mission is submitted, the information stored is parsed by structured query language sql resolver；The relationship obtained after each table information in every table between information is parsed, and the information of every table and relationship are merged, obtains the pooling information of each dimension of every table；The pooling information of each dimension of every table is read, is shown for web page.The embodiment of the present invention can comb complicated database table dependence, optimize adjustment to cluster task, so that manager is observed each dimension of data warehouse, improve monitoring convenience, reduce management cost.
Description
Technical field
The present invention relates to Hive data warehouse technology, the method for espespecially a kind of pair of Hive data warehouse progress visual control
And device.
Background technique
Hadoop is a distributed system infrastructure developed by apache foundation, and Hive is based on Hadoop
A Tool for Data Warehouse, the data file of structuring can be mapped as to a database table, and provide simple structure
Change query language (Structured Query Language, SQL) query function, sql sentence can be converted to and run on money
Computational frame MapReduce task on source manager yarn is run.Wherein, SQL is a kind of data base querying and journey
Sequence design language, for accessing data and querying, updating, and managing relational database system；It is simultaneously also database script text
The extension name of part.
During the prior art is monitored enterprise-level Hive data warehouse, complicated database table cannot be relied on and be closed
System is combed, and cannot optimize adjustment to cluster task；To make manager be not easy to observe each dimension of data warehouse,
The triviality of management service is increased, the combing of business datum table and the cost controlled are very high.
Summary of the invention
In order to solve the above-mentioned technical problem, the embodiment of the invention provides a kind of pair of Hive data warehouses to carry out visualization prison
The method and apparatus of control can comb complicated database table dependence, optimize tune to cluster task
It is whole, so that manager is observed each dimension of data warehouse, improves monitoring convenience, reduce management cost.
In order to reach the object of the invention, on the one hand, the embodiment of the invention provides a kind of pair of Hive data warehouse progress can
Method depending on changing monitoring, comprising:
The specific information and task routine message of each table of Hive data warehouse and subregion are stored by buffer；
When routine mission is submitted, the information stored is parsed by structured query language sql resolver；
Parse the relationship obtained after each table information in every table between information, and by the information of every table and relationship into
Row merges, and obtains the pooling information of each dimension of every table；
The pooling information of each dimension of every table is read, is shown for web page.
Further, which comprises
In Hive data warehouse, the information of each subregion of each table in periodic refreshing storing data warehouse as type I information,
It is stored by buffer；
After the submission of each Hive script, the sql sentence in the type I information is solved by Sql resolver
Analysis analyzes the Data source table of every section of sql and corresponding purpose table, by the Data source table and purpose table Dependency Specification work
The buffer is stored in for the second category information；
The sql sentence is converted to the Computational frame MapReduce task on resource manager yarn of running on into
Row operation, calculate and capture task specific information as third category information, be stored in the buffer.
Further, which comprises
The buffer merges the type I information of storage to third category information, forms the merging for being directed to every table
Information.
Further, which comprises
The pooling information of every table is the detailed data of every table, comprising:
Partition size, output time, resource consumption, upstream-downstream relationship.
Optionally, the method also includes:
The final amalgamation result of the buffer is read, for providing specific setting in front end.
Further, the method also includes: the buffers according to configuration, carries out whole scan to Hive data warehouse
Or specified bank scanning, the information of table each in database is cached in the form of local file, and specified according to user
Time interval is timed refreshing, and refreshing frequency can be arranged in web interface.
On the other hand, the embodiment of the invention also provides the device that a kind of pair of Hive data warehouse carries out visual control,
Include:
Memory module, for storing the specific information and task of each table of Hive data warehouse and subregion by buffer
Routine message；
Parsing module, when being submitted for routine mission, by structured query language sql resolver to the information stored
It is parsed；
Merging module, for parsing the relationship obtained after each table information in every table between information, and by every table
Information and relationship merge, obtain the pooling information of each dimension of every table；
Display module, the pooling information of each dimension for reading every table are shown for web page.
Further, the memory module is used for:
In Hive data warehouse, the information of each subregion of each table in periodic refreshing storing data warehouse as type I information,
It is stored by buffer；
After the submission of each Hive script, the sql sentence in the type I information is solved by Sql resolver
Analysis analyzes the Data source table of every section of sql and corresponding purpose table, by the Data source table and purpose table Dependency Specification work
The buffer is stored in for the second category information；
The sql sentence is converted to the Computational frame MapReduce task on resource manager yarn of running on into
Row operation, calculate and capture task specific information as third category information, be stored in the buffer.
Further, the merging module is used for:
The buffer merges the type I information of storage to third category information, forms the merging for being directed to every table
Information.
Further, described device is also used to:
The final amalgamation result of the buffer is read, for providing specific setting in front end.
The embodiment of the present invention stores the specific information and task of each table of Hive data warehouse and subregion by buffer
Routine message；When routine mission is submitted, the information stored is parsed by structured query language sql resolver；Solution
The relationship obtained after each table information in every table between information is analysed, and the information of every table and relationship are merged, is obtained
To the pooling information of each dimension of every table；The pooling information of each dimension of every table is read, web page exhibition is used for
Show.The embodiment of the present invention can comb complicated database table dependence, optimize tune to cluster task
It is whole, so that manager is observed each dimension of data warehouse, improves monitoring convenience, reduce management cost.
Other features and advantages of the present invention will be illustrated in the following description, also, partly becomes from specification
It obtains it is clear that understand through the implementation of the invention.The objectives and other advantages of the invention can be by specification, right
Specifically noted structure is achieved and obtained in claim and attached drawing.
Detailed description of the invention
Attached drawing is used to provide to further understand technical solution of the present invention, and constitutes part of specification, with this
The embodiment of application technical solution for explaining the present invention together, does not constitute the limitation to technical solution of the present invention.
Fig. 1 is the flow chart for the method that the embodiment of the present invention carries out visual control to Hive data warehouse；
Fig. 2 is the schematic illustration that the method that the embodiment of the present invention carries out visual control to Hive data warehouse is realized；
Fig. 3 is the structure chart for the device that the embodiment of the present invention carries out visual control to Hive data warehouse.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to the present invention
Embodiment be described in detail.It should be noted that in the absence of conflict, in the embodiment and embodiment in the application
Feature can mutual any combination.
Step shown in the flowchart of the accompanying drawings can be in a computer system such as a set of computer executable instructions
It executes.Also, although logical order is shown in flow charts, and it in some cases, can be to be different from herein suitable
Sequence executes shown or described step.
Fig. 1 is the flow chart for the method that the embodiment of the present invention carries out visual control to Hive data warehouse, such as Fig. 1 institute
Show, the method for the embodiment of the present invention the following steps are included:
Step 101: the specific information and task for storing each table of Hive data warehouse and subregion by buffer are customary
Information；
Wherein, the embodiment of the present invention proposes a kind of method for visually monitoring based on Hive data warehouse, main to apply
In to the progress visual control of enterprise-level Hive data warehouse.Wherein, monitoring information is accurate to table level, including each subregion, son
Partition data file total size, output time, EMS memory occupation, the superior and the subordinate's genetic connection figure of the table etc..Important information is passed through
The mode of web interface shows, and provides the customary situation that intuitive approach monitors each table for data warehouse management person, and more help
Complicated business relies between combing table.
Step 102: when routine mission is submitted, the information stored being carried out by structured query language sql resolver
Parsing；
Step 103: parsing the relationship obtained after each table information in every table between information, and by the information of every table
And relationship merges, and obtains the pooling information of each dimension of every table；
Step 104: reading the pooling information of each dimension of every table, shown for web page.
Further, which comprises
In Hive data warehouse, the information of each subregion of each table in periodic refreshing storing data warehouse as type I information,
It is stored by buffer；
After the submission of each Hive script, the sql sentence in the type I information is solved by Sql resolver
Analysis analyzes the Data source table of every section of sql and corresponding purpose table, by the Data source table and purpose table Dependency Specification work
The buffer is stored in for the second category information；
The sql sentence is converted to the Computational frame MapReduce task on resource manager yarn of running on into
Row operation, calculate and capture task specific information as third category information, be stored in the buffer.
Further, which comprises
The buffer merges the type I information of storage to third category information, forms the merging for being directed to every table
Information.
Fig. 2 is the schematic illustration that the method that the embodiment of the present invention carries out visual control to Hive data warehouse is realized,
As shown in Fig. 2, the realization process of technical solution of the embodiment of the present invention is as follows:
The embodiment of the present invention includes buffer, sql resolver and web platforms.
Specifically, buffer is introduced except Hive data warehouse, each subregion of each table in periodic refreshing storing data warehouse
Information.
Sql resolver is introduced, after the submission of each Hive script, sql sentence is parsed, analyzes every section of sql's
This Dependency Specification is stored in buffer by the purpose table of Data source table and output.
It submits yarn to carry out mapReduce calculating the sql task after parsing, and captures the beginning of task at the end of
Between, the information such as EMS memory occupation are stored in buffer.
Wherein, yarn is a kind of resource manager, is responsible for the management and scheduling of cluster resource, it may be implemented to cluster institute
There is the distribution of the various resources such as cpu, memory, file system, disk.And MapReduce is Computational frame, is run on yarn
's.
Buffer merges the information that above-mentioned steps obtain, and forms the detailed data for being directed to every table, including subregion
Size, output time, resource consumption, upstream-downstream relationship etc..
The final amalgamation result for reading buffer is shown for front end web, and provides related setting in front end.
Wherein, the method for carrying out Memory Allocation for single hive task is realized, after being included in task submission,
Introducing preprocessor, mapReduce configurator, schematic diagram are as shown in Fig. 2 before mapReduce is executed.Specific implementation process
It is as follows:
A1, buffer is introduced except Hive data warehouse, buffer can carry out Hive data warehouse complete according to configuration
Office's scanning or specified bank scanning, by the table name of table each in database, the data file total size of zone name and each subregion,
The information such as creation time are cached in the form of local file, and are timed refreshing according to the time interval that user specifies,
Refreshing frequency can be arranged in web interface；
A2, Sql resolver is introduced, after the submission of each Hive script, scan script full content simultaneously carries out sql sentence
Parsing, according to sentences such as INSERT/FROM/JOIN/UNION, analyzes the Data source table of every section of sql and the purpose table of output,
This Dependency Specification is stored in buffer.
A3, it submits yarn to carry out mapReduce calculating the sql task after parsing, and captures the beginning and end of task
Time, the information such as memory highest occupancy are stored in buffer.
The superior and the subordinate for each table that a4, buffer obtain the step a1 table obtained and partition information, step a2 rely on letter
Breath, step a3 task execution information merge, and form the partition data size for being directed to every table, upstream data source and downstream
Data whereabouts, every time customary time loss and memory consumption etc. when generating data.
A5 reads the final amalgamation result of buffer, realizes that front end is shown by Java Web etc..Show that content includes each
The table for being included in a library Hive；The partition data size that each table is included；The Data source table immediately upstream of each table and under
Swim the sublist that data import；Time when customary every time, resource consumption etc..Front end provides setting interface, for the specified caching of user
The refreshing frequency of device, caches the Hive number storehouse information in nearly XX days, and the information of caching retains XX days in buffer etc..
The embodiment of the present invention makes several storehouse managers to complicated by carrying out visual control to Hive data warehouse
Table dependence is easier to comb；Memory consumption and output time more accurate control when for each table routine, then just
In optimizing adjustment to cluster task；Meanwhile all routine partition data amount sizes in showing certain table for a period of time, it can
Across comparison is carried out for user, it is easier to abnormal positioning, such as: the partitioned file of certain customary output faces much smaller than front and back
Close timing node can be traced according to this customary carry out problem.To which manager is easier to each dimension of observation data warehouse
Degree, considerably increases the convenience of management service, reduces the cost of business datum table combing and control.
The embodiment of the present invention introduces buffer and sql resolver, by the important information of each table of Hive data warehouse into
Row storage and displaying, the partition size including table, genetic connection, customary output situation etc., by these information with visualization interface
Mode show the manager in several storehouses, each table of the angle monitoring Hive data warehouse of macroscopic view is provided, makes maintenance more just
Benefit.
In the method for visually monitoring of Hive data warehouse: storing each table of Hive data warehouse and subregion by buffer
Important information and the routine relevant information of task, routine mission the superior and the subordinate of each table are parsed when submitting by sql resolver
Genetic connection is stored by buffer, and these information are merged, and each dimensional information of every table is used for web page exhibition
Show；
Fig. 3 is the structure chart for the device that the embodiment of the present invention carries out visual control to Hive data warehouse, such as Fig. 3 institute
Show, on the other hand a kind of pair of Hive data warehouse that the embodiment of the present invention provides carries out the device of visual control, comprising:
Memory module 301, for stored by buffer each table of Hive data warehouse and subregion specific information and
Task routine message；
Parsing module 302, when being submitted for routine mission, by structured query language sql resolver to being stored
Information is parsed；
Merging module 302, for parsing the relationship obtained after each table information in every table between information, and will be every described
The information and relationship of table merge, and obtain the pooling information of each dimension of every table；
Display module 304, the pooling information of each dimension for reading every table are shown for web page.
Further, the memory module 301 is used for:
In Hive data warehouse, the information of each subregion of each table in periodic refreshing storing data warehouse as type I information,
It is stored by buffer；
After the submission of each Hive script, the sql sentence in the type I information is solved by Sql resolver
Analysis analyzes the Data source table of every section of sql and corresponding purpose table, by the Data source table and purpose table Dependency Specification work
The buffer is stored in for the second category information；
The sql sentence is converted to the Computational frame MapReduce task on resource manager yarn of running on into
Row operation, calculate and capture task specific information as third category information, be stored in the buffer.
Further, the merging module 302 is used for:
The buffer merges the type I information of storage to third category information, forms the merging for being directed to every table
Information.
Further, described device is also used to:
The final amalgamation result of the buffer is read, for providing specific setting in front end.
Wherein, described device is used for: realizing buffer and sql resolver, the parsing of table important information each for several storehouses
With storage, mainly comprise the following steps:
Buffer, the information of each subregion of each table in periodic refreshing storing data warehouse are introduced except Hive data warehouse；
Sql resolver is introduced, after the submission of each Hive script, sql sentence is parsed, analyzes every section of sql's
This Dependency Specification is stored in buffer by the purpose table of Data source table and output；
It submits yarn to carry out mapReduce calculating the sql task after parsing, and captures the beginning of task at the end of
Between, the information such as EMS memory occupation are stored in buffer；
The information that buffer will acquire merges, and forms the detailed data for being directed to every table, including partition size, output
Time, resource consumption, upstream-downstream relationship etc.；
The final amalgamation result for reading buffer is shown for front end web, and provides related setting in front end.
The embodiment of the present invention stores the specific information and task of each table of Hive data warehouse and subregion by buffer
Routine message；When routine mission is submitted, the information stored is parsed by structured query language sql resolver；Solution
The relationship obtained after each table information in every table between information is analysed, and the information of every table and relationship are merged, is obtained
To the pooling information of each dimension of every table；The pooling information of each dimension of every table is read, web page exhibition is used for
Show.The embodiment of the present invention can comb complicated database table dependence, optimize tune to cluster task
It is whole, so that manager is observed each dimension of data warehouse, improves monitoring convenience, reduce management cost.
Although disclosed herein embodiment it is as above, the content only for ease of understanding the present invention and use
Embodiment is not intended to limit the invention.Technical staff in any fields of the present invention is taken off not departing from the present invention
Under the premise of the spirit and scope of dew, any modification and variation, but the present invention can be carried out in the form and details of implementation
Scope of patent protection, still should be subject to the scope of the claims as defined in the appended claims.
Claims (10)
1. the method that a kind of pair of Hive data warehouse carries out visual control characterized by comprising
The specific information and task routine message of each table of Hive data warehouse and subregion are stored by buffer；
When routine mission is submitted, the information stored is parsed by structured query language sql resolver；
The relationship obtained after each table information in every table between information is parsed, and the information of every table and relationship are closed
And obtain the pooling information of each dimension of every table；
The pooling information of each dimension of every table is read, is shown for web page.
2. the method according to claim 1 for carrying out visual control to Hive data warehouse characterized by comprising
In Hive data warehouse, the information of each subregion of each table in periodic refreshing storing data warehouse passes through as type I information
Buffer is stored；
After the submission of each Hive script, the sql sentence in the type I information is parsed by Sql resolver, point
The Data source table of every section of sql and corresponding purpose table is precipitated, using the Data source table and the purpose table Dependency Specification as the
Two category informations are stored in the buffer；
The sql sentence is converted to the Computational frame MapReduce task run on resource manager yarn to transport
Row, calculate and capture task specific information as third category information, be stored in the buffer.
3. the method according to claim 2 for carrying out visual control to Hive data warehouse characterized by comprising
The buffer merges the type I information of storage to third category information, forms the merging letter for every table
Breath.
4. the method according to claim 3 for carrying out visual control to Hive data warehouse characterized by comprising
The pooling information of every table is the detailed data of every table, comprising:
Partition size, output time, resource consumption, upstream-downstream relationship.
5. the method according to claim 1 for carrying out visual control to Hive data warehouse, which is characterized in that also wrap
It includes:
The final amalgamation result of the buffer is read, for providing specific setting in front end.
6. the method according to claim 1 for carrying out visual control to Hive data warehouse, which is characterized in that also wrap
Include: the buffer carries out whole scan to Hive data warehouse or specified bank scans, by table each in database according to configuration
Information cached in the form of local file, and refreshing, refreshing frequency are timed according to the time interval that user specifies
It can be arranged in web interface.
7. the device that a kind of pair of Hive data warehouse carries out visual control characterized by comprising
Memory module, the specific information and task for storing each table of Hive data warehouse and subregion by buffer are customary
Information；
Parsing module when submitting for routine mission, carries out the information stored by structured query language sql resolver
Parsing；
Merging module, for parsing the relationship obtained after each table information in every table between information, and by the letter of every table
Breath and relationship merge, and obtain the pooling information of each dimension of every table；
Display module, the pooling information of each dimension for reading every table are shown for web page.
8. the device according to claim 7 for carrying out visual control to Hive data warehouse, which is characterized in that described to deposit
Storage module is used for:
In Hive data warehouse, the information of each subregion of each table in periodic refreshing storing data warehouse passes through as type I information
Buffer is stored；
After the submission of each Hive script, the sql sentence in the type I information is parsed by Sql resolver, point
The Data source table of every section of sql and corresponding purpose table is precipitated, using the Data source table and the purpose table Dependency Specification as the
Two category informations are stored in the buffer；
The sql sentence is converted to the Computational frame MapReduce task run on resource manager yarn to transport
Row, calculate and capture task specific information as third category information, be stored in the buffer.
9. the device according to claim 8 for carrying out visual control to Hive data warehouse, which is characterized in that the conjunction
And module is used for:
The buffer merges the type I information of storage to third category information, forms the merging letter for every table
Breath.
10. the device according to claim 9 for carrying out visual control to Hive data warehouse, which is characterized in that described
Device is also used to:
The final amalgamation result of the buffer is read, for providing specific setting in front end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910672433.4A CN110532261B (en) | 2019-07-24 | 2019-07-24 | Method and device for visually monitoring Hive data warehouse |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910672433.4A CN110532261B (en) | 2019-07-24 | 2019-07-24 | Method and device for visually monitoring Hive data warehouse |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110532261A true CN110532261A (en) | 2019-12-03 |
| CN110532261B CN110532261B (en) | 2022-09-20 |
Family
ID=68660867
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910672433.4A Active CN110532261B (en) | 2019-07-24 | 2019-07-24 | Method and device for visually monitoring Hive data warehouse |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110532261B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111026568A (en) * | 2019-12-04 | 2020-04-17 | 深圳前海环融联易信息科技服务有限公司 | Data and task relation construction method and device, computer equipment and storage medium |
| CN111158990A (en) * | 2019-12-31 | 2020-05-15 | 重庆富民银行股份有限公司 | Data warehouse intelligent scheduling task batch running system and method |
| CN114328568A (en) * | 2022-01-20 | 2022-04-12 | 重庆长安汽车股份有限公司 | Hive job management method and system based on web application and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110302151A1 (en) * | 2010-06-04 | 2011-12-08 | Yale University | Query Execution Systems and Methods |
| CN109739893A (en) * | 2018-12-28 | 2019-05-10 | 上海连尚网络科技有限公司 | A kind of metadata management method, equipment and computer-readable medium |
-
2019
- 2019-07-24 CN CN201910672433.4A patent/CN110532261B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110302151A1 (en) * | 2010-06-04 | 2011-12-08 | Yale University | Query Execution Systems and Methods |
| CN109739893A (en) * | 2018-12-28 | 2019-05-10 | 上海连尚网络科技有限公司 | A kind of metadata management method, equipment and computer-readable medium |
Non-Patent Citations (2)
| Title |
|---|
| VARUN GARG 等: "《Optimization of Multiple Queries for Big Data with Apache Hadoop/Hive》", 《2015 INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND COMMUNICATION NETWORKS (CICN)》 * |
| 陈耀旺 等: "《基于Hive的大数据在线分析处理》", 《计算机时代》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111026568A (en) * | 2019-12-04 | 2020-04-17 | 深圳前海环融联易信息科技服务有限公司 | Data and task relation construction method and device, computer equipment and storage medium |
| CN111026568B (en) * | 2019-12-04 | 2023-09-29 | 深圳前海环融联易信息科技服务有限公司 | Data and task relation construction method and device, computer equipment and storage medium |
| CN111158990A (en) * | 2019-12-31 | 2020-05-15 | 重庆富民银行股份有限公司 | Data warehouse intelligent scheduling task batch running system and method |
| CN114328568A (en) * | 2022-01-20 | 2022-04-12 | 重庆长安汽车股份有限公司 | Hive job management method and system based on web application and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110532261B (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10217256B2 (en) | Visually exploring and analyzing event streams | |
| CN110825882B (en) | Knowledge graph-based information system management method | |
| US10120907B2 (en) | Scaling event processing using distributed flows and map-reduce operations | |
| CN110532261A (en) | The method and apparatus that a kind of pair of Hive data warehouse carries out visual control | |
| JP5377897B2 (en) | Stream data ranking query processing method and stream data processing system having ranking query processing mechanism | |
| US20160085809A1 (en) | Enriching events with dynamically typed big data for event processing | |
| US6871321B2 (en) | System for managing networked information contents | |
| KR20190137972A (en) | Access control for data resources | |
| CN111666490A (en) | Information pushing method, device, equipment and storage medium based on kafka | |
| CN109075988A (en) | Task schedule and resource delivery system and method | |
| CN109840298B (en) | Multi-information-source acquisition method and system for large-scale network data | |
| CN108959337A (en) | Big data acquisition methods, device, equipment and storage medium | |
| CN110400029A (en) | A kind of method and system of mark management | |
| CN103207882A (en) | Shop visiting data processing method and system | |
| CN108021450A (en) | Job analysis method and apparatus based on YARN | |
| CN108874757A (en) | Report form generation method and system, computer-readable medium, electronic equipment | |
| CN112286957A (en) | API application method and system of BI system based on structured query language | |
| CN107315753B (en) | Paging method and device across multiple databases | |
| CN108519908A (en) | A kind of task dynamic management approach and device | |
| CN110825526B (en) | Distributed scheduling method and device based on ER relationship, equipment and storage medium | |
| CN110175917B (en) | Device and method for parameter graphical processing | |
| CN111124482A (en) | Method and device for configuring file information | |
| US20160171372A1 (en) | Method And Device for Real-Time Knowledge Processing Based on an Ontology With Temporal Extensions | |
| CN116070860A (en) | Scheduling method, device, equipment, medium and program product for business processing rule | |
| US11727503B2 (en) | System and method for serverless modification and execution of machine learning algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |