CN106250460A - 一种考虑任务相关性的Hive优化方法及系统 - Google Patents
一种考虑任务相关性的Hive优化方法及系统 Download PDFInfo
- Publication number
- CN106250460A CN106250460A CN201610607561.7A CN201610607561A CN106250460A CN 106250460 A CN106250460 A CN 106250460A CN 201610607561 A CN201610607561 A CN 201610607561A CN 106250460 A CN106250460 A CN 106250460A
- Authority
- CN
- China
- Prior art keywords
- mapreduce
- mapreduce job
- job
- parent
- child
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000005457 optimization Methods 0.000 title claims abstract description 13
- 230000006870 function Effects 0.000 claims description 202
- 238000012545 processing Methods 0.000 claims description 52
- 230000008569 process Effects 0.000 claims description 28
- 238000013507 mapping Methods 0.000 claims description 6
- 101150064138 MAP1 gene Proteins 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 101150009249 MAP2 gene Proteins 0.000 description 4
- 101100456045 Schizosaccharomyces pombe (strain 972 / ATCC 24843) map3 gene Proteins 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012812 general test Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610607561.7A CN106250460A (zh) | 2016-07-28 | 2016-07-28 | 一种考虑任务相关性的Hive优化方法及系统 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610607561.7A CN106250460A (zh) | 2016-07-28 | 2016-07-28 | 一种考虑任务相关性的Hive优化方法及系统 |
Publications (1)
Publication Number | Publication Date |
---|---|
CN106250460A true CN106250460A (zh) | 2016-12-21 |
Family
ID=57604052
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610607561.7A Pending CN106250460A (zh) | 2016-07-28 | 2016-07-28 | 一种考虑任务相关性的Hive优化方法及系统 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106250460A (zh) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103699441A (zh) * | 2013-12-05 | 2014-04-02 | 深圳先进技术研究院 | 基于任务粒度的MapReduce报表任务执行方法 |
CN103699696A (zh) * | 2014-01-13 | 2014-04-02 | 中国人民大学 | 一种云计算环境下的数据在线聚集方法 |
CN105808746A (zh) * | 2016-03-14 | 2016-07-27 | 中国科学院计算技术研究所 | 一种基于Hadoop体系的关系型大数据无缝接入方法及系统 |
-
2016
- 2016-07-28 CN CN201610607561.7A patent/CN106250460A/zh active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103699441A (zh) * | 2013-12-05 | 2014-04-02 | 深圳先进技术研究院 | 基于任务粒度的MapReduce报表任务执行方法 |
CN103699696A (zh) * | 2014-01-13 | 2014-04-02 | 中国人民大学 | 一种云计算环境下的数据在线聚集方法 |
CN105808746A (zh) * | 2016-03-14 | 2016-07-27 | 中国科学院计算技术研究所 | 一种基于Hadoop体系的关系型大数据无缝接入方法及系统 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
AU2018272840B2 (en) | Automated dependency analyzer for heterogeneously programmed data processing system | |
US12013873B2 (en) | Interactive visual analysis of datasets using a specialized virtual machine | |
CN110908997B (zh) | 数据血缘构建方法、装置、服务器及可读存储介质 | |
Parsian | Data algorithms: Recipes for scaling up with hadoop and spark | |
KR102230661B1 (ko) | Sql 검토 방법, 장치, 서버 및 저장 매체 | |
CN107077480B (zh) | 一种构建列存储数据库的方法和系统 | |
CN101432684B (zh) | 对非结构化内容进行高效索引存储的方法和装置 | |
Lin et al. | Incrementally mining high utility patterns based on pre-large concept | |
CN110119393B (zh) | 代码版本管理系统及方法 | |
US20110219045A1 (en) | Data processing device | |
US20130179863A1 (en) | Bug variant detection using program analysis and pattern identification | |
US20150220597A1 (en) | Decorrelation of user-defined function invocations in queries | |
CN112434046B (zh) | 一种数据血缘分析方法、装置、设备及存储介质 | |
CN103631601A (zh) | 一种代码生成方法和装置 | |
US9846714B2 (en) | Database device | |
CN109710220B (zh) | 关系型数据库查询方法、装置、设备及存储介质 | |
WO2019161645A1 (zh) | 基于Shell的数据表提取方法、终端、设备及存储介质 | |
CN112883030A (zh) | 数据收集方法、装置、计算机设备和存储介质 | |
KR102172255B1 (ko) | 분산형 컴퓨팅 태스크를 실행하기 위한 방법 및 장치 | |
CN114556317A (zh) | 通过版本散列链接图形的文档跟踪 | |
WO2023086322A1 (en) | Late materialization of queried data in database cache | |
US20090307200A1 (en) | System and Method For Providing Suppression of Zero, Null, Overflow, and Divide by Zero Values in Sparse Query Results | |
Kuszera et al. | Exploring data structure alternatives in the RDB to NoSQL document store conversion process | |
Chu et al. | Automatic data extraction of websites using data path matching and alignment | |
CN111143356B (zh) | 报表检索方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
CB02 | Change of applicant information | ||
CB02 | Change of applicant information |
Address after: 102209 18 Riverside Avenue, Changping District science and Technology City, Beijing Applicant after: GLOBAL ENERGY INTERCONNECTION RESEARCH INSTITUTE Co.,Ltd. Applicant after: STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER Co. Applicant after: STATE GRID CORPORATION OF CHINA Address before: 102209 Beijing Changping District future science and Technology North District Smart Grid Research Institute Applicant before: GLOBAL ENERGY INTERCONNECTION Research Institute Applicant before: STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER Co. Applicant before: State Grid Corporation of China |
|
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161221 |