CN110222123A - The method and electronic equipment that Hive dynamic partition for teaching platform loads - Google Patents
The method and electronic equipment that Hive dynamic partition for teaching platform loads Download PDFInfo
- Publication number
- CN110222123A CN110222123A CN201910334360.8A CN201910334360A CN110222123A CN 110222123 A CN110222123 A CN 110222123A CN 201910334360 A CN201910334360 A CN 201910334360A CN 110222123 A CN110222123 A CN 110222123A
- Authority
- CN
- China
- Prior art keywords
- hive
- data
- dynamic
- teaching platform
- partition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005192 partition Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000004590 computer program Methods 0.000 claims description 12
- 230000004048 modification Effects 0.000 abstract description 4
- 238000012986 modification Methods 0.000 abstract description 4
- 238000011161 development Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 12
- 238000003780 insertion Methods 0.000 description 4
- 230000037431 insertion Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 239000003638 chemical reducing agent Substances 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Tourism & Hospitality (AREA)
- Educational Administration (AREA)
- Software Systems (AREA)
- Educational Technology (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention discloses the method and electronic equipment of the Hive dynamic partition load for teaching platform, and method is comprising steps of open two configurations relevant to subregion in Hive；It is inserted into data into Hive tables of data, and passes through the incoming value of a variable specified partition；The data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value.In such a way that dynamic partition loads, it can be the data file write onces of all subregions into corresponding catalogue file folder, it can dynamically be added in calculating process by way of function since subregion is passed to value, there is no need to the incoming values of multiple manual modification subregion, save computing resource and improve development efficiency.
Description
Technical field
The present invention relates to field of computer technology, the more particularly, to method of the Hive dynamic partition load of teaching platform
And electronic equipment.
Background technique
Hive is the data warehouse handling implement that a kind of bottom encapsulates Hadoop, uses the HiveQL language of class SQL real
Existing data query, the data of all Hive are stored in the compatible file system (such as Amazon S3, HDFS) of Hadoop.
Hive will not carry out any modification in load data procedures to data, only move the data into Hive in HDFS and set
Under catalogue.Data file after being calculated by Hive will be stored in the partition directory of Hive tables of data.Data platform at present
For off-line calculation the result is that being added in partition directory by way of static partition, a subregion can only be added every time by calculating,
It is exactly that data file can only be once stored under a catalogue file folder, adds multiple subregions and need that static partition is varied multiple times
Incoming value, and repeatedly calculated, this is unfavorable for saving computing resource, comparatively laborious in operation.
Therefore, the existing technology needs to be improved and developed.
Summary of the invention
In view of above-mentioned deficiencies of the prior art, the purpose of the present invention is to provide the Hive dynamic partitions for teaching platform
The method and electronic equipment of load, it is intended to solve the problems, such as that Hive data loading efficiency is low, cumbersome in the prior art.
Technical scheme is as follows:
A method of the Hive dynamic partition for teaching platform loads, wherein comprising steps of
Open two configurations relevant to subregion in Hive；
It is inserted into data into Hive tables of data, and passes through the incoming value of a variable specified partition；
The data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value.
The method that the described Hive dynamic partition for teaching platform loads, wherein in the unlatching Hive with subregion
Relevant two configurations include:
Open the switch of subregion；
Unlatching allows all subregions for dynamic.
The method that the Hive dynamic partition for teaching platform loads, wherein described to be inserted into Hive tables of data
Enter data, and includes: by the step of incoming value of a variable specified partition
The incoming value of the variable is dynamically specified by function.
The method that the Hive dynamic partition for teaching platform loads, wherein described dynamically to be referred to by function
The step of incoming value of the fixed variable, specifically includes:
The incoming value of the variable is dynamically specified by the last one field in query statement.
The method that the Hive dynamic partition for teaching platform loads, wherein described to be incited somebody to action according to the incoming value
Include: after the step that data file after being calculated by Hive is stored under different catalogue file folders
Data file under each catalogue file folder is verified, judges whether to load successfully.
The method that the Hive dynamic partition for teaching platform loads, wherein described to be inserted into Hive tables of data
Enter data, and includes: by the step of incoming value of a variable specified partition
Data are inserted into Hive tables of data by being inserted into sentence.
The method that the Hive dynamic partition for teaching platform loads, wherein last in the query statement
One field is variable expression expr.
A kind of electronic equipment characterized by comprising
Processor is adapted for carrying out each instruction, and
Equipment is stored, is suitable for storing a plurality of instruction, described instruction is suitable for being loaded and being executed by processor:
Open two configurations relevant to subregion in Hive；
It is inserted into data into Hive tables of data, and passes through the incoming value of a variable specified partition；
The data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value.
A kind of non-volatile computer readable storage medium storing program for executing, wherein the non-volatile computer readable storage medium storing program for executing is deposited
Computer executable instructions are contained, when which is executed by one or more processors, may make described one
A or multiple processors execute the method that the Hive dynamic partition for teaching platform loads.
A kind of computer program product, wherein non-volatile computer is readable to be deposited computer program product including being stored in
Computer program on storage media, computer program include program instruction, when program instruction is executed by processor, make the place
It manages device and executes the method that the Hive dynamic partition for teaching platform loads.
It, can be the data file write onces of all subregions into phase the utility model has the advantages that in such a way that dynamic partition loads
In the catalogue file folder answered, it can dynamically be added in calculating process by way of function since subregion is passed to value, nothing
The incoming value of multiple manual modification subregion is needed, computing resource is saved and improves development efficiency.
Detailed description of the invention
Fig. 1 is the flow chart for the method preferred embodiment that the present invention is loaded for the Hive dynamic partition of teaching platform.
Fig. 2 is the structural block diagram of electronic equipment preferred embodiment of the present invention.
Specific embodiment
It is of the invention to make the present invention is provided to the method and electronic equipment of the load of the Hive dynamic partition of teaching platform
Purpose, technical solution and effect are clearer, clear, and the present invention is described in more detail below.It should be appreciated that this place is retouched
The specific embodiment stated is only used to explain the present invention, is not intended to limit the present invention.
Referring to Fig. 1, Fig. 1 is that a kind of method that the Hive dynamic partition for teaching platform loads of the present invention is preferably implemented
Example flow chart, as shown, itself comprising steps of
S1, two configurations relevant to subregion in Hive are opened；
S2, it is inserted into data into Hive tables of data, and passes through the incoming value of a variable specified partition；
S3, the data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value.
In the present invention, by variable come the incoming value of specified partition, dynamic partition load thus may be implemented, thus will
It in the data file one-time write catalogue file folder of each subregion, and does not need user and repeatedly modifies the incoming value of subregion, improve
Development efficiency.
Specifically, in the step S1, two configurations relevant to subregion in Hive are opened.
The purpose for opening this two configurations is to realize subregion dynamic loading function.
Further, two configurations relevant to subregion include: in the unlatching Hive
Open the switch of subregion；
Unlatching allows all subregions for dynamic.
That is, one of configuration is the switch whether subregion is opened, another configuration is to allow all subregions to be
Dynamically.
Two kinds of subregion: static partition SP (static partition), dynamic partition DP is supported in Hive
(dynamic partition)。
The switch for wherein opening subregion, specifically according to realization of such as issuing orders: set
Hive.exec.dynamic.partition=true.Its default value be false, i.e., do not allow using.
Wherein opening allows all subregions for dynamic, specifically according to realization of such as issuing orders: set hive.exec.dynam
Ic.partition.mode=nonstrict.Can permit all subregion column is all dynamic partition column, but to be set first
A parameter hive.exec.dynamic.partition.mode is set, that is, is set to nonstrict.
It also needs to set following parameter using dynamic partition:
1、hive.exec.max.dynamic.partitions.pernode
Its default value is 100, is indicated each execution MR's (mapreduce, the programming framework of distributed arithmetic program)
On node, each mapper or reducer can permit the maximum dynamic partition number of creation.The parameter is needed according to actual
Data are set.Such as: 1 year data is contained in source data, i.e., day field there are 365 values, then the parameter just needs
365 are set greater than, if can report an error using default value 100.
2、hive.exec.max.dynamic.partitions
Default value: 1000, it indicates on all nodes for executing MR, the maximum that a dynamic partition sentence can create is dynamic
State subregion number, beyond reporting an error.
3、hive.exec.max.created.files
Default value: 100000, it indicates in entire MR Job, maximum can create how many a HDFS files, and (overall situation can be created
The maximum file number built, beyond reporting an error).General default value is enough, unless data volume is very big, the number of files for needing to create is big
In 100000, can be adjusted according to the actual situation.
4、hive.error.on.empty.partition
Default value: false is indicated when there is the generation of free subregion, if throw exception.
In the step S2, data are inserted into Hive tables of data, and pass through the incoming value of a variable specified partition.
In above-mentioned steps, the incoming value using occurrence come specified partition is eliminated, but using variable come specified partition
Incoming value, be for example named as dt, can be achieved with dynamic partition load function in this way.
Further, described that data, and the step of passing through the incoming value of a variable specified partition are inserted into Hive tables of data
Include:
Data are inserted into Hive tables of data by being inserted into sentence.
That is, insertion data are realized by insertion sentence, data are inserted into for example, by using such as under type:
insert table_A partition(dt)。
Hive includes following four data lead-in mode:
(1), data are imported from local file system to Hive table；
(2), data are imported from HDFS file system to Hive table；
(3), in creation table by recording and being inserted into created table accordingly from being inquired in other table；
(4), from inquiring corresponding data in other table and imported into Hive table.
Insertion data can use insert into or insert overwrite mode；That is insert into
Insert overwrite can be inserted into data into hive table, but insert into is directly appended to data in table
Tail portion, and insert overwrite can rewrite data, both first be deleted, then be written.The case where if there is subregion,
Insert overwrite can only rewrite current partition data.
Further, described that data, and the step of passing through the incoming value of a variable specified partition are inserted into Hive tables of data
Include:
The incoming value of the variable is dynamically specified by function, substantially dynamically specifies the occurrence of the variable,
The occurrence is incoming value.
Since incoming value is a variable, so present invention preferably employs the modes of function dynamically to specify the variable,
The incoming value of the variable described in this way will be dynamic value, rather than fixed value.
Further, the step of incoming value by function dynamically to specify the variable specifically includes:
The incoming value of the variable is dynamically specified by the last one field in query statement.
In select query statement, the last one field is the incoming value of subregion, that is, the occurrence of dt above-mentioned.
During stating insert insertion sentence and select query statement in use, if importing data, the word of inquiry
Section number must be identical with the field number of target, and Bu Nengduo can not lack, and otherwise can report an error.But if the type of field is not
Unanimously, then it will use the filling of null value, will not report an error.And use load data form into hive table when loading data, then not
It can check.It can be abandoned if field is more, will use the filling of null value if having lacked.If same field type is inconsistent,
It is filled using null value.
Further, the last one field in the query statement is variable expression expr.
Such as a specific example, expr_for_dt below are exactly the function that a specified partition is passed to value:
select column_A,column_B,expr_for_dt from table_B
In the step S3, the data file after calculating by Hive is stored in by different mesh according to the incoming value
It records under file.
Since by the way of the incoming value in dynamic specified partition, i.e., subregion, which is passed to value, can pass through function in step s 2
Mode dynamically added in calculating process, it is achieved that dynamic partition load function, by dynamic partition load can will
The data file write once of all subregions into corresponding catalogue file press from both sides in.
Further, described that data file after being calculated by Hive is stored according to the incoming value by different catalogue texts
Include: after step under part folder
Data file under each catalogue file folder is verified, judges whether to load successfully.
Data file under can finally pressing from both sides to each catalogue file verifies, and mainly judges the data file of data platform
Whether (such as off-line calculation result) has correctly been added in corresponding catalogue file folder.Such as have been added to corresponding catalogue file folder
In, then determine to load successfully, is such as not added into corresponding catalogue file folder, then determines to load unsuccessful.
The present invention also provides a kind of electronic equipment 10, as shown in Fig. 2, comprising:
Processor 110 is adapted for carrying out each instruction, and
Equipment 120 is stored, is suitable for storing a plurality of instruction, described instruction is suitable for being loaded and being executed by processor 110:
Open two configurations relevant to subregion in Hive；
It is inserted into data into Hive tables of data, and passes through the incoming value of a variable specified partition；
The data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value.
The processor 110 can for general processor, digital signal processor (DSP), specific integrated circuit (ASIC),
Field programmable gate array (FPGA), single-chip microcontroller, ARM (Acorn RISC Machine) or other programmable logic device are divided
Any combination of vertical door or transistor logic, discrete hardware component or these components.In addition, processor can also be any
Conventional processors, microprocessor or state machine.Processor also may be implemented as calculating the combination of equipment, for example, DSP and Wei Chu
Manage combination, multi-microprocessor, one or more microprocessors combination DSP core, any other this configuration of device.
It stores equipment 120 and is used as a kind of non-volatile computer readable storage medium storing program for executing, can be used for storing non-volatile software
Program, non-volatile computer executable program and module, as the Hive for teaching platform in the embodiment of the present invention is dynamic
The corresponding program instruction of method of state subregion load.Processor is stored in the non-volatile software journey in storage equipment by operation
Sequence, instruction and unit, thereby executing the method loaded for the Hive dynamic partition of teaching platform various function application with
And data processing, that is, realize above method embodiment.
It has been described in detail in the method for particular technique details in front about above-mentioned electronic equipment 10, so it will not be repeated.
The present invention also provides a kind of non-volatile computer readable storage medium storing program for executing, the non-volatile computer readable storage
Media storage has computer executable instructions, when which is executed by one or more processors, may make
One or more of processors execute the method that the Hive dynamic partition for teaching platform loads.
The present invention also provides a kind of computer program product, computer program product includes being stored in non-volatile computer
Computer program on readable storage medium storing program for executing, computer program include program instruction, when program instruction is executed by processor, are made
The processor executes the method that the Hive dynamic partition for teaching platform loads.
It should be understood that the application of the present invention is not limited to the above for those of ordinary skills can
With improvement or transformation based on the above description, all these modifications and variations all should belong to the guarantor of appended claims of the present invention
Protect range.
Claims (10)
1. a kind of method that the Hive dynamic partition for teaching platform loads, which is characterized in that comprising steps of
Open two configurations relevant to subregion in Hive；
It is inserted into data into Hive tables of data, and passes through the incoming value of a variable specified partition；
The data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value.
2. the method that the Hive dynamic partition according to claim 1 for teaching platform loads, which is characterized in that described
Opening two configurations relevant to subregion in Hive includes:
Open the switch of subregion；
Unlatching allows all subregions for dynamic.
3. the method that the Hive dynamic partition according to claim 1 for teaching platform loads, which is characterized in that described
It is inserted into data into Hive tables of data, and includes: by the step of incoming value of a variable specified partition
The incoming value of the variable is dynamically specified by function.
4. the method that the Hive dynamic partition according to claim 3 for teaching platform loads, which is characterized in that described
The step of incoming value by function dynamically to specify the variable, specifically includes:
The incoming value of the variable is dynamically specified by the last one field in query statement.
5. the method that the Hive dynamic partition according to claim 1 for teaching platform loads, which is characterized in that described
It is wrapped after the step that the data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value
It includes:
Data file under each catalogue file folder is verified, judges whether to load successfully.
6. the method that the Hive dynamic partition according to claim 1 for teaching platform loads, which is characterized in that described
It is inserted into data into Hive tables of data, and includes: by the step of incoming value of a variable specified partition
Data are inserted into Hive tables of data by being inserted into sentence.
7. the method that the Hive dynamic partition according to claim 4 for teaching platform loads, which is characterized in that described
The last one field in query statement is variable expression expr.
8. a kind of electronic equipment characterized by comprising
Processor is adapted for carrying out each instruction, and
Equipment is stored, is suitable for storing a plurality of instruction, described instruction is suitable for being loaded and being executed by processor:
Open two configurations relevant to subregion in Hive；
It is inserted into data into Hive tables of data, and passes through the incoming value of a variable specified partition；
The data file after being calculated by Hive is stored under different catalogue file folders according to the incoming value.
9. a kind of non-volatile computer readable storage medium storing program for executing, which is characterized in that the non-volatile computer readable storage medium
Matter is stored with computer executable instructions, when which is executed by one or more processors, may make institute
Stating one or more processors perform claim requires the described in any item Hive dynamic partitions for teaching platform of 1-7 to load
Method.
10. a kind of computer program product, which is characterized in that computer program product can including being stored in non-volatile computer
The computer program on storage medium is read, computer program includes program instruction, when program instruction is executed by processor, makes institute
State the method that processor perform claim requires the described in any item Hive dynamic partitions for teaching platform of 1-7 to load.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910334360.8A CN110222123A (en) | 2019-04-24 | 2019-04-24 | The method and electronic equipment that Hive dynamic partition for teaching platform loads |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910334360.8A CN110222123A (en) | 2019-04-24 | 2019-04-24 | The method and electronic equipment that Hive dynamic partition for teaching platform loads |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110222123A true CN110222123A (en) | 2019-09-10 |
Family
ID=67820042
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910334360.8A Pending CN110222123A (en) | 2019-04-24 | 2019-04-24 | The method and electronic equipment that Hive dynamic partition for teaching platform loads |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110222123A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347097A (en) * | 2020-09-23 | 2021-02-09 | 中国科学院信息工程研究所 | Data loading and query method supporting multi-version Hive table partition and electronic device |
| CN116226894A (en) * | 2023-05-10 | 2023-06-06 | 杭州比智科技有限公司 | Data security treatment system and method based on meta bin |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106600302A (en) * | 2015-10-19 | 2017-04-26 | 玺阅信息科技（上海）有限公司 | Hadoop-based commodity recommendation system |
| CN109189764A (en) * | 2018-09-20 | 2019-01-11 | 北京桃花岛信息技术有限公司 | A kind of colleges and universities' data warehouse layered design method based on Hive |
| CN109445856A (en) * | 2018-09-17 | 2019-03-08 | 深圳点猫科技有限公司 | A kind of method and electronic equipment of the acceleration application starting based on educational system |
| CN109492057A (en) * | 2018-10-26 | 2019-03-19 | 深圳点猫科技有限公司 | A kind of educational system is used for Hive offline computing method and electronic equipment |
-
2019
- 2019-04-24 CN CN201910334360.8A patent/CN110222123A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106600302A (en) * | 2015-10-19 | 2017-04-26 | 玺阅信息科技（上海）有限公司 | Hadoop-based commodity recommendation system |
| CN109445856A (en) * | 2018-09-17 | 2019-03-08 | 深圳点猫科技有限公司 | A kind of method and electronic equipment of the acceleration application starting based on educational system |
| CN109189764A (en) * | 2018-09-20 | 2019-01-11 | 北京桃花岛信息技术有限公司 | A kind of colleges and universities' data warehouse layered design method based on Hive |
| CN109492057A (en) * | 2018-10-26 | 2019-03-19 | 深圳点猫科技有限公司 | A kind of educational system is used for Hive offline computing method and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 涤生手记: "Hive使用动态分区插入数据详解", 《HTTPS://BLOG.CSDN.NET/QQ_26442553/ARTICLE/DETAILS/80382174》 * |
| 郭文明: "《MYSQL数据库应用》", 30 January 2016 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347097A (en) * | 2020-09-23 | 2021-02-09 | 中国科学院信息工程研究所 | Data loading and query method supporting multi-version Hive table partition and electronic device |
| CN112347097B (en) * | 2020-09-23 | 2022-07-29 | 中国科学院信息工程研究所 | Data loading and query method supporting multi-version Hive table partition and electronic device |
| CN116226894A (en) * | 2023-05-10 | 2023-06-06 | 杭州比智科技有限公司 | Data security treatment system and method based on meta bin |
| CN116226894B (en) * | 2023-05-10 | 2023-08-04 | 杭州比智科技有限公司 | Data security treatment system and method based on meta bin |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11720583B2 (en) | Processing data from multiple sources | |
| US11663208B2 (en) | Computer data system current row position query language construct and array processing query language constructs | |
| CN108536761A (en) | Report data querying method and server | |
| JP2022545303A (en) | Generation of software artifacts from conceptual data models | |
| CN106599167B (en) | System and method for supporting increment updating of database | |
| CN108037967A (en) | A kind of menu loading method and electronic equipment based on more parent-child structures | |
| WO2022135592A1 (en) | Method and apparatus for generating model training program mirror image, device, and storage medium | |
| CN109145055A (en) | A kind of method of data synchronization and system based on Flink | |
| CN110222123A (en) | The method and electronic equipment that Hive dynamic partition for teaching platform loads | |
| CN111368097A (en) | Knowledge graph extraction method and device | |
| CN110489401B (en) | Method and device for deploying database objects | |
| CN115237599A (en) | Rendering task processing method and device | |
| US9589036B2 (en) | Query-level access to external petabyte-scale distributed file systems | |
| CN114547165A (en) | Data standardization intelligent processing method and device | |
| US9679092B1 (en) | Constraint handling for parameterizable hardware description language | |
| US8762421B2 (en) | Accessing entities of data access layer | |
| JP2003248635A (en) | Database accessing method and system in management information base of network management protocol | |
| CN104750482A (en) | Method for constructing dynamic script execution engine based on MapReduce | |
| CN109492057A (en) | A kind of educational system is used for Hive offline computing method and electronic equipment | |
| US20210165726A1 (en) | Transferring data from a source complex variable to a target complex variable | |
| CN114281818A (en) | Data processing method, device, server and storage medium | |
| CN109542893A (en) | A kind of the compound data structure setting method and electronic equipment of language based on programming | |
| Ventocilla | Big data programming with Apache spark | |
| CN110647535A (en) | Method, terminal and storage medium for updating service data to Hive | |
| CN110221821A (en) | A kind of project function decoupling method and electronic equipment educated based on programming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190910 |
|
| RJ01 | Rejection of invention patent application after publication |