CN110019293A - List data deriving method and device based on impala database - Google Patents
List data deriving method and device based on impala database Download PDFInfo
- Publication number
- CN110019293A CN110019293A CN201710800784.XA CN201710800784A CN110019293A CN 110019293 A CN110019293 A CN 110019293A CN 201710800784 A CN201710800784 A CN 201710800784A CN 110019293 A CN110019293 A CN 110019293A
- Authority
- CN
- China
- Prior art keywords
- data
- exported
- storage
- array
- export
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2423—Interactive query statement specification based on a database schema
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of list data deriving methods and device based on impala database, are related to technical field of data processing, main purpose is in order to which when list data exports in impala database, raising exports efficiency and invents.The method comprise the steps that determining the data to be exported of the target table according to the content in target table；The data to be exported is stored into storage array, the storage array has corresponding array serial number；According to the array serial number, the data to be exported of the storage storage of array is exported according to pre-set sequence.The present invention is for exporting the list data in impala database.
Description
Technical field
The present invention relates to technical field of data processing more particularly to a kind of list data export based on impala database
Method and device.
Background technique
With the rapid development of internet, the use of distributed data base gradually increases.Wherein, impala database conduct
One kind of distributed data base is often used by code compilation personnel.In impala database, data are all in a tabular form
Storage.In actual use, code writers are often required to using the data in table, it is therefore desirable to from impala database
Middle export list data simultaneously uses.Since the data of the database are storing datas in a tabular form, though this storage mode
It is easy to read, but when exporting data, required step is complex, is not easy to the extraction and export of data.
Currently, when needing to export list data from impala database, it generally will be every in table by manual type
Data successively exports.However, in actual use, the mode by manually exporting data generally requires to consume a large amount of manpower,
Especially when the derived table of needs is more, data volume is larger in table, the artificial process for exporting list data can be consumed greatly
The time of amount, and then influence to export table data speed from impala database, lead to list data in impala database
It is lower to export efficiency.
Summary of the invention
In view of the above problems, the present invention provides a kind of list data deriving method and device based on impala database,
Main purpose is in order to which when list data exports in impala database, raising exports efficiency.
In order to solve the above technical problems, in a first aspect, the present invention provides a kind of table numbers based on impala database
According to deriving method, this method comprises:
According to the content in target table, the data to be exported of the target table is determined；
The data to be exported is stored into storage array, the storage array has corresponding array serial number；
According to the array serial number, the data to be exported of the storage storage of array is exported according to pre-set sequence.
Optionally, the content according in target table determines that the data to be exported of the target table includes:
The export instruction assigned according to user obtains target table from impala database, wraps in the export instruction
The Data Identification of data to be exported containing the table；
According to the Data Identification, data to be exported is determined in the content of the target table.
Optionally, the export instruction assigned according to user, obtaining target table from impala database includes:
The export instruction assigned according to user parses the table mark of target table from the export instruction；
It is identified according to the table of the target table, table of the inquiry with the presence or absence of the corresponding table mark from database
Lattice；
When inquiring the table of the corresponding table mark, the table is determined as target table.
Optionally, described store the data to be exported includes: into storage array
When in the data to be exported there are when multinomial data, determine the data item number of multinomial data in data to be exported with
And the data relationship between the multinomial data；
Multiple storage arrays of the corresponding data item number are obtained, and the multinomial data are stored respectively to multiple storages
In different storage arrays in array；
According to the data relationship, corresponding array sequence is distributed for the different storage arrays in the multiple storage array
Number.
Optionally, described according to the array serial number, according to pre-set sequence by described in the storage storage of array wait lead
Data, which export, out includes:
The export sequence of the corresponding array serial number is parsed from export sequential instructions；
According to the export sequence, successively the data to be exported stored in the storage array of the correspondence array serial number is led
Out.
Optionally, described according to export sequence, it will successively be stored in the storage array of the correspondence array serial number
Data to be exported exports
Obtain preset separator, the preset separator be used for the data in the different storage arrays in export into
Row separates；
Successively the data to be exported stored in the storage array of the correspondence array serial number is exported, and is utilized described preset
Separator separates the data to be exported stored in different storage arrays.
Second aspect, the present invention also provides a kind of list data guiding device based on impala database, the devices
Include:
Determination unit, for determining the data to be exported of the target table according to the content in target table；
Storage unit, the data to be exported for determining the determination unit are stored into storage array, the storage
Array has corresponding array serial number；
Lead-out unit is used for according to the array serial number, the storage number for storing the storage unit according to pre-set sequence
The data to be exported export of group storage.
Optionally, the determination unit includes:
Module is obtained, the export instruction for assigning according to user obtains target table from impala database, described
The Data Identification of data to be exported comprising the table in export instruction；
Determining module is used for according to the Data Identification, in the content for the target table that the acquisition module obtains really
Determine data to be exported.
Optionally, the acquisition module includes:
Analyzing sub-module, the export instruction for being assigned according to user parse target table from the export instruction
Table mark；
Inquire submodule, the table mark of the target table for parsing according to the analyzing sub-module, from database
Table of the middle inquiry with the presence or absence of the corresponding table mark；
Determine submodule, it, will be described when for inquiring the table of the corresponding table mark when the inquiry submodule
Table is determined as target table.
Optionally, the storage unit includes:
Determining module, for when there are when multinomial data, determine multinomial number in data to be exported in the data to be exported
According to data item number and the multinomial data between data relationship；
Memory module, for obtaining the multiple storage arrays for the data item number that the corresponding determining module determines, and by institute
It states in the different storage arrays that multinomial data are stored respectively into multiple storage arrays；
Distribution module, the data relationship for being determined according to the determining module, is stored more by the memory module
Different storage arrays in a storage array distribute corresponding array serial number.
Optionally, which is characterized in that the lead-out unit includes:
Parsing module, for parsing the export sequence of the corresponding array serial number from export sequential instructions；
Export module, the export sequence for being parsed according to the parsing module successively will the corresponding array serial number
Storage array in store data to be exported export.
Optionally, the export module includes:
Acquisition submodule, for obtaining preset separator, the preset separator is used for in the different storage arrays
Data export when be separated；
Submodule is exported, for successively leading the data to be exported stored in the storage array of the correspondence array serial number
Out, and using the acquisition submodule the preset separator obtained is to the data to be exported stored in different storage arrays point
Every.
To achieve the goals above, according to the third aspect of the invention we, a kind of storage medium, the storage medium are provided
Program including storage, wherein equipment where controlling the storage medium in described program operation executes base described above
In the list data deriving method of impala database.
To achieve the goals above, according to the fourth aspect of the invention, a kind of processor is provided, the processor is used for
Run program, wherein described program executes the list data deriving method described above based on impala database when running.
By above-mentioned technical proposal, the list data deriving method and dress provided by the invention based on impala database
It sets, for the prior art when from data are exported in list data in impala database, is exported, led using manual type
The problem for causing export efficiency lower stores data to be exported after the present invention is by determining data to be exported in target table
Into storage array, then the data to be exported stored in array is led according to pre-set sequence according to the corresponding serial number of storage array
Out, the export function to list data in impala database is realized.Also, by storing storage of array data to be exported,
And corresponding data to be exported in array will be stored according to the corresponding array serial number of storage array and exported, it may be implemented table
The effect that lattice data are derived automatically from, solve prior art export efficiency caused when using artificial derived table lattice data compared with
Low problem, to improve export efficiency when exporting list data from impala database.Meanwhile when table is more,
When data to be exported is more in table, to people when method according to the present invention can be avoided by manually exporting list data
The consumption of power, then saves human resources.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows a kind of list data deriving method process based on impala database provided in an embodiment of the present invention
Figure；
Fig. 2 shows another list data deriving method streams based on impala database provided in an embodiment of the present invention
Cheng Tu；
Fig. 3 shows a kind of group of list data guiding device based on impala database provided in an embodiment of the present invention
At block diagram；
Fig. 4 shows another list data guiding device based on impala database provided in an embodiment of the present invention
Composition block diagram.
Specific embodiment
The exemplary embodiment that the present invention will be described in more detail below with reference to accompanying drawings.Although showing the present invention in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the present invention without should be by embodiments set forth here
It is limited.It is to be able to thoroughly understand the present invention on the contrary, providing these embodiments, and can be by the scope of the present invention
It is fully disclosed to those skilled in the art.
In order to improve efficiency derived from the data in the list data export process based on impala database, the present invention is real
It applies example and provides a kind of list data deriving method based on impala database, as shown in Figure 1, this method comprises:
101, according to the content in target table, the data to be exported of the target table is determined.
In general, wherein list data is all not to be led when using the list data in impala database
It out and uses, also, in impala database, not all table is all current desired is wanted.Therefore, according to this
Method described in step needs first to determine which is to need to carry out derived data in the content in table, i.e. the present invention is implemented
Data to be exported described in example.
For example, only needing to use the first row in list data sometimes when the editorial staff of program uses list data
Part, and other data are not need derived from progress, or do not need currently.Therefore, according to this step
Method determined in content in the table first according to the needs of users and need derived data, i.e. data to be exported.
In this step, determine that the mode of data to be exported can solve in the instruction that user assigns according to user demand
Derived data needed for being precipitated, can also according to position of the preset rule settings data to be exported in list data, such as
Setting data to be exported is the first row in list data as data to be exported.Certainly, specific method of determination can basis
It needs voluntarily to select from above two mode, can also be set according to actual needs, not do specific restriction herein,
It can choose according to actual needs.
102, the data to be exported is stored into storage array.
Due to having determined that data to be exported in a step 101, the following method according to this step can
Storing the data to be exported determined into storage array, wherein the storage array can be understood as one and be corresponding with storage
The array in space, there are ordinal relations between different arrays, i.e., each storage array has its own corresponding array sequence
Number, to distinguish other storage arrays.
In practical applications, data to be exported may include multinomial different data, and different data are stored in one
It stores clearly inappropriate in array.Therefore, it is necessary to be respectively stored in not according to the data of the different item in data to be exported
It in same storage array, and is that storage array distributes corresponding array serial number according to the relationship between different item datas.Exist as a result,
In this step, the data of different item can be specifically distinguished in data to be exported according to preset rule first, then successively
Different data are stored in storage array.Or according to preset storage array quantity, data to be exported is divided into pair
The data group of data bulk is answered, then data group is sequentially stored in storage array.Certainly, above two by data to be exported
The mode stored into storage array is only two kinds of examples of this step the method, specifically, can also be according to practical need
To determine specific storage mode.However, it is desirable to which explanation is which kind of specific storage mode no matter selected, require to ensure
Data to be exported can be stored entirely in storage array, to avoid subsequent because failing to cause due to all storing data to be exported
Occurs the problem of shortage of data in step.
103, according to the array serial number, the data to be exported of the storage storage of array is led according to pre-set sequence
Out.
Since each storage array has matched an array serial number, number can will be stored according to array serial number
The data stored in group are exported according to pre-set sequence.Wherein, the pre-set sequence can be preset permanent order, can also
To be that the instruction assigned by user determines.In this regard, the pre-set sequence can voluntarily be chosen according to the actual situation, herein
Specific restriction is not done.
List data deriving method provided in an embodiment of the present invention based on impala database, for the prior art from
It when list data in impala database exports data, is exported using manual type, leads to export efficiency is lower to ask
Topic, the embodiment of the present invention will be stored by storage storage of array data to be exported, and according to the corresponding array serial number of storage array
Corresponding data to be exported is exported in array, and the effect for being derived automatically from list data may be implemented, and solves existing skill
The lower problem of art export efficiency caused when using artificial derived table lattice data, to improve from impala database
Export efficiency when middle export list data, meanwhile, when table is more, data to be exported is more in table, according to the present invention
To the consumption of manpower when method described in embodiment can be avoided by manually exporting list data, manpower money is then saved
Source.
Further, as the refinement and extension to embodiment illustrated in fig. 1, the embodiment of the invention also provides another bases
In the list data deriving method of impala database, as shown in Fig. 2, its specific steps includes:
201, according to the content in target table, the data to be exported of the target table is determined.
It in general, all include a large amount of data in table, but data not all in actual use are all
It is exported and is used, meanwhile, it include table more than one in impala database, but include multiple tables,
Therefore in this step, it can be instructed first according to the export that user assigns, determine and needed from impala data from the instruction
The table obtained in library, i.e. target table described in the embodiment of the present invention.Then, true further according to the Data Identification in export instruction
Data to be exported in the table content that sets the goal.Further, when obtaining target table from impala database, specifically
Whether the table mark that target table can be first parsed from export instruction, then inquire in the database according to the mark and deposit
It can then determine that the table is when inquiring the table with table mark in the table with table mark
Target table described in the embodiment of the present invention.
Wherein, in this step, table mark can be the title of table, can also be with form serial number, either
The creation time of table or date, specifically, it is not limited here, can be determined according to practical situation.Meanwhile the number
It can be the title of data according to mark, one or more parameters such as content or position in the table are constituted, do not limited herein
It is fixed, the other parameters of data can also be chosen as Data Identification.
The data to be exported in target table is determined by using Data Identification and table mark as a result, can be improved
The accuracy rate of result derived from list data can be avoided in subsequent export data and the case where export error in data occurs.
202, the data to be exported is stored into storage array.
In this step, described to store array, array serial number and data to be exported is stored into the side into storage array
Formula is identical as the description in step 102 in previous embodiment, and this is no longer going to repeat them, wherein specifically in this step may be used
With are as follows: firstly, when determine exported in the data to be exported there are multinomial data when, in order to ensure export after number
According to the consistency of data structure or data relationship with the former data in table, it is thus necessary to determine that the multinomial data in data to be exported
Between data relationship, meanwhile, obtain and need derived data item number, and obtain multiple storages of identical data according to the item number
Then array stores the multinomial data in the different storage arrays into multiple storage arrays respectively.Finally, according to described
Data relationship distributes corresponding array serial number for the different storage arrays in the multiple storage array.
It should be noted that guarantee the selected mode stored and store the quantity of array can be by described wait export
Data all store, to avoid there is the problem of shortage of data in subsequent step.
203, according to the array serial number, the data to be exported of the storage storage of array is led according to pre-set sequence
Out.
It after step 202, stores in array and stores different data, need when being exported according to user's
It needs to export data according to the export of setting sequence, specifically, the export sequential instructions that can be assigned from user first carry out
Parsing parses the export sequence of corresponding array serial number.Then, according to export sequence, successively by the correspondence array serial number
The data to be exported stored in storage array carries out export operation.
Further, it when according to export sequence, will successively be stored in the storage array of the correspondence array serial number
When data to be exported exports, it is also necessary to the blank character between data is determined, to avoid that will occur being difficult to know after the export of multiple data
Other phenomenon.Therefore, preset separator can also be obtained in this step, then by the data to be exported in different storage arrays
According to sequentially being exported, while the data between each storage array are separated with the preset separator got.It needs
Illustrate, when carrying out export operation, it is preset as needed that the separator can be user, can not also be into
Row setting.When without setting, the separator of system meeting allocating default, such as selection branch "；" as default separator.
The method according to this step as a result, by parsing export sequence from user instruction, and according to the export
Sequence exports the data to be exported stored in array, realizes the control effect of the export sequence to data to be exported.
Meanwhile the data in different storage arrays are separated by obtaining preset separator, and using separator, it can make to lead
Data after out are easier to be identified, and ensure that the accuracy of the export result of data to be exported.
Further, method described in middle above-mentioned steps 201-203 according to embodiments of the present invention, in conjunction with specific applied field
Scape is exemplified below:
After the export for receiving user instructs, target table can be parsed according to the information in the export instruction of user
Table mark.In this example, table mark can be the title of table, and specific title is
" DailyUserEvent ", then according to the title, Query Name is " DailyUserEvent " in impala database
Table, after inquiring the table, determine the table be target table, i.e., it is as shown in table 1, specific as follows:
Table 1
Then according to the content in export instruction, it is determined that it is the interior of the Name column in table that user, which needs derived data,
Hold, then determines that data to be exported is the data in Name column according to the Data Identification Name in instruction.
Then, determine data to be exported be " tracker_version ", " profile_id ", " user_id ",
" final_user_id ", " is_new_visitor ", " week " are stored these data respectively to storage array with after " age "
In field, wherein each storage array includes a corresponding array serial number, i.e. field [1] to field [7], according to
The sequence of data to be exported in the table, successively by " tracker_version ", " profile_id ", " user_id ",
" final_user_id ", " is_new_visitor ", " week " and " age " are stored respectively to storage array field [1] extremely
In field [7].
After storing data to be exported into storage array, the export sequence in the export sequential instructions of user is parsed.
Wherein, export sequence can be parsed from the export sequential instructions of user.Alternatively, when user does not assign export sequential instructions
When, then it can be exported according to the default sequence that system is arranged.Meanwhile after export sequence has been determined, it is also necessary to determine
For separator used when exporting data, in this example, when preset separator be comma, " when, then export to
It is separated when exporting data using comma.Further, it when carrying out export operation, can also be added before derived data
The title " DailyUserEvent " of the target table, then exports corresponding data to be exported, specifically after the title
, result after being exported can be with are as follows:
" DailyUserEvent==t.tracker_version, t.profile_id, t.user_id, t.final_
user_id,t.is_new_visitor,t.week,t.age”
Further, as the realization to method shown in above-mentioned Fig. 1, the embodiment of the invention also provides one kind to be based on
The list data guiding device of impala database, for being realized to above-mentioned method shown in FIG. 1.The Installation practice with
Preceding method embodiment is corresponding, be it is easy to read, present apparatus embodiment no longer to the detail content in preceding method embodiment into
Row repeats one by one, it should be understood that the device in the present embodiment can correspond to the full content realized in preceding method embodiment.
As shown in figure 3, the device comprises determining that unit 31, storage unit 32 and lead-out unit 33, wherein
Determination unit 31 can be used for determining the data to be exported of the target table according to the content in target table.
Storage unit 32 can be used for storing the data to be exported that the determination unit 31 determines into storage array,
The storage array has corresponding array serial number.
Lead-out unit 33, can be used for according to the array serial number, store the storage unit 32 according to pre-set sequence
Storage storage of array the data to be exported export.
Further, as the realization to method shown in above-mentioned Fig. 2, the embodiment of the invention also provides another kinds to be based on
The list data guiding device of impala database, for being realized to above-mentioned method shown in Fig. 2.The Installation practice with
Preceding method embodiment is corresponding, be it is easy to read, present apparatus embodiment no longer to the detail content in preceding method embodiment into
Row repeats one by one, it should be understood that the device in the present embodiment can correspond to the full content realized in preceding method embodiment,
Another list data guiding device based on impala database provided in this embodiment, can be as shown in figure 4, the dress
It sets and comprises determining that unit 41, storage unit 42 and lead-out unit 43, wherein
Determination unit 41 can be used for determining the data to be exported of the target table according to the content in target table.
Storage unit 42 can be used for storing the data to be exported that the determination unit 41 determines into storage array,
The storage array has corresponding array serial number.
Lead-out unit 43, can be used for according to the array serial number, store the storage unit 42 according to pre-set sequence
Storage storage of array the data to be exported export.
Further, the determination unit 41 includes:
Module 411 is obtained, the export instruction that can be used for assigning according to user obtains object table from impala database
Lattice, the Data Identification for exporting the data to be exported in instruction comprising the table.
Determining module 412 can be used for according to the Data Identification, in the target table that the acquisition module 411 obtains
Content in determine data to be exported.
Further, the acquisition module 411 includes:
Analyzing sub-module 4111, the export instruction that can be used for being assigned according to user, parses mesh from export instruction
Mark the table mark of table.
Inquire submodule 4112, the table mark for the target table that can be used for parsing according to the analyzing sub-module 4111
Know, table of the inquiry with the presence or absence of the corresponding table mark from database.
It determines submodule 4113, can be used for inquiring the table of the corresponding table mark when the inquiry submodule 4112
When lattice, the table is determined as target table.
Further, the storage unit 42 includes:
Determining module 421 can be used for when, there are when multinomial data, determining in data to be exported in the data to be exported
Data relationship between the data item number of multinomial data and the multinomial data；
Memory module 422 can be used for obtaining the multiple storage numbers for the data item number that the corresponding determining module 421 determines
Group, and the multinomial data are stored respectively in the different storage arrays into multiple storage arrays；
Distribution module 423 can be used for the data relationship determined according to the determining module 421, be the memory module
Different storage arrays in the 422 multiple storage arrays stored distribute corresponding array serial number.
Further, the lead-out unit 43 includes:
Parsing module 431 can be used for parsing the export sequence of the corresponding array serial number from export sequential instructions.
Export module 432, the export sequence that can be used for being parsed according to the parsing module 431 successively will corresponding institute
State the data to be exported export stored in the storage array of array serial number.
Further, the export module 432 includes:
Acquisition submodule 4321 can be used for obtaining preset separator, and the preset separator is for depositing the difference
Data in storage array are separated in export.
Submodule 4322 is exported, can be used for successively being stored in the storage array of the correspondence array serial number wait export
Data export, and the preset separator obtained using the acquisition submodule 4321 in different storage arrays to storing wait lead
Interval data out.
By above-mentioned technical proposal, the embodiment of the present invention provides a kind of list data export side based on impala database
Method and device.For the prior art when exporting data from the list data in impala database, carried out using manual type
Export causes to export the lower problem of efficiency, and the embodiment of the present invention passes through storage storage of array data to be exported, and according to storage
The corresponding array serial number of array will store corresponding data to be exported in array and export, and may be implemented list data is automatic
Derived effect solves the problems, such as that prior art export efficiency caused when using artificial derived table lattice data is lower,
Export efficiency when list data is exported from impala database to improve, meanwhile, when table is more, wait lead in table
When data are more out, to manpower when the method can be avoided by manually exporting list data according to embodiments of the present invention
Consumption, then saves human resources.Meanwhile by using Data Identification and table mark come determine in target table wait lead
The accuracy rate of list data export result can be improved in data out, can be avoided in subsequent export data and export data occurs
The situation of mistake.In addition, by parsing export sequence from user instruction, and will be in storage array according to the export sequence
Data to be exported is exported, and the control effect of the export sequence to data to be exported is realized.In addition, by obtaining preset point
The data in different storage arrays are separated every symbol, and using separator, the data after can making export are easier quilt
Identification, ensures that the accuracy of the export result of data to be exported.
The list data guiding device based on impala database includes processor and memory, above-mentioned determining list
Member, storage unit and lead-out unit etc. store in memory as program unit, are stored in memory by processor execution
In above procedure unit realize corresponding function.
Include kernel in processor, is gone in memory to transfer corresponding program unit by kernel.Kernel can be set one
Or more, the export speed in the list data export process based on impala database is improved by adjusting kernel parameter,
Improve the export efficiency of list data.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, if read-only memory (ROM) or flash memory (flash RAM), memory include that at least one is deposited
Store up chip.
The embodiment of the invention provides a kind of storage mediums, are stored thereon with program, real when which is executed by processor
The existing list data deriving method based on impala database.
The embodiment of the invention provides a kind of processor, the processor is for running program, wherein described program operation
Based on the list data deriving method of impala database described in Shi Zhihang.
The embodiment of the invention provides a kind of equipment, equipment include processor, memory and storage on a memory and can
The program run on a processor, processor perform the steps of when executing program according to the content in target table, determine institute
State the data to be exported of target table；The data to be exported is stored into storage array, the storage array is with correspondence
Array serial number；According to the array serial number, the data to be exported of the storage storage of array is led according to pre-set sequence
Out.
Further, the content according in target table determines that the data to be exported of the target table includes:
The export instruction assigned according to user obtains target table from impala database, wraps in the export instruction
The Data Identification of data to be exported containing the table；
According to the Data Identification, data to be exported is determined in the content of the target table.
Further, the export instruction assigned according to user, obtaining target table from impala database includes:
The export instruction assigned according to user parses the table mark of target table from the export instruction；
It is identified according to the table of the target table, table of the inquiry with the presence or absence of the corresponding table mark from database
Lattice；
When inquiring the table of the corresponding table mark, the table is determined as target table.
Further, described store the data to be exported includes: into storage array
When in the data to be exported there are when multinomial data, determine the data item number of multinomial data in data to be exported with
And the data relationship between the multinomial data；
Multiple storage arrays of the corresponding data item number are obtained, and the multinomial data are stored respectively to multiple storages
In different storage arrays in array；
According to the data relationship, corresponding array sequence is distributed for the different storage arrays in the multiple storage array
Number.
Further, described according to the array serial number, according to pre-set sequence by described in the storage storage of array to
Export data, which export, includes:
The export sequence of the corresponding array serial number is parsed from export sequential instructions；
According to the export sequence, successively the data to be exported stored in the storage array of the correspondence array serial number is led
Out.
Further, described according to the export sequence, it will successively be stored in the storage array of the correspondence array serial number
Data to be exported export include:
Obtain preset separator, the preset separator be used for the data in the different storage arrays in export into
Row separates；
Successively the data to be exported stored in the storage array of the correspondence array serial number is exported, and is utilized described preset
Separator separates the data to be exported stored in different storage arrays.
Equipment in the embodiment of the present invention can be server, PC, PAD, mobile phone etc..
The embodiment of the invention also provides a kind of computer program products, when executing on data processing equipment, are suitable for
Execute the program of initialization there are as below methods step: according to the content in target table, determine the target table wait export
Data；The data to be exported is stored into storage array, the storage array has corresponding array serial number；According to described
Array serial number exports the data to be exported of the storage storage of array according to pre-set sequence.
Further, the content according in target table determines that the data to be exported of the target table includes:
The export instruction assigned according to user obtains target table from impala database, wraps in the export instruction
The Data Identification of data to be exported containing the table；
According to the Data Identification, data to be exported is determined in the content of the target table.
Further, the export instruction assigned according to user, obtaining target table from impala database includes:
The export instruction assigned according to user parses the table mark of target table from the export instruction；
It is identified according to the table of the target table, table of the inquiry with the presence or absence of the corresponding table mark from database
Lattice；
When inquiring the table of the corresponding table mark, the table is determined as target table.
Further, described store the data to be exported includes: into storage array
When in the data to be exported there are when multinomial data, determine the data item number of multinomial data in data to be exported with
And the data relationship between the multinomial data；
Multiple storage arrays of the corresponding data item number are obtained, and the multinomial data are stored respectively to multiple storages
In different storage arrays in array；
According to the data relationship, corresponding array sequence is distributed for the different storage arrays in the multiple storage array
Number.
Further, described according to the array serial number, according to pre-set sequence by described in the storage storage of array to
Export data, which export, includes:
The export sequence of the corresponding array serial number is parsed from export sequential instructions；
According to the export sequence, successively the data to be exported stored in the storage array of the correspondence array serial number is led
Out.
Further, described according to the export sequence, it will successively be stored in the storage array of the correspondence array serial number
Data to be exported export include:
Obtain preset separator, the preset separator be used for the data in the different storage arrays in export into
Row separates；
Successively the data to be exported stored in the storage array of the correspondence array serial number is exported, and is utilized described preset
Separator separates the data to be exported stored in different storage arrays.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flashRAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including element
There is also other identical elements in process, method, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The above is only embodiments herein, are not intended to limit this application.To those skilled in the art,
Various changes and changes are possible in this application.It is all within the spirit and principles of the present application made by any modification, equivalent replacement,
Improve etc., it should be included within the scope of the claims of this application.
Claims (10)
1. a kind of list data deriving method based on impala database characterized by comprising
According to the content in target table, the data to be exported of the target table is determined；
The data to be exported is stored into storage array, the storage array has corresponding array serial number；
According to the array serial number, the data to be exported of the storage storage of array is exported according to pre-set sequence.
2. the method according to claim 1, wherein the content according in target table, determines the mesh
Mark table data to be exported include:
The export instruction assigned according to user obtains target table from impala database, includes institute in the export instruction
State the Data Identification of the data to be exported of table；
According to the Data Identification, data to be exported is determined in the content of the target table.
3. according to the method described in claim 2, it is characterized in that, the export assigned according to user instructs, from impala
Target table is obtained in database includes:
The export instruction assigned according to user parses the table mark of target table from the export instruction；
It is identified according to the table of the target table, table of the inquiry with the presence or absence of the corresponding table mark from database；
When inquiring the table of the corresponding table mark, the table is determined as target table.
4. the method according to claim 1, wherein described store the data to be exported into storage array
Include:
When there are the data item number for when multinomial data, determining multinomial data in data to be exported and institutes in the data to be exported
State the data relationship between multinomial data；
Multiple storage arrays of the corresponding data item number are obtained, and the multinomial data are stored respectively to multiple storage arrays
In different storage arrays in；
According to the data relationship, corresponding array serial number is distributed for the different storage arrays in the multiple storage array.
5. method according to claim 1-4, which is characterized in that it is described according to the array serial number, according to pre-
Sequence, which is set, by the data to be exported export of the storage storage of array includes:
The export sequence of the corresponding array serial number is parsed from export sequential instructions；
According to the export sequence, successively the data to be exported stored in the storage array of the correspondence array serial number is exported.
6. according to the method described in claim 5, it is characterized in that, described according to export sequence, successively will it is corresponding described in
The data to be exported stored in the storage array of array serial number exports
Preset separator is obtained, the preset separator is used to divide the data in the different storage arrays in export
Every；
Successively the data to be exported stored in the storage array of the correspondence array serial number is exported, and utilizes the preset separation
It accords with and the data to be exported stored in different storage arrays is separated.
7. a kind of list data guiding device based on impala database characterized by comprising
Determination unit, for determining the data to be exported of the target table according to the content in target table；
Storage unit, the data to be exported for determining the determination unit are stored into storage array, the storage array
With corresponding array serial number；
Lead-out unit, for depositing the storage array that the storage unit stores according to pre-set sequence according to the array serial number
The data to be exported export of storage.
8. device according to claim 7, which is characterized in that the determination unit includes:
Module is obtained, the export instruction for assigning according to user obtains target table, the export from impala database
The Data Identification of data to be exported in instruction comprising the table；
Determining module, for according to the Data Identification, determined in the content for the target table that the acquisition module obtains to
Export data.
9. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program
When control the storage medium where equipment perform claim require 1 to described in any one of claim 6 based on impala
The list data deriving method of database.
10. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit requires 1 to the list data deriving method based on impala database described in any one of claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710800784.XA CN110019293A (en) | 2017-09-07 | 2017-09-07 | List data deriving method and device based on impala database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710800784.XA CN110019293A (en) | 2017-09-07 | 2017-09-07 | List data deriving method and device based on impala database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110019293A true CN110019293A (en) | 2019-07-16 |
Family
ID=67186209
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710800784.XA Pending CN110019293A (en) | 2017-09-07 | 2017-09-07 | List data deriving method and device based on impala database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019293A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685481A (en) * | 2019-10-17 | 2021-04-20 | 北京京东振世信息技术有限公司 | Data processing method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944084A (en) * | 2009-07-09 | 2011-01-12 | 镇江华扬信息科技有限公司 | Database automatic operation system based on SQL script and FTP transmission technique |
| CN103530346A (en) * | 2013-09-27 | 2014-01-22 | 丁跃潮 | Drawing method and system for converting data sheets into data visual graphs |
| US20140304275A1 (en) * | 2013-03-14 | 2014-10-09 | LightMiner Systems, Inc. | Optimizing wide data-type storage and analysis of data in a column store database |
| CN105574002A (en) * | 2014-10-11 | 2016-05-11 | 镇江鼎拓科技信息有限公司 | SQL database-data file transmission method |
-
2017
- 2017-09-07 CN CN201710800784.XA patent/CN110019293A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944084A (en) * | 2009-07-09 | 2011-01-12 | 镇江华扬信息科技有限公司 | Database automatic operation system based on SQL script and FTP transmission technique |
| US20140304275A1 (en) * | 2013-03-14 | 2014-10-09 | LightMiner Systems, Inc. | Optimizing wide data-type storage and analysis of data in a column store database |
| CN103530346A (en) * | 2013-09-27 | 2014-01-22 | 丁跃潮 | Drawing method and system for converting data sheets into data visual graphs |
| CN105574002A (en) * | 2014-10-11 | 2016-05-11 | 镇江鼎拓科技信息有限公司 | SQL database-data file transmission method |
Non-Patent Citations (1)
| Title |
|---|
| 廖战瑞: "VB数据库编程-批量导出数据到Excel", 《软件导刊》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685481A (en) * | 2019-10-17 | 2021-04-20 | 北京京东振世信息技术有限公司 | Data processing method and device |
| CN112685481B (en) * | 2019-10-17 | 2023-11-03 | 北京京东振世信息技术有限公司 | Data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106980669B (en) | A kind of storage of data, acquisition methods and device | |
| CN109002474A (en) | A kind of data migration method and device | |
| CN108205960A (en) | Method and apparatus, electron map manufacturing system and the navigation system that a kind of word renders | |
| CN108052281A (en) | Business Information storage method, application server and computer storage media | |
| CN109657427A (en) | The method and device of permission control | |
| CN109597678A (en) | Task processing method and device | |
| CN108268289A (en) | Method for parameter configuration, the apparatus and system of web applications | |
| CN104182295B (en) | A kind of data back up method and device | |
| CN110457182A (en) | A kind of load balancing cluster example operating index monitoring system | |
| CN110069488A (en) | A kind of date storage method, method for reading data and its device | |
| CN105528381A (en) | Database data migration method and system | |
| CN110245137A (en) | A kind of processing method of index, device and equipment | |
| CN110019293A (en) | List data deriving method and device based on impala database | |
| CN108829790A (en) | A kind of data batch processing method, apparatus and system | |
| CN108073595A (en) | It is a kind of to realize data update and the method and device of snapshot in olap database | |
| TWI534704B (en) | Processing method for time series and system thereof | |
| CN109753157A (en) | The gestural control method and device of display screen | |
| CN109446268A (en) | A kind of method of data synchronization and device | |
| CN106708902A (en) | Database data migration method and system | |
| CN110069453A (en) | Operation/maintenance data treating method and apparatus | |
| CN109388644A (en) | A kind of data-updating method and device | |
| CN108345603A (en) | A kind of SQL statement analysis method and device | |
| CN110134701A (en) | A kind of buffering benefit account method and device | |
| CN108536759B (en) | Sample playback data access method and device | |
| CN108132936A (en) | Data lead-in method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190716 |
|
| RJ01 | Rejection of invention patent application after publication |