CN110019225A - Method, apparatus, equipment and the medium of data processing - Google Patents
Method, apparatus, equipment and the medium of data processing Download PDFInfo
- Publication number
- CN110019225A CN110019225A CN201711392640.1A CN201711392640A CN110019225A CN 110019225 A CN110019225 A CN 110019225A CN 201711392640 A CN201711392640 A CN 201711392640A CN 110019225 A CN110019225 A CN 110019225A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- dimension
- father node
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Abstract
The embodiment of the invention provides a kind of method, apparatus of data processing, equipment and media.This method comprises: acquiring data from database, and collected data are converted into structural data；According to preset data structure, the structural data is pushed to data memory module by storage；The structural data is loaded into the structure tree of classification by the data memory module.Using the technical solution of the embodiment of the present invention, by carrying out structuring to data and structural data being loaded into the structure tree of classification, realize the standardization processing and storage to data, all system and platform can be compatible with, shield the otherness of business, suitable for various businesses, reduces big data on the whole and handle the time it takes.
Description
Technical field
The present invention relates to technical field of data processing more particularly to a kind of method, apparatus of data processing, equipment and Jie
Matter.
Background technique
With the development of big data technology, the visual demand of big data is more and more.Such as: according to the layer of organization
Grade relationship (such as: parent company -> branch company, districts and cities -> district branch company -> small towns) display data, another example is: according to the layer of product
Grade relationship (major class product -> second level subclass product -> three-level subclass product) display data.
Currently, can realize big data visualization by two ways.A kind of mode is: requiring for each business
A set of program is developed, realize the storage of data and is showed.However, being easy to produce in this way, data are inconsistent and data are lonely
Island, and higher cost, the period for developing program are longer.Another way is: using the tool of multidimensional analysis, such as: IBM is public
The cognos provided is provided, is generated cube (i.e. cube multi dimension information model), then foreground carries out multidimensional analysis.However, using
The shortcomings that this mode is will be additional using calculation procedure calculating cube, and the calculating time is long, influences the time for obtaining data.
In conclusion at present there is the big number of processing during handling big data in the visual solution of big data
According to the time of cost longer problem.
Summary of the invention
The embodiment of the invention provides a kind of method, apparatus of data processing, equipment and media, to reduce processing big data
The time it takes.
In a first aspect, the embodiment of the invention provides a kind of method of data processing, method includes:
Data are acquired from database, and collected data are converted into structural data；
The structural data is pushed to data memory module；
The structural data is loaded into the structure tree of classification by the data memory module.
Second aspect, the embodiment of the invention provides a kind of device of data processing, device include: data acquisition module,
Data-pushing module and data memory module；
The data acquisition module is used to acquire data from database, and collected data are converted to structuring number
According to；
The data-pushing module is used to the structural data being pushed to the data memory module；
The data memory module is for the structural data to be loaded into the structure tree of classification.
The third aspect, the embodiment of the invention provides a kind of equipment of data processing, comprising: at least one processor, extremely
A few memory and computer program instructions stored in memory, when computer program instructions are executed by processor
Realize the method such as first aspect in above embodiment.
Fourth aspect, the embodiment of the invention provides a kind of computer readable storage mediums, are stored thereon with computer journey
The method such as first aspect in above embodiment is realized in sequence instruction when computer program instructions are executed by processor.
Method, apparatus, equipment and the medium of data processing provided in an embodiment of the present invention, by carrying out structuring to data
And structural data is loaded into the structure tree of classification, the standardization processing to data is realized, can be compatible with and all be
System and platform, shield the otherness of business, are suitable for various businesses, when reducing big data on the whole and handling spent
Between.Also, the technical solution of the embodiment of the present invention can realize that directly data carry out lower brill analysis, are not necessarily to by recursive algorithm
It is individually created cube multi dimension information model, accelerates the formation speed of analysis data.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, will make below to required in the embodiment of the present invention
Attached drawing is briefly described, for those of ordinary skill in the art, without creative efforts, also
Other drawings may be obtained according to these drawings without any creative labor.
Fig. 1 shows the schematic diagram of the device of data processing provided in an embodiment of the present invention.
Fig. 2 shows the flow charts of the method for data processing provided in an embodiment of the present invention.
Fig. 3 shows another flow chart of the method for data processing provided in an embodiment of the present invention.
Fig. 4 shows the schematic diagram of multiway tree provided in an embodiment of the present invention.
Fig. 5 shows the schematic diagram of data query result provided in an embodiment of the present invention.
Fig. 6 shows the hardware structural diagram of the equipment of data processing provided in an embodiment of the present invention.
Specific embodiment
The feature and exemplary embodiment of various aspects of the invention is described more fully below, in order to make mesh of the invention
, technical solution and advantage be more clearly understood, with reference to the accompanying drawings and embodiments, the present invention is further retouched in detail
It states.It should be understood that specific embodiment described herein is only configured to explain the present invention, it is not configured as limiting the present invention.
To those skilled in the art, the present invention can be real in the case where not needing some details in these details
It applies.Below the description of embodiment is used for the purpose of better understanding the present invention to provide by showing example of the invention.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence " including ... ", it is not excluded that including
There is also other identical elements in the process, method, article or equipment of the element.
Before being illustrated to the method for data processing provided in an embodiment of the present invention, to provided in an embodiment of the present invention
The device of data processing is illustrated.As shown in Figure 1, Fig. 1 shows the device of data processing provided in an embodiment of the present invention
Schematic diagram.The device includes: data acquisition module 101, data-pushing module 102, data memory module 103.Optionally, the dress
Set further includes data inquiry module 104, data display module 105 and database 106.Wherein, the quantity of database can be more
A, Fig. 1 is 3 with the quantity of database, i.e., for database 1061, database 1062 and database 1063.Multiple databases
106 are connected with data acquisition module 101 respectively, and data acquisition module 101 is connected with data-pushing module 102, data-pushing mould
Block 102 is connected with data memory module 103, and data memory module 103 is connected with data inquiry module 104, data inquiry module
104 are connected with data display module 105.The method that the device is used to execute the data processing of embodiment of the present disclosure offer.Below
The process for executing the method for data processing provided in an embodiment of the present invention to the device is illustrated.
As shown in Fig. 2, Fig. 2 shows the flow charts of the method for data processing provided in an embodiment of the present invention.This method packet
Include following steps:
Step S21: data are acquired from database, and collected data are converted into structural data；
Step S22: the structural data is pushed to data memory module；
Step S23: the structural data is loaded into the structure tree of classification by the data memory module.
Step S21 is executed by data acquisition module 101.Data acquisition module 101 acquires data from database 106 first,
The quantity of database 106 can be it is multiple, then data acquisition module 101 repeat from each database acquisition data step
Suddenly.The meaning that data in each database represent is different, content is different and format is different, in order to standardize and standardize, makes
Method for exhibiting data provided in an embodiment of the present invention is suitable for a variety of different databases, data acquisition module 101 is from number
After acquiring data in library 106, collected data are processed, structural data is converted into, in case after
Continuous data-pushing module 102, data memory module 103, data inquiry module 104 and data display module 105 to data into
Row is further processed.
Optionally, in an embodiment of the present invention, step S21 the following steps are included:
The connection that JDBC is established between the database is connected by java database；
Data are acquired from the database；
Collected data are collected according to each dimension, to form data flow；
According to preset structured format, the stream compression is changed to obtain the structural data.
Optionally, in an embodiment of the present invention, data are acquired from the database, comprising:
The interface document generated by JDBC mode or sql like language, acquires data from the data.
Optionally, the preset structured format includes following field: time, keyword, level-one dimension, second level dimension
Degree, three-level dimension, level Four dimension, Pyatyi dimension, value.
Wherein, time of the timetable registration according to generation, keyword (being denoted as keyid) expression index name, such as: sale
Volume.Level-one dimension, second level dimension, three-level dimension, level Four dimension, Pyatyi dimension indicate hierarchical relationship, such as: parent company -> districts and cities
Branch company -> district branch company -> small towns, another example is: major class product -> second level subclass product -> three-level subclass product.Value indicates number
According to.
Illustratively, if 2017 2 month parent company sales volume be 1,000,000 yuan, parent company is divided into 5 branch companies: point
Company a, branch company b, branch company c, branch company d and branch company e, the sales volume of each branch company are 200,000 yuan.It is then corresponding
Structural data it is as follows:
201702|1|999|||||1000000
201702|1|999|a||||200000
201702|1|999|b||||200000
201702|1|999|c||||200000
201702|1|999|d||||200000
201702|1|999|e||||200000
Wherein, field " 201702 | " indicates 2 months in 2017, and field " 1 | " indicates that sales volume, field " 999 | " indicate total
Company's (level-one dimension), field " 1000000 " indicate 1,000,000 yuan, therefore " 201702 | 1 | 999 | | | | | 1000000 " it can table
The sales volume for showing parent company is 1,000,000 yuan, wherein second level dimension, three-level dimension, level Four dimension and Pyatyi dimension are sky,
So after field " 999 | " " | | | | | " it is sky.Similarly, the field " a | " " b | " " c | " " d | " " e | " after field " 999 | "
It indicates branch company (second level dimension), and is the branch company of parent company, so field " a | " " b | " " c | " " d | " and " e | "
It is placed on after field " 999 | ".Therefore " 201702 | 1 | 999 | a | | | | 200000 " it can indicate the branch company a of parent company
Sales volume is 200,000 yuan, wherein three-level dimension, level Four dimension and Pyatyi dimension are sky, so after field " a | " " | |
| | " it is sky.
Fig. 3 shows another flow chart of the method for data processing provided in an embodiment of the present invention.As shown in figure 3, the party
Method includes step S01 to step S08, wherein step S01 to step S06 is executed by data acquisition module 101, step S07
It is to be executed by data-pushing module 102, step S08 is executed by data memory module 103, and step S09 is looked by data
Ask what module 104 executed, step S010 is executed by data display module 105.
Firstly, data acquisition module 101 executes step S01, the connection with database is established by JDBC.
Then, data acquisition module 101 executes step S02, configuration acquisition time, collection period and data to be collected
Library, wherein acquisition time refers to the data which period is acquired from database to be collected, and collection period refers to every more
Data acquisition is carried out to database to be collected for a long time, database to be collected refers to which database to carry out data for adopts
Collection.Then, data memory module 101 executes step S03, selects data acquisition modes.In an embodiment of the present invention, data are adopted
Collection module 101 supports two kinds of data acquisition modes: the first is the mode of JDBC, is for second the mode of interface document.
Then, data memory module 101 executes step S0401 or step S0402 according to the implementing result of step S03.
Wherein, when the implementing result of step S03 is by the way of JDBC, data acquisition module executes step S0401,
Read data from database by the way of java, and Batch sending is to data-pushing module.
When the implementing result of step S03 is by the way of interface document, data acquisition module 101 executes step
S0402 is generated interface document, then interface document is loaded in data-pushing module, in interface document by sql like language
When detecting new data input, new data are imported into data-pushing module from database.
Then, data acquisition module 101 executes step S05, and collected data are collected according to each dimension, with
Form data flow.Then, data acquisition module 101 executes step S06, according to preset structured format, by the data flow
Be converted to structural data.
Then, data-pushing module 102 executes step S07, and the structural data is pushed to data memory module.
Then, data memory module 103 executes step S08 namely step S23, by the data memory module by institute
Data flow is stated to be loaded into the structure tree of classification.
Optionally, in an embodiment of the present invention, the structure tree of the classification is multiway tree, the root node of the multiway tree
Data be dimensions at different levels data summation, sequence of the data of the dimensions at different levels according to dimension level from high to low, point
For cloth on the node of the equivalent layer of the multiway tree, the same layer node of the multiway tree characterizes the data of same grade dimension.
Fig. 4 shows the schematic diagram of multiway tree provided in an embodiment of the present invention.In Fig. 4, (number is 1 namely id to root node
1) corresponding level-one dimension, the data of root node are 1000000, indicate 2017 2 month parent company sales volume be 1,000,000
Member, root node as father node, connect 5 child nodes (id is 2 respectively, 3,4,5, parentid be 1) 5 child nodes
Corresponding second level dimension indicates that parent company is divided into 5 branch companies: branch company a, branch company b, branch company c, branch company d and point public affairs
E is taken charge of, the data of each child node are 200000, indicate that the sales volume of each branch company is 200,000 yuan.
Similarly, as shown in figure 4, in the second layer of multiway tree number be 2 node as father node, connect 2 child nodes
(id is 6 and 7 respectively, and parentid is 2, namely it is 6 and 7) that number, which is the childrenid of 2 node, 2 child nodes
Corresponding three-level dimension indicates that branch company a points are 2 departments, and the data of each child node are 100000, indicate the pin of each department
The volume of selling is 100,000 yuan.
Then, data inquiry module 104 executes step S09, is instructed according to the data query detected, according to recursive algorithm
Extract corresponding data.Optionally, in an embodiment of the present invention, step S09 the following steps are included:
Father's section of the father node is judged for the father node in the multiway tree according to the data query operation detected
Whether point number is empty；
If the father node number of a father node is sky, the data of the father node are extracted；
For the child node in the multiway tree, following steps are executed repeatedly, until the child node is leaf node:
Judge the father node of the child node father node number whether be upper one layer of father node father node number, if the son
The father node number of the father node of node is the father node number of upper one layer of father node, then extracts the data of the child node.
Since the structural data that data memory module 103 is stored meets preset structured format,
So the data query instruction that data inquiry module 104 receives also should correspondingly meet the preset structured format, with
Extract the data for needing to inquire from data memory module 103 according to recursive algorithm convenient for data inquiry module 104.
Illustratively, data inquiry module 104 constructs root data:
SELECT id, name, this month value, last month value, amplification, parentid as null from ST_PX_TREE_
DATA_CDR where key=# { key } and querying condition
ORDER BY current value DESC
104 constructor catalogue data of data inquiry module (id of setting root is the parentid of subdirectory):
SELECT id, name, this month value, last month value, amplification, parentid from ST_PX_TREE_DATA_CDR
Where key=# { key } and querying condition
ORDER BY current value DESC
Then, data inquiry module 104 is according to following business logic processing: in the number for detecting that user inputs in client
After inquiry instruction (parameter of user's input includes: time, keyid etc.), with recursive algorithm root data and specific item
Data staging is recorded to provide；The data inquired are re-configured to data required for data display module.Specific step is such as
Under:
Step a, obtain a father node number (such as: id=1), and judge the father node parentid whether be
Empty (i.e. null) then illustrates that the father node is root node if it is sky, extracts the data of the father node, as some keyid's
Whole aggregate values；
Step b, obtain child node (node for having father node) parentid, judge the child node parentid whether
Equal to the id in step a, if be equal to step a in id (such as: parentid=1), then it represents that the child node be step a in
The child node of father node extracts the data of the child node, the data for as needing to inquire in going down process；
Step c, recursion step b, until child node is leaf node (node of no child node).
D, the Data Concurrent extracted is encapsulated in a manner of json gives data display module.
Then, data display module 105 executes step S10, obtains the data extracted, and show according to the form of table
The data extracted.Data display module 105 obtains data using js technology from data inquiry module 104, and with table
The form (such as: passing through ui-grid control) of lattice is presented to user.Fig. 5 shows data query provided in an embodiment of the present invention
As a result schematic diagram.As shown in figure 5, data display module 105 illustrates the result of data query in table form.
Optionally, in an embodiment of the present invention, the data extracted are obtained, and are mentioned according to described in the displaying of the form of table
The data got, comprising the following steps:
It obtains lower bore and checks instruction；
According to the corresponding relationship of each node of the data and the multiway tree extracted, tieed up step by step according to the form of table
Degree underground drilling shows the data extracted.
Optionally, in an alternative embodiment of the invention, the data extracted are obtained, and according to described in the displaying of the form of table
The data extracted, comprising the following steps:
Obtain the authority information of the lower user for boring and checking that instruction is checked in instruction and the triggering lower brill；
According to the authority information, corresponding dimension limit value is determined；
It is pressed according to the corresponding relationship of each node of the data and the multiway tree extracted in conjunction with the dimension limit value
According to the form data that lower brill displaying is extracted step by step of table.
During data 105 display data of display module, user may also want to understand the data of next stage dimension, because
And lower brill can be inputted on the client and checks operation.It bores after checking instruction, executes as follows under the acquisition of data display module 105
Step:
Step A, into showTree method；
Step B, each node layer is recycled, judges whether the parientid of each node in same node layer is null,
Step C or step D are transferred to according to judging result；
If step C, the parentid of a node is null, for the node at the 0th layer, characterizing the node is root node；
If step D, the parentid of a node is not null, for the node at the 1st layer, characterizing the node is root node
Child node；
Step E, judge again the node childrenid whether non-empty, step F or end are transferred to according to judging result；
If step F, the childrenid non-empty of the node, enters back into showTree method, to the last a node layer
Until, the node layer is without childrenid.
Step G, the data of each node are extracted according to step A to step F, then by data set into ui-grid control
Lower brill is carried out to show.As schemed optionally during carrying out the displaying of lower brill, the authority information of user to be obtained, according to user's
Authority information determines the minimum dimension for allowing the user to check, and then during lower brill is shown, being shown to allows the user
Until the data for the minimum latitude checked.It is guaranteed data security with this, avoids leaking data.
In addition, in conjunction with Fig. 2 embodiment of the present invention described the method that shows of data can data as shown in Figure 1 show
Device realize.As shown in Figure 1, the device includes: data acquisition module, data-pushing module and data memory module；
The data acquisition module is used to acquire data from database, and collected data are converted to structuring number
According to；
The data-pushing module is used to the structural data being pushed to the data memory module；
The data memory module is for the structural data to be loaded into the structure tree of classification.
Optionally, the structure tree of the classification is multiway tree, and the data of the root node of the multiway tree are dimensions at different levels
The summation of data, sequence of the data of the dimensions at different levels according to dimension level from high to low, is distributed in the phase of the multiway tree
It answers on the node of layer, the same layer node of the multiway tree characterizes the data of same grade dimension.
Optionally, described device further include:
Data inquiry module, for extracting corresponding data according to recursive algorithm according to the data query instruction detected；
Data display module, for obtaining the data inquired, and according to the number extracted described in the displaying of the form of table
According to.Optionally, the data inquiry module is used for:
Father's section of the father node is judged for the father node in the multiway tree according to the data query operation detected
Whether point number is empty；
If the father node number of a father node is sky, the data of the father node are extracted；
For the child node in the multiway tree, following steps are executed repeatedly, until the child node is leaf node:
Judge the father node of the child node father node number whether be upper one layer of father node father node number, if the son
The father node number of the father node of node is the father node number of upper one layer of father node, then extracts the data of the child node.
Optionally, the data display module is used for:
It obtains lower bore and checks instruction；
According to the corresponding relationship of each node of the data and the multiway tree extracted, tieed up step by step according to the form of table
Degree underground drilling shows the data extracted.
Optionally, the data display module is used for:
Obtain the authority information of the lower user for boring and checking that operation is checked in instruction and the triggering lower brill；
According to the authority information, corresponding dimension limit value is determined；
It is pressed according to the corresponding relationship of each node of the data and the multiway tree extracted in conjunction with the dimension limit value
According to the form data that lower brill displaying is extracted step by step of table.
Optionally, the data acquisition module is used for:
JDBC is connected by java database to be connected with the database；
Data are acquired from the database；
Collected data are collected according to each dimension, to form data flow；
According to preset structured format, the stream compression is changed to the structural data.
Optionally, the preset structured format includes following field: time, keyword, level-one dimension, second level dimension
Degree, three-level dimension, level Four dimension, Pyatyi dimension, value.
Optionally, the data acquisition module is used for:
The interface document generated by JDBC mode or sql like language, acquires data from the data.
In addition, in conjunction with Fig. 2 describe the embodiment of the present invention data processing method can by the equipment of data processing Lai
It realizes.Fig. 6 shows the hardware structural diagram of the equipment of data processing provided in an embodiment of the present invention.
The equipment that data are shown may include processor 601 and the memory 602 for being stored with computer program instructions.
Specifically, above-mentioned processor 601 may include central processing unit (CPU) or specific integrated circuit
(Application Specific Integrated Circuit, ASIC), or may be configured to implement implementation of the present invention
One or more integrated circuits of example.
Memory 602 may include the mass storage for data or instruction.For example it rather than limits, memory
602 may include hard disk drive (Hard Disk Drive, HDD), floppy disk drive, flash memory, CD, magneto-optic disk, tape or logical
With the combination of universal serial bus (Universal Serial Bus, USB) driver or two or more the above.It is closing
In the case where suitable, memory 602 may include the medium of removable or non-removable (or fixed).In a suitable case, it stores
Device 602 can be inside or outside data processing equipment.In a particular embodiment, memory 602 is nonvolatile solid state storage
Device.In a particular embodiment, memory 602 includes read-only memory (ROM).In a suitable case, which can be mask
ROM, programming ROM (PROM), erasable PROM (EPROM), the electric erasable PROM (EEPROM), electrically-alterable ROM of programming
(EAROM) or the combination of flash memory or two or more the above.
Processor 601 is by reading and executing the computer program instructions stored in memory 602, to realize above-mentioned implementation
The method that any one data in example are shown.
In one example, the equipment that data are shown may also include communication interface 603 and bus 610.Wherein, such as Fig. 6 institute
Show, processor 601, memory 602, communication interface 603 connect by bus 610 and complete mutual communication.
Communication interface 603 is mainly used for realizing in the embodiment of the present invention between each module, device, unit and/or equipment
Communication.
Bus 610 includes hardware, software or both, and the component for the equipment that data are shown is coupled to each other together.Citing
For rather than limit, bus may include accelerated graphics port (AGP) or other graphics bus, enhancing Industry Standard Architecture (EISA)
Bus, front side bus (FSB), super transmission (HT) interconnection, the interconnection of Industry Standard Architecture (ISA) bus, infinite bandwidth, low pin count
(LPC) bus, memory bus, micro- channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express
(PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association part (VLB) bus or other conjunctions
The combination of suitable bus or two or more the above.In a suitable case, bus 610 may include one or more
Bus.Although specific bus has been described and illustrated in the embodiment of the present invention, the present invention considers any suitable bus or interconnection.
The equipment that the data are shown can execute the data processing in the embodiment of the present invention based on the data in database
Method, to realize the method in conjunction with Fig. 2 data processing described.
In addition, in conjunction with the method for the data processing in above-described embodiment, the embodiment of the present invention can provide a kind of computer can
Storage medium is read to realize.Computer program instructions are stored on the computer readable storage medium；The computer program instructions
The method of any one data processing in above-described embodiment is realized when being executed by processor.
It should be clear that the invention is not limited to specific configuration described above and shown in figure and processing.
For brevity, it is omitted here the detailed description to known method.In the above-described embodiments, several tools have been described and illustrated
The step of body, is as example.But method process of the invention is not limited to described and illustrated specific steps, this field
Technical staff can be variously modified, modification and addition after understanding spirit of the invention, or suitable between changing the step
Sequence.
Functional block shown in structures described above block diagram can be implemented as hardware, software, firmware or their group
It closes.When realizing in hardware, it may, for example, be electronic circuit, specific integrated circuit (ASIC), firmware appropriate, insert
Part, function card etc..When being realized with software mode, element of the invention is used to execute program or the generation of required task
Code section.Perhaps code segment can store in machine readable media program or the data-signal by carrying in carrier wave is passing
Defeated medium or communication links are sent." machine readable media " may include any medium for capableing of storage or transmission information.
The example of machine readable media includes electronic circuit, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), soft
Disk, CD-ROM, CD, hard disk, fiber medium, radio frequency (RF) link, etc..Code segment can be via such as internet, inline
The computer network of net etc. is downloaded.
It should also be noted that, the exemplary embodiment referred in the present invention, is retouched based on a series of step or device
State certain methods or system.But the present invention is not limited to the sequence of above-mentioned steps, that is to say, that can be according in embodiment
The sequence referred to executes step, may also be distinct from that the sequence in embodiment or several steps are performed simultaneously.
The above description is merely a specific embodiment, it is apparent to those skilled in the art that,
For convenience of description and succinctly, the system, module of foregoing description and the specific work process of unit can refer to preceding method
Corresponding process in embodiment, details are not described herein.It should be understood that scope of protection of the present invention is not limited thereto, it is any to be familiar with
Those skilled in the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or substitutions,
These modifications or substitutions should be covered by the protection scope of the present invention.
Claims (12)
1. a kind of data processing method characterized by comprising
Data are acquired from database, and collected data are converted into structural data；
The structural data is pushed to data memory module；
The structural data is loaded into the structure tree of classification by the data memory module.
2. the method according to claim 1, wherein the structure tree of the classification is multiway tree, the multiway tree
Root node data be dimensions at different levels data summation, the data of the dimensions at different levels according to dimension level from high to low
Sequentially, it is distributed on the node of the equivalent layer of the multiway tree, the same layer node of the multiway tree characterizes same grade dimension
Data.
3. method according to claim 1 or 2, which is characterized in that by the data memory module by the structure
After change data are loaded into the structure tree of classification, the method also includes:
According to the data query instruction detected, corresponding data are extracted according to recursive algorithm；
The data extracted are obtained, and according to the data extracted described in the displaying of the form of table.
4. according to the method described in claim 3, it is characterized in that, being calculated according to the data query operation detected according to recurrence
Method extracts corresponding data, comprising:
According to the data query operation detected, for the father node in the multiway tree, judge that the father node of the father node is compiled
It number whether is empty；
If the father node number of a father node is sky, the data of the father node are extracted；
For the child node in the multiway tree, following steps are executed repeatedly, until the child node is leaf node:
Judge the father node of the child node father node number whether be upper one layer of father node father node number, if the child node
Father node father node number be upper one layer of father node father node number, then extract the data of the child node.
5. according to the method described in claim 4, it is characterized in that, obtain the data extracted, and according to the form exhibition of table
Show the data extracted, comprising:
It obtains lower bore and checks instruction；
According to the corresponding relationship of the data extracted and each node of the multiway tree, according to table form step by step dimensionally
It is lower to bore the data for showing and extracting.
6. according to the method described in claim 4, it is characterized in that, obtain the data extracted, and according to the form exhibition of table
Show the data extracted, comprising:
Obtain the authority information of the lower user for boring and checking that instruction is checked in instruction and the triggering lower brill；
According to the authority information, corresponding dimension limit value is determined；
According to the corresponding relationship of each node of the data and the multiway tree extracted, in conjunction with the dimension limit value, according to table
The form of lattice is lower step by step to bore the data for showing and extracting.
7. the method according to claim 1, wherein acquiring data from database, and collected data are turned
It is changed to structural data, comprising:
The connection that JDBC is established between the database is connected by java database；
Data are acquired from the database；
Collected data are collected according to each dimension, to form data flow；
According to preset structured format, the stream compression is changed to the structural data.
8. the method according to the description of claim 7 is characterized in that the preset structured format includes following field: when
Between, keyword, level-one dimension, second level dimension, three-level dimension, level Four dimension, Pyatyi dimension, value.
9. the method according to the description of claim 7 is characterized in that acquiring data from the database, comprising:
The interface document generated by JDBC mode or sql like language, acquires data from the data.
10. a kind of device of data processing characterized by comprising
Data acquisition module, data-pushing module and data memory module；
The data acquisition module is used to acquire data from database, and collected data are converted to structural data；
The data-pushing module is used to the structural data being pushed to the data memory module；
The data memory module is for the structural data to be loaded into the structure tree of classification.
11. a kind of equipment of data processing characterized by comprising at least one processor, at least one processor and deposit
The computer program instructions of storage in the memory, are realized such as when the computer program instructions are executed by the processor
Method of any of claims 1-9.
12. a kind of computer readable storage medium, is stored thereon with computer program instructions, which is characterized in that when the calculating
Machine program instruction realizes method as claimed in any one of claims 1-9 wherein when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711392640.1A CN110019225A (en) | 2017-12-21 | 2017-12-21 | Method, apparatus, equipment and the medium of data processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711392640.1A CN110019225A (en) | 2017-12-21 | 2017-12-21 | Method, apparatus, equipment and the medium of data processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110019225A true CN110019225A (en) | 2019-07-16 |
Family
ID=67187037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711392640.1A Pending CN110019225A (en) | 2017-12-21 | 2017-12-21 | Method, apparatus, equipment and the medium of data processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019225A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110362595A (en) * | 2019-07-19 | 2019-10-22 | 北京首汽智行科技有限公司 | A kind of SQL statement dynamic analysis method |
| CN111210879A (en) * | 2020-01-06 | 2020-05-29 | 中国海洋大学 | Hierarchical storage optimization method for super-large-scale drug data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049580A (en) * | 2013-01-17 | 2013-04-17 | 北京工商大学 | Method and device for visualization of layering data |
| CN106227885A (en) * | 2016-08-08 | 2016-12-14 | 星河互联集团有限公司 | Processing method, device and the terminal of a kind of big data |
| US20160378821A1 (en) * | 2009-01-30 | 2016-12-29 | International Business Machines Corporation | Expansion of a tree heirarchy |
| CN106682986A (en) * | 2016-12-27 | 2017-05-17 | 南京搜文信息技术有限公司 | Construction method of complex financial transaction network activity map based on big data |
| CN106776995A (en) * | 2016-12-06 | 2017-05-31 | 北京神舟航天软件技术有限公司 | A kind of tree-like acquisition technique of structural data based on MDA |
-
2017
- 2017-12-21 CN CN201711392640.1A patent/CN110019225A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160378821A1 (en) * | 2009-01-30 | 2016-12-29 | International Business Machines Corporation | Expansion of a tree heirarchy |
| CN103049580A (en) * | 2013-01-17 | 2013-04-17 | 北京工商大学 | Method and device for visualization of layering data |
| CN106227885A (en) * | 2016-08-08 | 2016-12-14 | 星河互联集团有限公司 | Processing method, device and the terminal of a kind of big data |
| CN106776995A (en) * | 2016-12-06 | 2017-05-31 | 北京神舟航天软件技术有限公司 | A kind of tree-like acquisition technique of structural data based on MDA |
| CN106682986A (en) * | 2016-12-27 | 2017-05-17 | 南京搜文信息技术有限公司 | Construction method of complex financial transaction network activity map based on big data |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110362595A (en) * | 2019-07-19 | 2019-10-22 | 北京首汽智行科技有限公司 | A kind of SQL statement dynamic analysis method |
| CN111210879A (en) * | 2020-01-06 | 2020-05-29 | 中国海洋大学 | Hierarchical storage optimization method for super-large-scale drug data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107430611B (en) | Filtering data lineage graph | |
| US10740396B2 (en) | Representing enterprise data in a knowledge graph | |
| CN111522927B (en) | Entity query method and device based on knowledge graph | |
| US20140351241A1 (en) | Identifying and invoking applications based on data in a knowledge graph | |
| JP2003500748A (en) | Method and apparatus for selecting aggregate level and cross-product level in a data warehouse | |
| KR20150079689A (en) | Profiling data with source tracking | |
| CN107341220A (en) | A kind of multi-source data fusion method and device | |
| US20140372433A1 (en) | Analysis of Event Driven Information | |
| US20210334292A1 (en) | System and method for reconciliation of data in multiple systems using permutation matching | |
| US9158599B2 (en) | Programming framework for applications | |
| US20160246705A1 (en) | Data fabrication based on test requirements | |
| CN106649602A (en) | Way, device and server of processing business object data | |
| CN114491172A (en) | Method, device and equipment for quickly searching tree structure nodes and storage medium | |
| CN110019225A (en) | Method, apparatus, equipment and the medium of data processing | |
| EP2019362A2 (en) | Spatial data validation systems | |
| CN115328883A (en) | Data warehouse modeling method and system | |
| CN111061733B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN111831750A (en) | Block chain data analysis method and device, computer equipment and storage medium | |
| CN116097242A (en) | Knowledge graph construction method and device | |
| CN113535966A (en) | Knowledge graph creating method, information obtaining method, device and equipment | |
| CN113918606B (en) | Log query method and device | |
| CN111666278A (en) | Data storage method, data retrieval method, electronic device and storage medium | |
| US20130339372A1 (en) | System and method for contexual ranking of information facets | |
| CN117453856B (en) | Method and device for extracting hold court trial pieces of calendar series based on multi-source data fusion | |
| US11551464B2 (en) | Line based matching of documents |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190716 |