CN109829008A - A kind of data analysis real time visualized method and system - Google Patents
A kind of data analysis real time visualized method and system Download PDFInfo
- Publication number
- CN109829008A CN109829008A CN201811501757.3A CN201811501757A CN109829008A CN 109829008 A CN109829008 A CN 109829008A CN 201811501757 A CN201811501757 A CN 201811501757A CN 109829008 A CN109829008 A CN 109829008A
- Authority
- CN
- China
- Prior art keywords
- node
- data
- data analysis
- analysis model
- implementing result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of data analysis real time visualized method and systems, and wherein data analysis real time visualized method includes the following steps: for data analysis process to be decomposed into data and operational order to be analyzed；Data to be analyzed, as intermediate node, construct Data Analysis Model as start node, operational order and implementing result；The Data Analysis Model is shown on painting canvas；When to show a certain implementing result, the path for reaching the corresponding intermediate node of the implementing result in Data Analysis Model from start node is obtained；All operational orders on the path are successively executed, implementing result to be shown is obtained；The implementing result is shown on painting canvas with visualizing figure.This method can be to the intermediate result and final result progress visualization presentation in real time in data analysis process.
Description
Technical field
The invention belongs to technical field of information processing, and in particular to a kind of pair of data analysis process carries out real-time visual
Method and system.
Background technique
Data visualization is to be opened up data with Visual Chart using technologies such as computer graphics and image procossings
Show, and interacted by chart and user, realizes that the technology of Data Analysis Services, main purpose are intuitively easy using chart
The characteristic understood, by relationship complexity, it is difficult to the data understood are shown with having emphasis, thus to data carry out deeper into sight
It examines and analyzes.
The method for realizing data visualization at present is usually to use data database storing, carries out inquiry and data group from the background
Dress, last webpage front-end generate chart by visualization plug-in unit, and the completions such as report are shown, can not intuitively be checked at each step of data
The intermediate result and overall flow details of reason.
Data visualization common at present rests on mostly shows the fixed form of data with existing, for example opens webpage,
It is immediately seen a pie chart, shows that each province's population accounts for national population ratio or line chart shows a period of time temperature change feelings
Condition.This, which shows, often has the following problems: (1) when data exhibiting demand mode changes, for example by line chart being changed to column
Shape figure needs to modify the variation that code realizes chart ways of presentation or data used, this modification is in the deployed system used
Upper difficult to realize or cost of implementation is higher；(2) when data structure changes, if database amount field is originally whole
Number Int, it is old to show code and be compatible in order to accurately be changed to using decimal Double, it needs to modify from code level again
It shows；(3) user can only see final visualization result, can not understand the process flow of initial data, and defines needs and look into
The data visualization seen.Therefore, ways of presentation fix, non real-time has significant limitation, needs to pay when changing
High maintenance cost.
Summary of the invention
Goal of the invention: aiming at the problems existing in the prior art, the present invention provides a kind of data to analyze real-time visual
Method, this method can in data analysis process intermediate result and final result carry out in real time visualization present.
Technical solution: one aspect of the present invention discloses a kind of data analysis real time visualized method, includes the following steps:
Data analysis process is decomposed into data and operational order to be analyzed；Data to be analyzed are as start node, operation
Order and implementing result construct Data Analysis Model as intermediate node；The Data Analysis Model is shown on painting canvas；
When to show a certain implementing result, obtains in Data Analysis Model and reach the implementing result from start node
The path of corresponding intermediate node；All operational orders on the path are successively executed, implementing result to be shown is obtained；It should
Implementing result is shown on painting canvas with visualizing figure.
On the other hand, the invention discloses a kind of data to analyze real-time visualization systems, including Data Analysis Model is established
Module, path establish module, command execution module and display module；
The Data Analysis Model establishes module and data analysis process is decomposed into data and operational order to be analyzed；Wait divide
Data are analysed as start node, operational order and implementing result as intermediate node, construct Data Analysis Model；The path is built
Formwork erection block obtains the path for reaching target intermediate node in Data Analysis Model from start node；The command execution module
Successively execution route establishes all operational orders on the path of module acquisition, obtains implementing result to be shown；The display mould
Block be used in a manner of visualizing figure on painting canvas display data analysis model and implementing result.
The utility model has the advantages that compared with prior art, data analysis real time visualized method disclosed by the invention and system have
Following advantages: 1, intuitive visualization to whole process that data analyze is carried out by building Data Analysis Model and presented, is convenient for
User's modification, editor's data analysis process；2, each intermediate result in data analysis process can carry out real-time visual presentation.
Detailed description of the invention
Fig. 1 is the flow chart of data analysis real time visualized method disclosed by the invention；
Fig. 2 is the Data Analysis Model structural schematic diagram established in embodiment 1；
Fig. 3 is the composition block diagram of data analysis real-time visualization systems disclosed by the invention；
Fig. 4 is the Data Analysis Model structural schematic diagram established in embodiment 2；
Fig. 5 is the composition block diagram of the analysis real-time visualization systems of data disclosed in embodiment 3.
Specific embodiment
With reference to the accompanying drawings and detailed description, the present invention is furture elucidated.
Embodiment 1:
As shown in Figure 1, analyzing the flow chart of real time visualized method for data disclosed by the invention.The present embodiment is to certain
For the permanent population register information in city is analyzed, to illustrate data analysis real time visualized method disclosed by the invention and be
System.Table name is Base_info in mysql database on the server that IP is IP_str, is remembered respectively in 2 tables of Edu_info
The permanent population register information in the city Lu Liaomou and academic information, it is now desired to use this two tables as data source, count the age
It is shown in the academic information of 20-50 years old permanent resident population, and by statistical result with pie chart.
Firstly the need of data source is imported, the database to be connected is selected according to IP, test connection connects owning user, such as
Fruit information is correct, can get all tables of data having permission；Select table Base_info and Edu_info.It, will in the present embodiment
Pending data is imported in bottom Hive table and is stored, i.e., the data in table Base_info and Edu_info is imported Hive table.Under
Face is analysis and visualization step:
Data analysis process is decomposed into data and operational order to be analyzed by step 1；Data to be analyzed are as starting section
Point, operational order and implementing result construct Data Analysis Model as intermediate node；The Data Analysis Model is shown and is being drawn
On cloth；
In the present embodiment, data to be analyzed are the data in table Base_info and Edu_info, and operational order includes: to look into
The academic information for the population for looking for the age 20-50 years old permanent resident population, statistics to find out.Linked list type as shown in Figure 2 is established as a result,
Data Analysis Model, OP node indicates operational order node in figure, and RST node indicates the execution knot obtained after operational order executes
Fruit node.The start node of model is the data in table Base_info and Edu_info, and first operational order node OP_1 is
Search the age 20-50 years old permanent resident population, implementing result RST_1 is permanent at 20-50 years old for this city institute's has age of lookup
Population；Second running node OP_2 is the academic information for counting the population in RST_1, implementing result RST_2；Terminal node
Point is final result, i.e., generates pie chart to the data in RST_2.By editing the corresponding order of each operational order node, setting
The order of connection of each node obtains Data Analysis Model, and the model of foundation is shown on painting canvas.It is each to save in the present embodiment
The order of connection of point is showed by the preposition node of this node, and start node does not have preposition node.
The query facility that Hive is provided is based on MapReduce, and inquiry velocity is slower.In order to improve inquiry velocity,
The present embodiment is completed to inquire using presto memory-based.
Step 2, when to show a certain implementing result, obtain and reach this in Data Analysis Model from start node and hold
The path of the corresponding intermediate node of row result；All operational orders on the path are successively executed, obtain to be shown executing knot
Fruit；The implementing result is shown on painting canvas with visualizing figure.
Such as to check the data in RST_2, the path of acquisition is to pass through OP_1, RST_1, OP_2 from start node, successively
The corresponding sql sentence of OP_1 and OP_2 on execution route, obtains implementing result RST_2, generates visualization figure to RST_2 and shows
On painting canvas.It requires to show statistical result with pie chart in the present embodiment, so generating pie chart according to RST_2 to get to finally
As a result.Similarly, as long as checking that the data of RST_1 execute OP_1.In this way, user can check in data analysis process
Between as a result, understand analysis process flow.It shows that the visualization figure of data uses editable mode, is selected according to different data
Different visual means, such as curve, pie chart, histogram, network structure mode.
Embodiment 2:
If Fig. 3 is that the realization data provided in the present embodiment analyze the system composition block diagram of real-time visual, including data
Analysis model establishes module, module is established in path, command execution module and display module.
Data Analysis Model establishes module and data analysis process is decomposed into data and operational order to be analyzed；Number to be analyzed
, as intermediate node, Data Analysis Model is constructed according to as start node, operational order and implementing result；Path is established module and is obtained
Take the path for reaching target intermediate node in Data Analysis Model from start node；Command execution module successively execution route
All operational orders on the path of module acquisition are established, implementing result to be shown is obtained；Display module is used for visualize figure
Mode on painting canvas display data analysis model and implementing result.Each module can be deployed on same computer；It can also
Display module is deployed in front-end server, remaining each module is deployed on background server.
The linked list type Data Analysis Model established in embodiment 1 is unable to satisfy complicated data analysis, to this present embodiment
Operational order node may have multiple preposition nodes in the Data Analysis Model of foundation, i.e., the operational order be to multiple data into
Capable；One implementing result may also have multiple operational orders to carry out different processing to it, i.e., multiple operational order nodes
Preposition node is identical, as shown in Figure 4.Finally obtain multiple as a result, carrying out the visualization exhibition of various angles to the analysis of entire data
Show.
Data Analysis Model establishes module by addition, editor's node, and the operational order of operational order node is arranged, executes
The preposition node of the visual means of result node, each intermediate node, to create Data Analysis Model, and by display module with
The form of visualization figure is shown.Data to be analyzed can be obtained from database, can also be obtained by reading file.Aobvious
When showing a certain implementing result, path is established in module acquisition Data Analysis Model and reaches target intermediate node from start node
Path；Successively execution route establishes all operational orders on the path of module acquisition to command execution module, obtains to be shown
Implementing result；Display module obtains implementing result to be shown, it is shown on painting canvas in the form of visualizing figure.
Embodiment 3:
It include implementing if Fig. 5 is the system composition block diagram that realization data provided in this embodiment analyze real-time visual
Each comprising modules further include outside Data Analysis Model preserving module and Data Analysis Model recovery module in the system that example 2 provides.
Data Analysis Model preserving module is for each node order of connection of preservation model, the data to be analyzed in back end, operation life
Enable the corresponding operational order of node.Data Analysis Model recovery module restores data according to each node order of connection of model and analyzes mould
The structure of type, and it is corresponding with each node with operational order to be analysed to data.After restoring to data analysis execution module,
By the data to be analyzed imported in back end, data analysis can be re-executed, and in real time may be used to the progress of each implementing result
It is presented depending on changing.
Data Analysis Model preserving module can also save corresponding implementing result data of each implementing result node and visual
Change figure；The Data Analysis Model of preservation restore to further include by the data of each implementing result node and visualization figure and executing
Result node is corresponding.Data Analysis Model can also be reviewed in a manner of visual in the terminal for not having data analysis in this way
With each implementing result.
Claims (10)
1. a kind of data analyze real time visualized method, which comprises the steps of:
Data analysis process is decomposed into data and operational order to be analyzed；Data to be analyzed are as start node, operational order
With implementing result as intermediate node, Data Analysis Model is constructed；The Data Analysis Model is shown on painting canvas；
When to show a certain implementing result, the implementing result is reached from start node in acquisition Data Analysis Model and is corresponded to
Intermediate node path；All operational orders on the path are successively executed, implementing result to be shown is obtained；By the execution
As a result it is shown on painting canvas with visualization figure.
2. data according to claim 1 analyze real time visualized method, which is characterized in that the Data Analysis Model
In, data to be analyzed are source node, and the preposition node of operational order node is source node and/or implementing result node, execute knot
The preposition node of fruit node is running node.
3. data according to claim 1 analyze real time visualized method, which is characterized in that further include saving data analysis
Model, including the data to be analyzed in each node order of connection of preservation model, back end, the corresponding operation of operational order node
Order.
4. data according to claim 3 analyze real time visualized method, which is characterized in that further include the data to preservation
Analysis model is restored, and specifically includes: restoring the structure of Data Analysis Model according to each node order of connection of model；It will be wait divide
It is corresponding with each node with operational order to analyse data.
5. data according to claim 4 analyze real time visualized method, which is characterized in that save Data Analysis Model also
Including saving the corresponding implementing result data of each implementing result node and visualization figure；The Data Analysis Model of preservation is carried out extensive
It recovers including the data of each implementing result node and visualization figure are corresponding with implementing result node.
6. a kind of data analyze real-time visualization systems, which is characterized in that establish module including Data Analysis Model, path is established
Module, command execution module and display module；
The Data Analysis Model establishes module and data analysis process is decomposed into data and operational order to be analyzed；Number to be analyzed
, as intermediate node, Data Analysis Model is constructed according to as start node, operational order and implementing result；Mould is established in the path
Block obtains the path for reaching target intermediate node in Data Analysis Model from start node；The command execution module is successively
Execution route establishes all operational orders on the path of module acquisition, obtains implementing result to be shown；The display module is used
In in a manner of visualizing figure on painting canvas display data analysis model and implementing result.
7. data according to claim 6 analyze real-time visualization systems, which is characterized in that the Data Analysis Model
In, data to be analyzed are source node, and the preposition node of operational order node is source node and/or implementing result node, execute knot
The preposition node of fruit node is running node.
8. data according to claim 6 analyze real-time visualization systems, which is characterized in that further include Data Analysis Model
Preserving module, the Data Analysis Model preserving module for each node order of connection of preservation model, in back end to point
Analyse data, the corresponding operational order of operational order node.
9. data according to claim 8 analyze real-time visualization systems, which is characterized in that further include Data Analysis Model
Recovery module, the Data Analysis Model recovery module restore the knot of Data Analysis Model according to each node order of connection of model
Structure, and it is corresponding with each node with operational order to be analysed to data.
10. data according to claim 9 analyze real-time visualization systems, which is characterized in that the Data Analysis Model
Preserving module is also used to save the corresponding implementing result data of each implementing result node and visualization figure；The Data Analysis Model
Recovery module is also used to the data of each implementing result node and visualization figure are corresponding with implementing result node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811501757.3A CN109829008A (en) | 2018-12-10 | 2018-12-10 | A kind of data analysis real time visualized method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811501757.3A CN109829008A (en) | 2018-12-10 | 2018-12-10 | A kind of data analysis real time visualized method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109829008A true CN109829008A (en) | 2019-05-31 |
Family
ID=66859512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811501757.3A Pending CN109829008A (en) | 2018-12-10 | 2018-12-10 | A kind of data analysis real time visualized method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109829008A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182017A (en) * | 2020-09-30 | 2021-01-05 | 天津大学 | Method for optimizing data exploration based on user interaction process |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108762748A (en) * | 2018-05-22 | 2018-11-06 | 郑州云海信息技术有限公司 | A kind of method for exhibiting data and system based on data center |
| CN108874487A (en) * | 2018-06-13 | 2018-11-23 | 北京九章云极科技有限公司 | Data analysis processing method and system based on workflow |
-
2018
- 2018-12-10 CN CN201811501757.3A patent/CN109829008A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108762748A (en) * | 2018-05-22 | 2018-11-06 | 郑州云海信息技术有限公司 | A kind of method for exhibiting data and system based on data center |
| CN108874487A (en) * | 2018-06-13 | 2018-11-23 | 北京九章云极科技有限公司 | Data analysis processing method and system based on workflow |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182017A (en) * | 2020-09-30 | 2021-01-05 | 天津大学 | Method for optimizing data exploration based on user interaction process |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9466041B2 (en) | User selected flow graph modification | |
| CN103106188A (en) | Imaging analysis system and imaging analysis method of data model | |
| CN102222043B (en) | Testing method and testing device | |
| WO2021190379A1 (en) | Method and device for realizing automatic machine learning | |
| CN104573231A (en) | BIM based smart building system and method | |
| CN108053486A (en) | Construction method and system based on virtual technology | |
| US8918358B2 (en) | Information integration flow freshness cost | |
| CN103970646B (en) | A kind of automatic analysis method for the sequence of operation and system thereof | |
| CN109101590A (en) | Configuration item CI topological relation method for visualizing based on CMDB | |
| Alzraiee | Cost estimate system using structured query language in BIM | |
| US20130093771A1 (en) | Modified flow graph depiction | |
| CN109189865A (en) | The overall analysis system and method for project data | |
| CN114090601B (en) | Data screening method, device, equipment and storage medium | |
| CN111259090A (en) | Graph generation method and device of relational data, electronic equipment and storage medium | |
| CN109829008A (en) | A kind of data analysis real time visualized method and system | |
| CN111881301B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110389944B (en) | Metadata management system and method based on model | |
| CN111178615A (en) | Construction method and system of enterprise risk identification model | |
| WO2020147335A1 (en) | Method and system for clustering member data on electronic commerce platform | |
| CN104463694B (en) | A kind of data circulate the method and system that represent of distribution in information system | |
| CN115186738B (en) | Model training method, device and storage medium | |
| CN108132802B (en) | Application method of configuration model in system development | |
| CN110222032A (en) | A kind of generalised event model based on software data analysis | |
| CN112380849B (en) | Method and device for generating interest point extraction model and extracting interest points | |
| CN115169578A (en) | AI model production method and system based on meta-space data markers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190531 |
|
| RJ01 | Rejection of invention patent application after publication |