CN109829008A - A kind of data analysis real time visualized method and system - Google Patents

A kind of data analysis real time visualized method and system Download PDF

Info

Publication number
CN109829008A
CN109829008A CN201811501757.3A CN201811501757A CN109829008A CN 109829008 A CN109829008 A CN 109829008A CN 201811501757 A CN201811501757 A CN 201811501757A CN 109829008 A CN109829008 A CN 109829008A
Authority
CN
China
Prior art keywords
node
data
data analysis
analysis model
implementing result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811501757.3A
Other languages
Chinese (zh)
Inventor
王海荣
肖万来
刘娟
刘飞
彭耀栋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhongke Dawning Nanjing Research Institute Co Ltd
Original Assignee
Zhongke Dawning Nanjing Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhongke Dawning Nanjing Research Institute Co Ltd filed Critical Zhongke Dawning Nanjing Research Institute Co Ltd
Priority to CN201811501757.3A priority Critical patent/CN109829008A/en
Publication of CN109829008A publication Critical patent/CN109829008A/en
Pending legal-status Critical Current

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a kind of data analysis real time visualized method and systems, and wherein data analysis real time visualized method includes the following steps: for data analysis process to be decomposed into data and operational order to be analyzed;Data to be analyzed, as intermediate node, construct Data Analysis Model as start node, operational order and implementing result;The Data Analysis Model is shown on painting canvas;When to show a certain implementing result, the path for reaching the corresponding intermediate node of the implementing result in Data Analysis Model from start node is obtained;All operational orders on the path are successively executed, implementing result to be shown is obtained;The implementing result is shown on painting canvas with visualizing figure.This method can be to the intermediate result and final result progress visualization presentation in real time in data analysis process.

Description

A kind of data analysis real time visualized method and system
Technical field
The invention belongs to technical field of information processing, and in particular to a kind of pair of data analysis process carries out real-time visual Method and system.
Background technique
Data visualization is to be opened up data with Visual Chart using technologies such as computer graphics and image procossings Show, and interacted by chart and user, realizes that the technology of Data Analysis Services, main purpose are intuitively easy using chart The characteristic understood, by relationship complexity, it is difficult to the data understood are shown with having emphasis, thus to data carry out deeper into sight It examines and analyzes.
The method for realizing data visualization at present is usually to use data database storing, carries out inquiry and data group from the background Dress, last webpage front-end generate chart by visualization plug-in unit, and the completions such as report are shown, can not intuitively be checked at each step of data The intermediate result and overall flow details of reason.
Data visualization common at present rests on mostly shows the fixed form of data with existing, for example opens webpage, It is immediately seen a pie chart, shows that each province's population accounts for national population ratio or line chart shows a period of time temperature change feelings Condition.This, which shows, often has the following problems: (1) when data exhibiting demand mode changes, for example by line chart being changed to column Shape figure needs to modify the variation that code realizes chart ways of presentation or data used, this modification is in the deployed system used Upper difficult to realize or cost of implementation is higher;(2) when data structure changes, if database amount field is originally whole Number Int, it is old to show code and be compatible in order to accurately be changed to using decimal Double, it needs to modify from code level again It shows;(3) user can only see final visualization result, can not understand the process flow of initial data, and defines needs and look into The data visualization seen.Therefore, ways of presentation fix, non real-time has significant limitation, needs to pay when changing High maintenance cost.
Summary of the invention
Goal of the invention: aiming at the problems existing in the prior art, the present invention provides a kind of data to analyze real-time visual Method, this method can in data analysis process intermediate result and final result carry out in real time visualization present.
Technical solution: one aspect of the present invention discloses a kind of data analysis real time visualized method, includes the following steps:
Data analysis process is decomposed into data and operational order to be analyzed;Data to be analyzed are as start node, operation Order and implementing result construct Data Analysis Model as intermediate node;The Data Analysis Model is shown on painting canvas;
When to show a certain implementing result, obtains in Data Analysis Model and reach the implementing result from start node The path of corresponding intermediate node;All operational orders on the path are successively executed, implementing result to be shown is obtained;It should Implementing result is shown on painting canvas with visualizing figure.
On the other hand, the invention discloses a kind of data to analyze real-time visualization systems, including Data Analysis Model is established Module, path establish module, command execution module and display module;
The Data Analysis Model establishes module and data analysis process is decomposed into data and operational order to be analyzed;Wait divide Data are analysed as start node, operational order and implementing result as intermediate node, construct Data Analysis Model;The path is built Formwork erection block obtains the path for reaching target intermediate node in Data Analysis Model from start node;The command execution module Successively execution route establishes all operational orders on the path of module acquisition, obtains implementing result to be shown;The display mould Block be used in a manner of visualizing figure on painting canvas display data analysis model and implementing result.
The utility model has the advantages that compared with prior art, data analysis real time visualized method disclosed by the invention and system have Following advantages: 1, intuitive visualization to whole process that data analyze is carried out by building Data Analysis Model and presented, is convenient for User's modification, editor's data analysis process;2, each intermediate result in data analysis process can carry out real-time visual presentation.
Detailed description of the invention
Fig. 1 is the flow chart of data analysis real time visualized method disclosed by the invention;
Fig. 2 is the Data Analysis Model structural schematic diagram established in embodiment 1;
Fig. 3 is the composition block diagram of data analysis real-time visualization systems disclosed by the invention;
Fig. 4 is the Data Analysis Model structural schematic diagram established in embodiment 2;
Fig. 5 is the composition block diagram of the analysis real-time visualization systems of data disclosed in embodiment 3.
Specific embodiment
With reference to the accompanying drawings and detailed description, the present invention is furture elucidated.
Embodiment 1:
As shown in Figure 1, analyzing the flow chart of real time visualized method for data disclosed by the invention.The present embodiment is to certain For the permanent population register information in city is analyzed, to illustrate data analysis real time visualized method disclosed by the invention and be System.Table name is Base_info in mysql database on the server that IP is IP_str, is remembered respectively in 2 tables of Edu_info The permanent population register information in the city Lu Liaomou and academic information, it is now desired to use this two tables as data source, count the age It is shown in the academic information of 20-50 years old permanent resident population, and by statistical result with pie chart.
Firstly the need of data source is imported, the database to be connected is selected according to IP, test connection connects owning user, such as Fruit information is correct, can get all tables of data having permission;Select table Base_info and Edu_info.It, will in the present embodiment Pending data is imported in bottom Hive table and is stored, i.e., the data in table Base_info and Edu_info is imported Hive table.Under Face is analysis and visualization step:
Data analysis process is decomposed into data and operational order to be analyzed by step 1;Data to be analyzed are as starting section Point, operational order and implementing result construct Data Analysis Model as intermediate node;The Data Analysis Model is shown and is being drawn On cloth;
In the present embodiment, data to be analyzed are the data in table Base_info and Edu_info, and operational order includes: to look into The academic information for the population for looking for the age 20-50 years old permanent resident population, statistics to find out.Linked list type as shown in Figure 2 is established as a result, Data Analysis Model, OP node indicates operational order node in figure, and RST node indicates the execution knot obtained after operational order executes Fruit node.The start node of model is the data in table Base_info and Edu_info, and first operational order node OP_1 is Search the age 20-50 years old permanent resident population, implementing result RST_1 is permanent at 20-50 years old for this city institute's has age of lookup Population;Second running node OP_2 is the academic information for counting the population in RST_1, implementing result RST_2;Terminal node Point is final result, i.e., generates pie chart to the data in RST_2.By editing the corresponding order of each operational order node, setting The order of connection of each node obtains Data Analysis Model, and the model of foundation is shown on painting canvas.It is each to save in the present embodiment The order of connection of point is showed by the preposition node of this node, and start node does not have preposition node.
The query facility that Hive is provided is based on MapReduce, and inquiry velocity is slower.In order to improve inquiry velocity, The present embodiment is completed to inquire using presto memory-based.
Step 2, when to show a certain implementing result, obtain and reach this in Data Analysis Model from start node and hold The path of the corresponding intermediate node of row result;All operational orders on the path are successively executed, obtain to be shown executing knot Fruit;The implementing result is shown on painting canvas with visualizing figure.
Such as to check the data in RST_2, the path of acquisition is to pass through OP_1, RST_1, OP_2 from start node, successively The corresponding sql sentence of OP_1 and OP_2 on execution route, obtains implementing result RST_2, generates visualization figure to RST_2 and shows On painting canvas.It requires to show statistical result with pie chart in the present embodiment, so generating pie chart according to RST_2 to get to finally As a result.Similarly, as long as checking that the data of RST_1 execute OP_1.In this way, user can check in data analysis process Between as a result, understand analysis process flow.It shows that the visualization figure of data uses editable mode, is selected according to different data Different visual means, such as curve, pie chart, histogram, network structure mode.
Embodiment 2:
If Fig. 3 is that the realization data provided in the present embodiment analyze the system composition block diagram of real-time visual, including data Analysis model establishes module, module is established in path, command execution module and display module.
Data Analysis Model establishes module and data analysis process is decomposed into data and operational order to be analyzed;Number to be analyzed , as intermediate node, Data Analysis Model is constructed according to as start node, operational order and implementing result;Path is established module and is obtained Take the path for reaching target intermediate node in Data Analysis Model from start node;Command execution module successively execution route All operational orders on the path of module acquisition are established, implementing result to be shown is obtained;Display module is used for visualize figure Mode on painting canvas display data analysis model and implementing result.Each module can be deployed on same computer;It can also Display module is deployed in front-end server, remaining each module is deployed on background server.
The linked list type Data Analysis Model established in embodiment 1 is unable to satisfy complicated data analysis, to this present embodiment Operational order node may have multiple preposition nodes in the Data Analysis Model of foundation, i.e., the operational order be to multiple data into Capable;One implementing result may also have multiple operational orders to carry out different processing to it, i.e., multiple operational order nodes Preposition node is identical, as shown in Figure 4.Finally obtain multiple as a result, carrying out the visualization exhibition of various angles to the analysis of entire data Show.
Data Analysis Model establishes module by addition, editor's node, and the operational order of operational order node is arranged, executes The preposition node of the visual means of result node, each intermediate node, to create Data Analysis Model, and by display module with The form of visualization figure is shown.Data to be analyzed can be obtained from database, can also be obtained by reading file.Aobvious When showing a certain implementing result, path is established in module acquisition Data Analysis Model and reaches target intermediate node from start node Path;Successively execution route establishes all operational orders on the path of module acquisition to command execution module, obtains to be shown Implementing result;Display module obtains implementing result to be shown, it is shown on painting canvas in the form of visualizing figure.
Embodiment 3:
It include implementing if Fig. 5 is the system composition block diagram that realization data provided in this embodiment analyze real-time visual Each comprising modules further include outside Data Analysis Model preserving module and Data Analysis Model recovery module in the system that example 2 provides. Data Analysis Model preserving module is for each node order of connection of preservation model, the data to be analyzed in back end, operation life Enable the corresponding operational order of node.Data Analysis Model recovery module restores data according to each node order of connection of model and analyzes mould The structure of type, and it is corresponding with each node with operational order to be analysed to data.After restoring to data analysis execution module, By the data to be analyzed imported in back end, data analysis can be re-executed, and in real time may be used to the progress of each implementing result It is presented depending on changing.
Data Analysis Model preserving module can also save corresponding implementing result data of each implementing result node and visual Change figure;The Data Analysis Model of preservation restore to further include by the data of each implementing result node and visualization figure and executing Result node is corresponding.Data Analysis Model can also be reviewed in a manner of visual in the terminal for not having data analysis in this way With each implementing result.

Claims (10)

1. a kind of data analyze real time visualized method, which comprises the steps of:
Data analysis process is decomposed into data and operational order to be analyzed;Data to be analyzed are as start node, operational order With implementing result as intermediate node, Data Analysis Model is constructed;The Data Analysis Model is shown on painting canvas;
When to show a certain implementing result, the implementing result is reached from start node in acquisition Data Analysis Model and is corresponded to Intermediate node path;All operational orders on the path are successively executed, implementing result to be shown is obtained;By the execution As a result it is shown on painting canvas with visualization figure.
2. data according to claim 1 analyze real time visualized method, which is characterized in that the Data Analysis Model In, data to be analyzed are source node, and the preposition node of operational order node is source node and/or implementing result node, execute knot The preposition node of fruit node is running node.
3. data according to claim 1 analyze real time visualized method, which is characterized in that further include saving data analysis Model, including the data to be analyzed in each node order of connection of preservation model, back end, the corresponding operation of operational order node Order.
4. data according to claim 3 analyze real time visualized method, which is characterized in that further include the data to preservation Analysis model is restored, and specifically includes: restoring the structure of Data Analysis Model according to each node order of connection of model;It will be wait divide It is corresponding with each node with operational order to analyse data.
5. data according to claim 4 analyze real time visualized method, which is characterized in that save Data Analysis Model also Including saving the corresponding implementing result data of each implementing result node and visualization figure;The Data Analysis Model of preservation is carried out extensive It recovers including the data of each implementing result node and visualization figure are corresponding with implementing result node.
6. a kind of data analyze real-time visualization systems, which is characterized in that establish module including Data Analysis Model, path is established Module, command execution module and display module;
The Data Analysis Model establishes module and data analysis process is decomposed into data and operational order to be analyzed;Number to be analyzed , as intermediate node, Data Analysis Model is constructed according to as start node, operational order and implementing result;Mould is established in the path Block obtains the path for reaching target intermediate node in Data Analysis Model from start node;The command execution module is successively Execution route establishes all operational orders on the path of module acquisition, obtains implementing result to be shown;The display module is used In in a manner of visualizing figure on painting canvas display data analysis model and implementing result.
7. data according to claim 6 analyze real-time visualization systems, which is characterized in that the Data Analysis Model In, data to be analyzed are source node, and the preposition node of operational order node is source node and/or implementing result node, execute knot The preposition node of fruit node is running node.
8. data according to claim 6 analyze real-time visualization systems, which is characterized in that further include Data Analysis Model Preserving module, the Data Analysis Model preserving module for each node order of connection of preservation model, in back end to point Analyse data, the corresponding operational order of operational order node.
9. data according to claim 8 analyze real-time visualization systems, which is characterized in that further include Data Analysis Model Recovery module, the Data Analysis Model recovery module restore the knot of Data Analysis Model according to each node order of connection of model Structure, and it is corresponding with each node with operational order to be analysed to data.
10. data according to claim 9 analyze real-time visualization systems, which is characterized in that the Data Analysis Model Preserving module is also used to save the corresponding implementing result data of each implementing result node and visualization figure;The Data Analysis Model Recovery module is also used to the data of each implementing result node and visualization figure are corresponding with implementing result node.
CN201811501757.3A 2018-12-10 2018-12-10 A kind of data analysis real time visualized method and system Pending CN109829008A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811501757.3A CN109829008A (en) 2018-12-10 2018-12-10 A kind of data analysis real time visualized method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811501757.3A CN109829008A (en) 2018-12-10 2018-12-10 A kind of data analysis real time visualized method and system

Publications (1)

Publication Number Publication Date
CN109829008A true CN109829008A (en) 2019-05-31

Family

ID=66859512

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811501757.3A Pending CN109829008A (en) 2018-12-10 2018-12-10 A kind of data analysis real time visualized method and system

Country Status (1)

Country Link
CN (1) CN109829008A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112182017A (en) * 2020-09-30 2021-01-05 天津大学 Method for optimizing data exploration based on user interaction process

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108762748A (en) * 2018-05-22 2018-11-06 郑州云海信息技术有限公司 A kind of method for exhibiting data and system based on data center
CN108874487A (en) * 2018-06-13 2018-11-23 北京九章云极科技有限公司 Data analysis processing method and system based on workflow

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108762748A (en) * 2018-05-22 2018-11-06 郑州云海信息技术有限公司 A kind of method for exhibiting data and system based on data center
CN108874487A (en) * 2018-06-13 2018-11-23 北京九章云极科技有限公司 Data analysis processing method and system based on workflow

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112182017A (en) * 2020-09-30 2021-01-05 天津大学 Method for optimizing data exploration based on user interaction process

Similar Documents

Publication Publication Date Title
US9466041B2 (en) User selected flow graph modification
CN103106188A (en) Imaging analysis system and imaging analysis method of data model
CN102222043B (en) Testing method and testing device
WO2021190379A1 (en) Method and device for realizing automatic machine learning
CN104573231A (en) BIM based smart building system and method
CN108053486A (en) Construction method and system based on virtual technology
US8918358B2 (en) Information integration flow freshness cost
CN103970646B (en) A kind of automatic analysis method for the sequence of operation and system thereof
CN109101590A (en) Configuration item CI topological relation method for visualizing based on CMDB
Alzraiee Cost estimate system using structured query language in BIM
US20130093771A1 (en) Modified flow graph depiction
CN109189865A (en) The overall analysis system and method for project data
CN114090601B (en) Data screening method, device, equipment and storage medium
CN111259090A (en) Graph generation method and device of relational data, electronic equipment and storage medium
CN109829008A (en) A kind of data analysis real time visualized method and system
CN111881301B (en) Data processing method and device, electronic equipment and storage medium
CN110389944B (en) Metadata management system and method based on model
CN111178615A (en) Construction method and system of enterprise risk identification model
WO2020147335A1 (en) Method and system for clustering member data on electronic commerce platform
CN104463694B (en) A kind of data circulate the method and system that represent of distribution in information system
CN115186738B (en) Model training method, device and storage medium
CN108132802B (en) Application method of configuration model in system development
CN110222032A (en) A kind of generalised event model based on software data analysis
CN112380849B (en) Method and device for generating interest point extraction model and extracting interest points
CN115169578A (en) AI model production method and system based on meta-space data markers

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20190531

RJ01 Rejection of invention patent application after publication