CN112783869A - Big data visualization modeling and result display based method - Google Patents
Big data visualization modeling and result display based method Download PDFInfo
- Publication number
- CN112783869A CN112783869A CN202110220103.9A CN202110220103A CN112783869A CN 112783869 A CN112783869 A CN 112783869A CN 202110220103 A CN202110220103 A CN 202110220103A CN 112783869 A CN112783869 A CN 112783869A
- Authority
- CN
- China
- Prior art keywords
- data
- big data
- layer
- spark
- web application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000013079 data visualisation Methods 0.000 title claims abstract description 10
- 238000005516 engineering process Methods 0.000 claims abstract description 23
- 238000005314 correlation function Methods 0.000 claims description 4
- 230000002085 persistent effect Effects 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 238000012423 maintenance Methods 0.000 abstract description 3
- 238000013499 data model Methods 0.000 abstract description 2
- 238000012216 screening Methods 0.000 abstract 1
- 238000011161 development Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
- G06F16/212—Schema design and management with details for data modelling support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
- G06F8/22—Procedural
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/31—Programming languages or programming paradigms
- G06F8/315—Object-oriented languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/38—Creation or generation of source code for implementing user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- User Interface Of Digital Computer (AREA)
- Stored Programmes (AREA)
Abstract
A method based on big data visualization modeling and result display is provided. The invention relates to the field of big data application, in particular to a data matching method for screening, cross-comparing and obtaining a query result. And returning the data obtained by the query through the big data application components spark and hive and connecting the data with the data application layer. The application of the big data-based modeling technology provided by the invention solves the problems that the user demand is changeable and the data model cannot adapt to changeable demands. On the page display layer, a user clicks the conditions on the page, the condition data is returned to the data application layer, the data application layer inquires data through the big data spark, hive and other architecture technologies and returns the result set meeting the user-defined conditions, developers do not need to redevelop the application program due to the change of user requirements, the cost of the developers is reduced, and the application maintenance cost is greatly reduced.
Description
Technical Field
The invention relates to the field of big data application, in particular to a method based on big data visual modeling and result display.
Background
In the existing big data application data modeling technology, most of the technologies still meet the business requirements of customers through a back-end code development data model, and after the business requirements of the customers are changed, the business requirements of the customers can only be adapted to the change of the requirements of the customers by modifying programs and algorithms, so that the target requirements of the customers and the convenience of operation are difficult to meet quickly and efficiently. Based on a big data architecture, visual and self-defined data modeling is particularly important for meeting flexible and changeable customer requirements and business data mining, and the method has the advantages of challenging realization of developers, certain complexity and high development difficulty. Therefore, new techniques need to be introduced to optimize existing problems.
Disclosure of Invention
The invention aims to provide a method for visual modeling and result display based on big data, which is easy to use, high in query speed and high in query efficiency, meets the user requirements, can customize a desired query result set, reduces the cost of developers, and greatly reduces the application and maintenance cost.
In order to solve the technical problems, the method for modeling and displaying results based on big data visualization comprises a big data application framework, a data application layer and a page display layer, wherein a data source is a big data environment consisting of spark, hive and mysql, the data application layer is a web application program, the page display layer consists of visualized, editable and draggable JointJS + JqueryUI + BootBox front-end technologies, and the method comprises the following steps:
s1, connecting and debugging a big data environment consisting of spark, hive and mysql with the big data environment through a springboot technology frame and a MyBatis data persistent layer frame to enable a Web application program to be communicated with the big data environment;
s2, defining a page structure by using a front-end bootstrap frame, customizing and producing a public component based on JavaScript, dragging a data source or a model to a canvas area based on JQueryUI, and generating an element;
s3, dynamically connecting all elements in the canvas area by using a JointJS dynamic flow chart, wherein a data source element can only be directly connected with a query condition functional element, a dragged and dropped model is directly connected with any functional element, and any intermediate generated data source element can solidify memory data to the local;
s4, editing the canvas element by using a bootbox.js, inputting a user-defined query condition, binding the data source element, and establishing a node data source element association relation;
s5, connecting a page display layer realized by using the technologies of JointJS, JqueryUI, BootBox, Twitter BootStrap and the like with a Web application program through a springboot technology frame, and debugging and testing;
s6, the collision comparison system provides a model correlation function, the conditions selected by the page are sent to a web application layer, the web application layer connects the obtained conditions with a big data environment built by spark, live and mysql through springboot + mybasic to search data, a result set is returned to a page display layer to be displayed, and a process model and a result model of the operation among a series of data elements of a canvas panel are stored and are ready for the next use.
Further, the Web application program is edited by adopting a Java language architecture.
Furthermore, the data source is one or more of large data environments composed of spark, hive and mysql.
Further, the data application layer Springboot + Mybatis technical framework is built.
Furthermore, the page display layer is edited by the technologies of JointJS + JqueryUI + BootBox + Twitter BootStrap and the like.
The invention has the beneficial effects that: the webpage display system comprises a big data environment and a data application layer, wherein the big data environment is composed of spark, hive and mysql, the big data environment is connected with the data application layer through springboot + mybatis, the page display layer is realized by adopting the technologies of JointJS + JqueryUI + BootBox and Twitter BootStrap, and the data application layer is a Web application program. The invention solves the problems that the application program cannot be developed and modified in time due to the change of the user requirement, the workload and difficulty of developers are increased, the operation and maintenance cost is increased, the user requirement cannot be met quickly and the like. The page display layer realized by adopting the technologies of JointJS + JqueryUI + BootBox and Twitter BootStrap and the like returns the selected conditions to the data application layer according to the collision comparison conditions selected by the user, and the data application layer inquires data in a big data environment built by spark, hive and mysql and returns a result set to the page display layer. Users can change the user-defined query conditions to query according to business requirements, and the process model and the result model of the operations among a series of data elements of the canvas panel are stored and are ready for next use. Therefore, the requirement change of the user can be quickly responded, and the satisfaction degree of the user is improved.
Detailed Description
The invention relates to a big data visualization modeling and result displaying method, which comprises a big data environment and a data application layer, wherein the big data environment consists of spark, hive and mysql, the big data environment is connected with the data application layer through springboot + mybatis, the page display layer displays data returned by the data application layer, and the data application layer is a Web application program; the method comprises the following steps:
s1, a big data environment consisting of spark, hive and mysql is connected and debugged with the big data environment through a springboot technology frame and a MyBatis data persistent layer frame, so that a Web application program can be communicated with the big data environment;
s2, defining a page structure by using a front-end bootstrap frame, customizing and producing a common component based on JavaScript, dragging a data source or a model to a canvas area based on JQueryUI, and generating an element;
s3, dynamically connecting all elements in the canvas area by using a JointJS dynamic flow chart, wherein a data source element can only be directly connected with a query condition functional element, a dragged and dropped model is directly connected with any functional element, and any intermediate generated data source element can solidify memory data to the local;
s4, editing the canvas element by using a bootbox.js, inputting a user-defined query condition, binding the data source element, and establishing a node data source element association relation;
s5, connecting a page display layer realized by using the technologies of JointJS, JqueryUI, BootBox, Twitter BootStrap and the like with a Web application program through a springboot technology frame, and debugging and testing;
s6, the collision comparison system provides a model correlation function, the conditions selected by the page are sent to a web application layer, the web application layer connects the obtained conditions with a big data environment built by spark, live and mysql through springboot + mybasic to search data, a result set is returned to a page display layer to be displayed, and a process model and a result model of the operation among a series of data elements of a canvas panel are stored and are ready for the next use.
Further, the Web application program is edited by adopting a Java language architecture.
Furthermore, the data source is one or more of large data environments composed of spark, hive and mysql.
Further, the data application layer Springboot + Mybatis technical framework is built.
Furthermore, the page display layer is edited by the technologies of JointJS + JqueryUI + BootBox + Twitter BootStrap and the like.
The first embodiment is as follows:
s1, constructing a big data environment by using spark, hive and mysql components, and connecting and debugging the big data environment through a springboot technology frame and a MyBatis data persistent layer frame;
s2, using a bootstrap frame to self-define a page structure, customizing and producing a public component based on JavaScript, realizing that a data source or a model is dragged to a canvas area based on JQueryUI, and generating an element;
s3, dynamically connecting all elements in the canvas area by using a JointJS dynamic flow chart, wherein a data source element can only be directly connected with a query condition functional element, a dragged and dropped model is directly connected with any functional element, and any intermediate generated data source element can solidify memory data to the local;
s4, editing the canvas element by using a bootbox.js, inputting a user-defined query condition, binding the data source element, and establishing a node data source element association relation;
s5, connecting a page display layer realized by using the technologies of JointJS, JqueryUI, BootBox, Twitter BootStrap and the like with a Web application program through a springboot technology frame, and debugging and testing;
s6, the collision comparison system provides a model correlation function, the conditions selected by the page are sent to a web application layer, the web application layer connects the obtained conditions with a big data environment built by spark, live and mysql through springboot + mybasic to search data, a result set is returned to a page display layer to be displayed, and a process model and a result model of the operation among a series of data elements of a canvas panel are stored and are ready for the next use.
Claims (5)
1. A method based on big data visualization modeling and result display is characterized in that: the method comprises a big data application framework, a data application layer and a page display layer, wherein a data source is a big data environment consisting of spark, hive and mysql, the data application layer is a web application program, and the page display layer consists of visual, editable and draggable JointJS + JqueryUI + BootBox front-end technologies and comprises the following steps:
s1, connecting and debugging a big data environment consisting of spark, hive and mysql with the big data environment through a springboot technology frame and a MyBatis data persistent layer frame to enable a Web application program to be communicated with the big data environment;
s2, defining a page structure by using a front-end bootstrap frame, customizing and producing a public component based on JavaScript, dragging a data source or a model to a canvas area based on JQueryUI, and generating an element;
s3, dynamically connecting all elements in the canvas area by using a JointJS dynamic flow chart, wherein a data source element can only be directly connected with a query condition functional element, a dragged and dropped model is directly connected with any functional element, and any intermediate generated data source element can solidify memory data to the local;
s4, editing the canvas element by using a bootbox.js, inputting a user-defined query condition, binding the data source element, and establishing a node data source element association relation;
s5, connecting a page display layer realized by using the technologies of JointJS, JqueryUI, BootBox, Twitter BootStrap and the like with a Web application program through a springboot technology frame, and debugging and testing;
s6, the collision comparison system provides a model correlation function, the conditions selected by the page are sent to a web application layer, the web application layer connects the obtained conditions with a big data environment built by spark, live and mysql through springboot + mybasic to search data, a result set is returned to a page display layer to be displayed, and a process model and a result model of the operation among a series of data elements of a canvas panel are stored and are ready for the next use.
2. The big data visualization modeling and result displaying method according to claim 1, wherein: the Web application program is edited by adopting a Java language architecture.
3. The big data visualization modeling and result displaying method according to claim 1, wherein: the data source is one or more of large data environments consisting of spark, hive and mysql.
4. The big data visualization modeling and result displaying method according to claim 1, wherein: the data application layer Springboot + Mybatis technical framework is built.
5. The big data visualization modeling and result displaying method according to claim 1, wherein: the page display layer is edited by the technologies of JointJS + JqueryUI + BootBox + Twitter BootStrap and the like.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110220103.9A CN112783869A (en) | 2021-02-27 | 2021-02-27 | Big data visualization modeling and result display based method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110220103.9A CN112783869A (en) | 2021-02-27 | 2021-02-27 | Big data visualization modeling and result display based method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112783869A true CN112783869A (en) | 2021-05-11 |
Family
ID=75761967
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110220103.9A Pending CN112783869A (en) | 2021-02-27 | 2021-02-27 | Big data visualization modeling and result display based method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783869A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627935A (en) * | 2021-08-12 | 2021-11-09 | 北京联合货币兑换股份有限公司 | Intelligent foreign currency transaction system |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110035744A1 (en) * | 2006-12-29 | 2011-02-10 | Sanjay Bhatia | Self-service database reporting systems and methods |
| CN106528741A (en) * | 2016-11-04 | 2017-03-22 | 浪潮软件集团有限公司 | Visualization implementation method based on big data |
| CN107066499A (en) * | 2016-12-30 | 2017-08-18 | 江苏瑞中数据股份有限公司 | The data query method of multi-source data management and visualization system is stored towards isomery |
| CN108563666A (en) * | 2018-01-05 | 2018-09-21 | 成都兴政电子政务运营服务有限公司 | A kind of data visualization processing system and method based on big data technology |
| CN110609681A (en) * | 2018-06-14 | 2019-12-24 | 北京京东尚科信息技术有限公司 | System, method, device and medium for visually configuring and generating WEB application |
| CN111405032A (en) * | 2020-03-12 | 2020-07-10 | 威海精讯畅通电子科技有限公司 | General cloud platform of industry thing networking |
| CN111475564A (en) * | 2020-04-13 | 2020-07-31 | 恒安嘉新（北京）科技股份公司 | Streaming data processing method, system, computer equipment and storage medium |
| CN111798096A (en) * | 2020-06-02 | 2020-10-20 | 国网山西省电力公司电力科学研究院 | Visual analysis method, system and storage medium based on power grid dispatching data |
| CN111857659A (en) * | 2020-06-30 | 2020-10-30 | 太极计算机股份有限公司 | Data visualization design platform for dragging heterogeneous data source |
| CN111970138A (en) * | 2020-03-31 | 2020-11-20 | 贵州电网有限责任公司 | Network resource management and control system and resource management method |
| CN112100545A (en) * | 2020-09-11 | 2020-12-18 | 杭州安恒信息安全技术有限公司 | Visualization method, device and equipment of network assets and readable storage medium |
-
2021
- 2021-02-27 CN CN202110220103.9A patent/CN112783869A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110035744A1 (en) * | 2006-12-29 | 2011-02-10 | Sanjay Bhatia | Self-service database reporting systems and methods |
| CN106528741A (en) * | 2016-11-04 | 2017-03-22 | 浪潮软件集团有限公司 | Visualization implementation method based on big data |
| CN107066499A (en) * | 2016-12-30 | 2017-08-18 | 江苏瑞中数据股份有限公司 | The data query method of multi-source data management and visualization system is stored towards isomery |
| CN108563666A (en) * | 2018-01-05 | 2018-09-21 | 成都兴政电子政务运营服务有限公司 | A kind of data visualization processing system and method based on big data technology |
| CN110609681A (en) * | 2018-06-14 | 2019-12-24 | 北京京东尚科信息技术有限公司 | System, method, device and medium for visually configuring and generating WEB application |
| CN111405032A (en) * | 2020-03-12 | 2020-07-10 | 威海精讯畅通电子科技有限公司 | General cloud platform of industry thing networking |
| CN111970138A (en) * | 2020-03-31 | 2020-11-20 | 贵州电网有限责任公司 | Network resource management and control system and resource management method |
| CN111475564A (en) * | 2020-04-13 | 2020-07-31 | 恒安嘉新（北京）科技股份公司 | Streaming data processing method, system, computer equipment and storage medium |
| CN111798096A (en) * | 2020-06-02 | 2020-10-20 | 国网山西省电力公司电力科学研究院 | Visual analysis method, system and storage medium based on power grid dispatching data |
| CN111857659A (en) * | 2020-06-30 | 2020-10-30 | 太极计算机股份有限公司 | Data visualization design platform for dragging heterogeneous data source |
| CN112100545A (en) * | 2020-09-11 | 2020-12-18 | 杭州安恒信息安全技术有限公司 | Visualization method, device and equipment of network assets and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 姚颖: "基于模板的大数据统计分析报告生成方法研究与应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627935A (en) * | 2021-08-12 | 2021-11-09 | 北京联合货币兑换股份有限公司 | Intelligent foreign currency transaction system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110968943A (en) | Display method and device of terminal interface | |
| CN111506387B (en) | Page prerendering method and device, electronic equipment and storage medium | |
| CN110096277A (en) | A kind of dynamic page methods of exhibiting, device, electronic equipment and storage medium | |
| CN106021617A (en) | Page generation method and device | |
| Han et al. | Netv. js: A web-based library for high-efficiency visualization of large-scale graphs and networks | |
| CN107480893A (en) | A kind of flow method of combination and device | |
| CN111124533B (en) | Method and device for dynamic configuration of active page | |
| CA2375611A1 (en) | System and method for dynamic assistance in software applications using behavior and host application models | |
| CN112561640A (en) | Commodity page template generation method and device, computer equipment and storage medium | |
| JP7371317B2 (en) | Content recommendation method, device, electronic device, program and storage medium | |
| CN102662664A (en) | Interface animation realization method and system | |
| CN109426603B (en) | Method and device for analyzing application program blockage | |
| CN111966336A (en) | Page generation method and device based on VUE and visual graphic operation | |
| CN103473041B (en) | A kind of visual data processing method and system | |
| CN107704239A (en) | Configuration generating method, device and electronic equipment | |
| CN114048329A (en) | Knowledge graph construction and display method and device, electronic equipment and medium | |
| CN110221835A (en) | Increase the method and device of control in front end page | |
| CN112783869A (en) | Big data visualization modeling and result display based method | |
| CN109766373A (en) | Electric network data methods of exhibiting and computer readable storage medium | |
| CN113326044A (en) | Development method, system and storage medium based on control library | |
| US10289388B2 (en) | Process visualization toolkit | |
| CN102929636B (en) | A kind of human-computer interaction interface creation method and device | |
| WO2020035076A1 (en) | Method and system for visualizing data processing step of machine learning process | |
| CN103136669A (en) | Product display method and product display system based on product model base | |
| US11341705B1 (en) | Animated transitions in data visualizations according to characteristics of the data visualizations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210511 |
|
| RJ01 | Rejection of invention patent application after publication |