CN115827769A - Data visualization method, device, equipment and storage medium - Google Patents

Data visualization method, device, equipment and storage medium Download PDF

Info

Publication number
CN115827769A
CN115827769A CN202211642136.3A CN202211642136A CN115827769A CN 115827769 A CN115827769 A CN 115827769A CN 202211642136 A CN202211642136 A CN 202211642136A CN 115827769 A CN115827769 A CN 115827769A
Authority
CN
China
Prior art keywords
data
acquiring
visualized
storing
initial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211642136.3A
Other languages
Chinese (zh)
Inventor
陈锋
史进
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Songche Network Technology Co ltd
Original Assignee
Chongqing Songche Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Songche Network Technology Co ltd filed Critical Chongqing Songche Network Technology Co ltd
Priority to CN202211642136.3A priority Critical patent/CN115827769A/en
Publication of CN115827769A publication Critical patent/CN115827769A/en
Pending legal-status Critical Current

Links

Images

Abstract

The application relates to a data visualization method, a data visualization device, data visualization equipment and a storage medium, and relates to the field of data analysis. The data visualization method comprises the following steps: collecting initial data; performing data processing on the initial data to obtain processed data; storing the processed data in a preset storage position; reading data to be visualized from a preset storage position; performing data calculation on data to be visualized to obtain a data calculation result; and carrying out visual display on the data calculation result. The method and the device are used for solving the problem that various business data of the enterprise cannot be visually displayed.

Description

Data visualization method, device, equipment and storage medium
Technical Field
The present application relates to the field of data analysis, and in particular, to a data visualization method, apparatus, device, and storage medium.
Background
When the enterprise is operated to a larger scale, the operation data needs to be analyzed so as to be used for data display of the current operation status of the enterprise and prediction of future development direction.
At present, various business data of an enterprise cannot be visually displayed, and analysis of the enterprise on the operation condition and the operation trend of the enterprise is influenced.
Disclosure of Invention
The application provides a data visualization method, a data visualization device, data visualization equipment and a storage medium, which are used for solving the problem that various business data of an enterprise cannot be visually displayed.
In a first aspect, an embodiment of the present application provides a data visualization method, including:
collecting initial data;
performing data processing on the initial data to obtain processed data;
storing the processed data in a preset storage position;
reading data to be visualized from the preset storage position;
performing data calculation on the data to be visualized to obtain a data calculation result;
and visually displaying the data calculation result.
Optionally, the acquiring initial data includes:
the method comprises the steps of directly collecting initial data from an access log of an application server, wherein the access log of the application server comprises a user access log, a user internet protocol address, accessed resources, access time and client information;
initial data are collected through embedded points, wherein the embedded points comprise code embedded points, full embedded points and code-free embedded points.
Optionally, the collecting initial data through a buried point includes:
acquiring operation of a visual embedded point management operation tool on an embedded point management page, and generating a marked page element on a page needing embedded points according to the operation;
acquiring a trigger operation on the marked page element;
and acquiring initial data according to the triggering operation.
Optionally, the performing data processing on the initial data to obtain processed data includes:
acquiring a first dragging type operation of a user on a data preprocessing component, acquiring a data preprocessing type according to the first dragging type operation, and performing data preprocessing on the initial data by adopting the data preprocessing type to acquire preprocessed data, wherein the data preprocessing type comprises at least one of data extraction, cleaning, conversion and loading;
and acquiring a second dragging operation of a user on the data mining component, acquiring a data mining category according to the second dragging operation, and performing data mining on the preprocessed data by adopting the data mining category to acquire the processed data, wherein the data mining category comprises at least one of classification analysis, association analysis, regression analysis, cluster analysis and time sequence prediction.
Optionally, the processed data comprises at least one of structured data, semi-structured data, and unstructured data;
the storing the processed data in a preset storage position includes:
directly storing data which can be serialized in the structured data in a distributed file system;
data which cannot be serialized in the structured data are sorted and then are stored in a distributed database environment in a unified mode, and are stored in the distributed file system after being serialized;
storing the semi-structured data directly in a distributed file system;
storing the unstructured data directly in a distributed file system;
and taking the distributed file system as the preset storage position.
Optionally, the reading of the data to be visualized from the preset storage location includes:
reading the structured data corresponding to the first preset service category and the semi-structured data with a preset format relationship from the distributed file system, and storing the data in a data warehouse;
according to business requirements and preset business themes, a data mart is constructed in the data warehouse, and data to be visualized are read from the data mart;
and reading the semi-structured data which has a preset format relationship and corresponds to the second preset service type from the distributed file system, storing the semi-structured data into a database, and reading the data to be visualized from the database.
Optionally, the performing data calculation on the data to be visualized to obtain a data calculation result includes:
acquiring a third dragging operation of a user on a data computing frame component, and acquiring the category of a data computing frame according to the third dragging operation, wherein the category of the data computing frame comprises at least one of a distributed real-time big data processing frame, an offline data analysis frame and a big data parallel computing frame;
and performing data calculation on the data to be visualized by adopting the type of the data calculation frame to obtain a data calculation result.
In a second aspect, an embodiment of the present application provides a data visualization apparatus, including:
the data acquisition module is used for acquiring initial data;
the data processing module is used for carrying out data processing on the initial data to obtain processed data;
the data storage module is used for storing the processed data in a preset storage position;
the data reading module is used for reading data to be visualized from the preset storage position;
the data calculation module is used for performing data calculation on the data to be visualized to obtain a data calculation result;
and the data display module is used for visually displaying the data calculation result.
In a third aspect, an embodiment of the present application provides an electronic device, including: the system comprises a processor, a memory and a communication bus, wherein the processor and the memory are communicated with each other through the communication bus;
the memory for storing a computer program;
the processor is configured to execute the program stored in the memory to implement the data visualization method according to the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the data visualization method according to the first aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: in the embodiment of the application, initial data is collected, data processing is carried out on the initial data, processed data are obtained, the processed data are stored in a preset storage position, data to be visualized are read from the preset storage position, data calculation is carried out on the data to be visualized, a data calculation result is obtained, and the data calculation result is visually displayed. In the application, data acquisition, processing, storage, reading and calculation are carried out, and data calculation results are visually displayed, so that the visual display of business data of enterprises can be realized, the enterprises can conveniently and visually know the operation conditions and the operation trend, and the problem that various business data of the enterprises cannot be visually displayed is solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a schematic flow chart illustrating a method for data visualization according to an embodiment of the present disclosure;
FIG. 2 is a schematic structural diagram of a data visualization device in an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the protection scope of the present application.
In the embodiment of the application, a data visualization method is provided, which can be applied to terminal devices such as a computer, a tablet computer, a mobile phone and the like.
In the embodiment of the present application, as shown in fig. 1, the flow of the data visualization method mainly includes:
step 101, collecting initial data.
In one embodiment, the initial data is collected in a variety of ways, including but not limited to the following:
in one mode
The method comprises the steps of directly collecting initial data from an access log of an application server, wherein the access log of the application server comprises a user access log, a user internet protocol address, accessed resources, access time and client information.
Mode two
Initial data are collected through embedded points, wherein the embedded points comprise code embedded points, full embedded points and code-free embedded points.
No code burying point: the conventional code embedding needs the joint operation of front-end personnel and rear-end personnel, a link embedding point code is put into the front end of a code to be captured, and the rear end needs to make a corresponding receiving interface for the link embedding point to obtain the embedding point data. The interface embedding is a embedding mode that a user is connected with a data access management interface of a user behavior analysis tool through equipment, and page elements (such as pictures, buttons, links and the like) which can interact and have effects after interaction are directly operated on the interface to realize data embedding, and effective returns of acquisition codes are issued. The mode can be seen as the result, code deployment, test verification and version sending processes are skipped, and the productivity is greatly improved.
The code-free embedded point has the advantages that the embedded point can be directly operated on a WEB page or a real interface of mobile application, 1, whether the embedded point is correct or not can be immediately verified after the embedded point is embedded, and 2, the embedded point is deployed to all clients and is effective almost in real time. Because of these benefits of codeless landfills, the requesting party of the analysis, the business personnel, the person who does not have the authority to touch the code or who does not understand the programming can obtain data for analysis with very low thresholds. Can be a great progress of the buried point.
In one embodiment, the collecting of initial data by codeless burial points comprises: acquiring operation of a buried point management visual operation tool on a buried point management page, and generating a marked page element on a page needing to be buried according to the operation; acquiring trigger operation on a marked page element; and acquiring initial data according to the triggering operation.
It is necessary to write an SDK (Software Development Kit) in the front-end project, containing:
(1) And (4) logic codes of circle selection and click selection (namely, page element marking).
The logic code for marking page elements is mainly applied to the steps of marking embedded points on a page when the current page is operated or used by a non-developer, acquiring an Xpath path (selecting nodes or node sets in an XML document by using a path expression) of the elements in a way of circle selection or click selection, wherein the path is stored in an embedded data database and carries service parameters of items, such as a system, a menu, names of the embedded points, a triggering mode and the like of the page elements.
The way of marking the element is to use iframe (embedded frame in HTML) to nest the page needing the embedded point in a embedded point management page, and attach various related tools to the embedded point management page, for example: circle selection, point embedding element rendering, point embedding deletion, point embedding data list deletion and the like, and some simple and easy-to-use visual operation tools are developed around a page of the embedded points.
(2) Logic code that captures elements of the snoop tag.
When the SDK is initialized, a request is added to obtain all the buried point element Xpath paths of the page in the buried point database. When an operation event such as onClick (event for processing mouse click) is performed, the path of Xpah can be taken by acquiring the Dom element, and the element is monitored to be clicked at the moment.
Through operating the visual operation tool of the point management on the point management page, the marked page element can be generated on the page needing to be embedded, and the marked page element is triggered to acquire data, so that the method is convenient and quick and provides the visual operation of acquiring the data. And existing data can be automatically initialized, and missing data can be subjected to additional recording.
The initial data is collected, and the data collection can be divided into internal data collection and external data collection according to the data source. Internal data collection refers to collecting activity data inside an enterprise, and the data is usually from a business database. The external data acquisition refers to acquiring data from the outside of an enterprise by some methods, and the external data acquisition is mainly used for acquiring data of competitive products and some industry data published by official websites of official institutions.
And 102, performing data processing on the initial data to obtain processed data.
In one embodiment, the data processing the initial data to obtain the processed data includes: acquiring a first dragging type operation of a user on a data preprocessing component, acquiring a data preprocessing type according to the first dragging type operation, and performing data preprocessing on initial data by adopting the data preprocessing type to acquire preprocessed data, wherein the data preprocessing type comprises at least one of data extraction, cleaning, conversion and loading; and acquiring a second dragging operation of the user on the data mining component, acquiring a data mining category according to the second dragging operation, and performing data mining on the preprocessed data by adopting the data mining category to acquire the processed data, wherein the data mining category comprises at least one of classification analysis, association analysis, regression analysis, cluster analysis and time sequence prediction.
The reason for preprocessing the data is that the data collected in the early stage often contains noise and errors, and the quality of the data is low. The data mining is that the characteristics and the mode of the data are often hidden in massive data and can be obtained only by further data mining.
The data preprocessing may be ETL (Extract-Transform-Load, extract, clean, transform, load). The data preprocessing can delete repeated data, complement missing data, adjust data with non-uniform format to uniform format, and check logical errors of the data.
The method comprises the steps of obtaining a first dragging type operation of a user on a data preprocessing component, obtaining a data preprocessing category according to the first dragging type operation, carrying out data preprocessing on initial data according to the data preprocessing category, obtaining preprocessed data, achieving visualization operation of data preprocessing, and enabling the user to easily select needed data preprocessing operation.
The categories of data mining include at least one of classification analysis, association analysis, regression analysis, cluster analysis, and time series prediction, including a plurality of data mining algorithms.
The method comprises the steps of obtaining a second dragging operation of a user on a data mining component, obtaining a data mining type according to the second dragging operation, conducting data mining on preprocessed data according to the data mining type, obtaining the processed data, achieving visualization operation of data mining, enabling the user to easily select needed data mining operation, and enabling the user to easily master data mining without worrying even if the user does not understand an algorithm.
And 103, storing the processed data in a preset storage position.
In a particular embodiment, the processed data includes at least one of structured data, semi-structured data, and unstructured data. Storing the processed data in a preset storage position, including: directly storing data capable of being serialized in the structured data in a Distributed File System (HDFS); data which cannot be serialized in the structured data are sorted and then uniformly stored in a distributed database environment, and are serialized and then stored in a distributed file system; directly storing the semi-structured data in a distributed file system; storing unstructured data directly in a distributed file system; and taking the distributed file system as a preset storage position.
The structured data may be, for example, data in a database such as Oracle, mySQL, SQL Server, etc. The semi-structured data may be, for example, log data, click streams, and data in a data interface.
Data which cannot be serialized in the structured data are sorted and then uniformly stored in a distributed database environment, are serialized and then stored in a distributed file system, and the sorted data which cannot be serialized are also directly stored in the distributed file system.
Step 104, reading data to be visualized from a preset storage position.
In one embodiment, reading data to be visualized from a preset storage location includes: reading the structured data corresponding to the first preset service category and the semi-structured data with a preset format relationship from the distributed file system, and storing the data in a data warehouse; according to the business requirements and the preset business theme, a data mart is constructed in a data warehouse, and data to be visualized are read from the data mart; and reading the semi-structured data which corresponds to the second preset service type and has the preset format relationship from the distributed file system, storing the semi-structured data into a database, and reading the data to be visualized from the database.
Massive structured data, semi-structured data and unstructured data stored in the distributed file system need reasonable organization and storage.
The structured data corresponding to the first preset business category and the semi-structured data with the preset format relationship are read from the distributed file system and stored in the data warehouse, and the data mart is constructed in the data warehouse according to the business requirements and the preset business theme, so that the data mart can be constructed by an enterprise, and the statistical analysis of the business data required by the enterprise is facilitated.
And reading the semi-structured data which corresponds to the second preset service type and has the preset format relationship from the distributed file system, storing the semi-structured data into a database, and storing the semi-structured data into a Hadoop HBase column family database and other NoSQL databases. The semi-structured data which is corresponding to the second preset service type and does not have the preset format relationship can be stored in the distributed file system continuously.
And 105, performing data calculation on the data to be visualized to obtain a data calculation result.
In a specific embodiment, the data calculation for the data to be visualized to obtain the data calculation result includes: acquiring a third dragging operation of a user on the data computing frame assembly, and acquiring the category of the data computing frame according to the third dragging operation, wherein the category of the data computing frame comprises at least one of a distributed real-time big data processing frame, an offline data analysis frame and a big data parallel computing frame; and performing data calculation on the data to be visualized by adopting the category of the data calculation frame to obtain a data calculation result.
Aiming at the real-time and delay requirements of query analysis, different big data calculation frameworks can be selected to construct query analysis services. A distributed real-time big data processing framework (Storm), which is a distributed, fault-tolerant, real-time memory streaming computing system; an offline data analysis framework (Hadoop MapReduce) is a big data offline batch processing system and can be used for calculating a large amount of offline data; the big data parallel computing framework (Spark) is suitable for big data parallel computing and real-time analysis, and is better suitable for algorithms which need iteration, such as data mining and machine learning.
The method comprises the steps of obtaining a third dragging operation of a user on a data calculation frame assembly, obtaining the category of the data calculation frame according to the third dragging operation, carrying out data calculation on data to be visualized by adopting the category of the data calculation frame, obtaining a data calculation result, realizing the visualization operation of the data calculation frame, and enabling the user to easily select a required data calculation frame.
And 106, visually displaying the data calculation result.
And the data calculation result is visually displayed, and is visual mapping, namely a process of mapping the data calculation result into a visual element. Data is often faced with complexity and the information it contains is abundant. Therefore, organization and screening are performed during data visualization. If all the machines are placed, the whole page is not only bloated, disordered and lack of aesthetic feeling, but also has the problem of primary and secondary inseparability, so that the attention of a user cannot be concentrated, and the capability of the user for acquiring information in unit time is reduced.
A data visualization tool is an application that helps a user display data in a visualized, graphical format, presenting the complete outline of the data. Like pie charts, graphs, heat maps, histograms, radar/spider graphs are only a small part of the visualization, these methods can simply represent the data and demonstrate features and trends.
In summary, in the embodiment of the present application, initial data is collected, data processing is performed on the initial data to obtain processed data, the processed data is stored in a preset storage location, data to be visualized is read from the preset storage location, data calculation is performed on the data to be visualized to obtain a data calculation result, and the data calculation result is visually displayed. In the application, data acquisition, processing, storage, reading and calculation are carried out, and data calculation results are visually displayed, so that the visual display of business data of enterprises can be realized, the enterprises can conveniently and visually know the operation conditions and the operation trend, and the problem that various business data of the enterprises cannot be visually displayed is solved.
Based on the same concept, the embodiment of the present application provides a data visualization apparatus, and the specific implementation of the apparatus may refer to the description of the method embodiment section, and the repeated parts are not described again, as shown in fig. 2, the apparatus mainly includes:
a data acquisition module 201, configured to acquire initial data;
a data processing module 202, configured to perform data processing on the initial data to obtain processed data;
a data storage module 203, configured to store the processed data in a preset storage location;
a data reading module 204, configured to read data to be visualized from the preset storage location;
the data calculation module 205 is configured to perform data calculation on the data to be visualized to obtain a data calculation result;
and the data display module 206 is configured to perform visual display on the data calculation result.
Based on the same concept, an embodiment of the present application further provides an electronic device, as shown in fig. 3, the electronic device mainly includes: a processor 301, a memory 302 and a communication bus 303, wherein the processor 301 and the memory 302 communicate with each other via the communication bus 303. Wherein, the memory 302 stores programs that can be executed by the processor 301, and the processor 301 executes the programs stored in the memory 302, implementing the following steps:
collecting initial data; performing data processing on the initial data to obtain processed data; storing the processed data in a preset storage position; reading data to be visualized from a preset storage position; performing data calculation on data to be visualized to obtain a data calculation result; and carrying out visual display on the data calculation result.
The communication bus 303 mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus 303 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 3, but this does not mean only one bus or one type of bus.
The Memory 302 may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor 301.
The Processor 301 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like, and may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.
In a further embodiment of the present application, there is also provided a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the data visualization method described in the above embodiment.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wirelessly (e.g., infrared, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The available media may be magnetic media (e.g., floppy disks, hard disks, tapes, etc.), optical media (e.g., DVDs), or semiconductor media (e.g., solid state disks), among others.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A method of data visualization, comprising:
collecting initial data;
performing data processing on the initial data to obtain processed data;
storing the processed data in a preset storage position;
reading data to be visualized from the preset storage position;
performing data calculation on the data to be visualized to obtain a data calculation result;
and visually displaying the data calculation result.
2. The data visualization method according to claim 1, wherein the acquiring initial data comprises:
the method comprises the steps of directly collecting initial data from an access log of an application server, wherein the access log of the application server comprises a user access log, a user internet protocol address, accessed resources, access time and client information;
initial data are collected through embedded points, wherein the embedded points comprise code embedded points, full embedded points and code-free embedded points.
3. The data visualization method of claim 2, wherein the collecting initial data through a buried point comprises:
acquiring operation of a visual embedded point management operation tool on an embedded point management page, and generating a marked page element on a page needing embedded points according to the operation;
acquiring a trigger operation on the marked page element;
and acquiring initial data according to the triggering operation.
4. The data visualization method according to claim 1, wherein the data processing the initial data to obtain processed data comprises:
acquiring a first dragging type operation of a user on a data preprocessing component, acquiring a data preprocessing type according to the first dragging type operation, and performing data preprocessing on the initial data by adopting the data preprocessing type to acquire preprocessed data, wherein the data preprocessing type comprises at least one of data extraction, cleaning, conversion and loading;
and acquiring a second dragging operation of a user on the data mining component, acquiring a data mining category according to the second dragging operation, and performing data mining on the preprocessed data by adopting the data mining category to acquire the preprocessed data, wherein the data mining category comprises at least one of classification analysis, association analysis, regression analysis, cluster analysis and time series prediction 5.
5. The data visualization method of claim 1, wherein the processed data comprises at least one of structured data, semi-structured data, and unstructured data;
the storing the processed data in a preset storage position includes: 0, directly storing data which can be serialized in the structured data in a distributed file system;
data which cannot be serialized in the structured data are sorted and then are stored in a distributed database environment in a unified mode, and are stored in the distributed file system after being serialized;
5, directly storing the semi-structured data in a distributed file system;
storing the unstructured data directly in a distributed file system;
and taking the distributed file system as the preset storage position.
6. The data visualization method according to claim 5, wherein the reading of the data to be visualized from the preset storage location comprises:
0, reading out the structured data corresponding to the first preset service class and the semi-structured data with a preset format relationship from the distributed file system, and storing the data into a data warehouse;
according to business requirements and preset business themes, a data mart is built in the data warehouse, and data to be visualized are read from the data mart;
and 5, reading the semi-structured data with the preset format relationship corresponding to the second preset service type from the distributed file system, storing the semi-structured data into a database, and reading the data to be visualized from the database.
7. The data visualization method according to claim 1, wherein the performing data computation on the data to be visualized to obtain a data computation result comprises:
acquiring a third dragging operation of a user on a data computing frame component, and acquiring the category of a data computing frame according to the third dragging operation, wherein the category of the data computing frame comprises at least one of a distributed real-time big data processing frame, an offline data analysis frame and a big data parallel computing frame;
and performing data calculation on the data to be visualized by adopting the type of the data calculation frame to obtain a data calculation result.
8. A data visualization device, comprising:
the data acquisition module is used for acquiring initial data;
the data processing module is used for carrying out data processing on the initial data to obtain processed data;
the data storage module is used for storing the processed data in a preset storage position;
the data reading module is used for reading data to be visualized from the preset storage position;
the data calculation module is used for performing data calculation on the data to be visualized to obtain a data calculation result;
and the data display module is used for visually displaying the data calculation result.
9. An electronic device, comprising: the system comprises a processor, a memory and a communication bus, wherein the processor and the memory are communicated with each other through the communication bus;
the memory for storing a computer program;
the processor, configured to execute the program stored in the memory, and implement the data visualization method according to any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the data visualization method of any one of claims 1 to 7.
CN202211642136.3A 2022-12-20 2022-12-20 Data visualization method, device, equipment and storage medium Pending CN115827769A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211642136.3A CN115827769A (en) 2022-12-20 2022-12-20 Data visualization method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211642136.3A CN115827769A (en) 2022-12-20 2022-12-20 Data visualization method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115827769A true CN115827769A (en) 2023-03-21

Family

ID=85517080

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211642136.3A Pending CN115827769A (en) 2022-12-20 2022-12-20 Data visualization method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115827769A (en)

Similar Documents

Publication Publication Date Title
US10353756B2 (en) Cluster-based processing of unstructured log messages
AU2009238294B2 (en) Data transformation based on a technical design document
US10282197B2 (en) Open application lifecycle management framework
US20190058719A1 (en) System and a method for detecting anomalous activities in a blockchain network
US20170351989A1 (en) Providing supply chain information extracted from an order management system
CN107688568A (en) Acquisition method and device based on web page access behavior record
Salehi et al. SensorDB: a virtual laboratory for the integration, visualization and analysis of varied biological sensor data
US20150095298A1 (en) Method for Tracking a Schema in a Schema-Less Database
US11663172B2 (en) Cascading payload replication
WO2021072742A1 (en) Assessing an impact of an upgrade to computer software
CN113254320A (en) Method and device for recording user webpage operation behaviors
US20140006000A1 (en) Built-in response time analytics for business applications
WO2013143407A1 (en) Data processing, data collection
Ahsaan et al. Big data analytics: challenges and technologies
CN113010494A (en) Database auditing method and device and database proxy server
CN115827769A (en) Data visualization method, device, equipment and storage medium
CN115098568A (en) Data processing method, apparatus, device, medium, and program product
Anh Web Scraping: A Big Data Building Tool And Its Status In The Fintech Sector In Viet Nam
CN112650925A (en) APP information pushing system, method and medium for all-purpose card
CN113515715A (en) Generating method, processing method and related equipment of buried point event code
US20190087484A1 (en) Capturing context using network visualization
JP6287436B2 (en) Information processing apparatus, information processing system, information processing method, and program
CN116719986B (en) Python-based data grabbing method, device, equipment and storage medium
US20240070161A1 (en) Global status monitoring for open data platform
Jánki et al. Standardized Telemedicine Software Development Kit with Hybrid Cloud Support

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination