CN117573953B - Page big data visual rendering method, system and storage medium - Google Patents
Page big data visual rendering method, system and storage medium Download PDFInfo
- Publication number
- CN117573953B CN117573953B CN202410058515.0A CN202410058515A CN117573953B CN 117573953 B CN117573953 B CN 117573953B CN 202410058515 A CN202410058515 A CN 202410058515A CN 117573953 B CN117573953 B CN 117573953B
- Authority
- CN
- China
- Prior art keywords
- data
- record table
- level
- time range
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000009877 rendering Methods 0.000 title claims abstract description 89
- 230000000007 visual effect Effects 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000005070 sampling Methods 0.000 claims abstract description 51
- 238000012544 monitoring process Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000013079 data visualisation Methods 0.000 claims description 4
- 238000013500 data storage Methods 0.000 abstract description 2
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/904—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2219—Large Object storage; Management thereof
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a visual rendering method, a visual rendering system and a visual rendering storage medium for page big data, and belongs to the technical field of computers. The method comprises the following steps: creating a second-level, minute-level and hour-level record table in a database; recording second data of the target equipment; integrating minute-level and hour-level data; receiving a time range selected by a front end; sequentially determining sampling step length and time dimension to be loaded, and sending the sampling step length and the time dimension to the front end from the corresponding time record table to render data. The system comprises: creating a time dimension record table module; writing a second-level data module; integrating a minute-level hour-level data module; receiving a first time range module; acquiring a first display data module; and sending the first display data module. The method carries out logical sub-table recording on the data generated by the equipment, can effectively adjust the dimension of data storage, and obtains the data with proper time dimension, so that the data is smoothly loaded and rendered, and the influence of huge data quantity is avoided.
Description
Technical Field
The invention belongs to the technical field of computers, and relates to a visual rendering method, a visual rendering system and a storage medium for page big data.
Background
With the development and application of big data technology, big data rendering becomes an indispensable important link in many industries. The visual rendering of the page big data refers to the process of analyzing, processing, mining and visually displaying the massive data on the browser page by utilizing big data technology and algorithm. In general, the graphic is rendered in the coordinate axis in the display process, and is presented by using a histogram, a line graph and other graphics, so that a user can be helped to extract valuable information from mass data.
Currently, big data rendering is mainly the following steps: (1) Continuously acquiring real-time change data generated by equipment operation; (2) Integrating data, and recording the integrated data into a corresponding database table; (3) Packaging the integrated data and loading a rendering system, so that the integrated data is presented on a browser page through a graph; thus, visual rendering of big data can be realized. However, since the device may generate data in a long running time and every second, the data recorded in the database may be huge, and once the data is excessive, normal rendering may be affected, where the affecting situations include: the data loading time is long and uncontrollable, the browser page is blocked, the browser page is crashed, and the system is down.
Therefore, how to improve the loading rendering efficiency of data when facing huge data volume, and quickly realize visual presentation on a browser page is a technical problem which needs to be solved currently.
Disclosure of Invention
The invention aims to solve the technical problems in the background art and provides a method, a system and a storage medium for visual rendering of page big data.
The technical scheme for solving the technical problems is as follows:
in a first aspect, a visual rendering method for big page data is provided, which is applied to a server, and includes the steps:
s1, respectively creating a second-level record table, a minute-level record table and an hour-level record table in a database;
s2, monitoring and collecting change data generated by target equipment every second, and storing the change data every second into a second-level record table;
s3, according to the time interval of continuously generating the change data by the target equipment, correspondingly integrating the change data, and correspondingly storing the integrated data into a minute-level record table or an hour-level record table;
s4, receiving a time range selected by a user and collected by a front end;
s5, determining a sampling step length of the selected time range according to the selected time range, calculating a time dimension to be loaded according to the sampling step length of the selected time range, and acquiring display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table;
s6, sending the display data in the selected time range to the front end for loading rendering.
In one embodiment, the step S3 includes:
when the time interval of continuously generating the change data by the target equipment reaches a preset minute value, integrating the change data in the preset minute, and storing the integrated data in the preset minute into a minute-level record table;
when the time interval of continuously generating the change data by the target equipment reaches a preset hour value, integrating the change data in the preset hour, and storing the integrated data in the preset hour into an hour-level record table.
In one embodiment, in the step S4, the time range selected by the user includes a default selectable time range in the front-end page preset information list, a custom selectable time range in the page preset information list, or a time range selected on a time axis.
In one embodiment, the step S5 includes:
second-level time stamp T of acquisition start time 1 Second-level timestamp T of end time 2 Determining the sampling step length as (T) 2 -T 1 ）；
According to the sampling step (T) of the selected time range 2 -T 1 ) Calculating the time dimension to be loaded, wherein the calculation formula is as follows: (T) 2 -T 1 ) P=n, where P is the maximum number of rendering points in the visual range, 100+.p+. 10000, a variable constant; n is a variable, when 1<n<24. Time second level record table, 24<n<24×30 minute recording table, 24×30<n takes the hour level record table.
In one embodiment, before the step S2, the method further includes a step a:
A. creating an event record table in a database, wherein the event record table is an automatic tag point table and/or a manual tag point table;
between step S3 and step S4, further comprising step B:
B. when the target equipment generates an event, capturing event data and event occurrence time, storing the event data and the event occurrence time into an event record table, and generating a time positioning label of a corresponding event in the event record table;
after step S4, step C is further included:
C. and according to the selected time range, sending the time positioning label and the event data in the selected time range to the front end to render the current broken line with the time positioning label.
In one embodiment, after step S6, steps D to G are further included:
D. receiving an instruction of zooming the size of the rendering graph input by a user;
E. acquiring a current time range in real time according to an instruction of zooming the size of the rendering graph;
F. determining a sampling step length of the current time range according to the current time range, calculating the time dimension to be loaded in real time according to the sampling step length of the current time range, and acquiring display data in the current time range from a second-level record table, a minute-level record table or an hour-level record table;
G. and sending the display data in the current time range to the front end for loading rendering.
In a second aspect, a visual rendering method of page big data is provided, which is applied to a front end, and includes the steps:
a collected user-selected time range;
the time range selected by the user is sent to the server;
receiving display data in a selected time range sent by a server, and loading, rendering and displaying the display data in the selected time range; the database of the server is created with a second-level record table, a minute-level record table and an hour-level record table; the server is used for monitoring and collecting change data generated by the target equipment every second, and storing the change data every second into a second-level record table; the server is also used for correspondingly integrating the change data according to the time interval of continuously generating the change data by the target equipment, and correspondingly storing the integrated data into a minute-level record table or an hour-level record table; the server is further used for determining the sampling step length of the selected time range according to the selected time range, calculating the time dimension to be loaded according to the sampling step length of the selected time range, and acquiring the display data in the selected time range from the second-level record table, the minute-level record table or the hour-level record table.
In a third aspect, a visual rendering system for big page data is provided, and the visual rendering system is applied to a server and comprises:
the time dimension record table creating module is used for creating a second-level record table, a minute-level record table and an hour-level record table in the database respectively;
the second-level data writing module is used for monitoring and collecting the change data generated by the target equipment every second and storing the change data every second into the second-level record table;
the minute-level hour-level data integrating module is used for correspondingly integrating the change data according to the time interval of continuously generating the change data by the target equipment and correspondingly storing the integrated data into a minute-level record table or an hour-level record table;
the first time range receiving module is used for receiving a time range selected by a user and collected by the front end;
the first display data acquisition module is used for determining the sampling step length of the selected time range according to the selected time range, calculating the time dimension to be loaded according to the sampling step length of the selected time range, and acquiring display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table;
the first display data sending module is used for sending display data in a selected time range to the front end for loading rendering.
In one embodiment, the system, before the writing second-level data module, further includes:
the event record table creating module is used for creating an event record table in the database, wherein the event record table is an automatic tag point table and/or a manual tag point table;
between the integrating minute-level hour-level data module and the receiving first time range module, further comprising:
the event information storing module is used for capturing event data and event occurrence time when the target equipment generates an event, storing the event data and the event occurrence time into the event record table, and generating a time positioning label of a corresponding event in the event record table;
after receiving the first time range module, further comprising:
and the event information sending module is used for sending the time positioning label and the event data in the selected time range to the front end to render the current broken line with the time positioning label according to the selected time range.
In one embodiment, the system, after transmitting the first presentation data module, further comprises
The zoom graphic instruction receiving module is used for receiving an instruction of zooming the size of the rendering graphic input by a user;
the sampling step length module is used for determining the current time range and acquiring the current time range in real time according to the instruction of zooming the size of the rendering graph;
the second display data acquisition module is used for determining the sampling step length of the current time range according to the current time range, calculating the time dimension to be loaded in real time according to the sampling step length of the current time range, and acquiring display data in the current time range from a second-level record table, a minute-level record table or an hour-level record table;
the second display data sending module is used for sending display data in the current time range to the front end for loading rendering.
In a fourth aspect, a computer readable storage medium is provided, on which a computer program is stored, which program, when executed by a processor, implements a page big data visualization rendering method as described above.
The beneficial effects of the invention are as follows:
(1) According to the method, the data generated by the equipment are subjected to logic sub-table recording, and the change data are respectively stored in the second-level recording table, the minute-level recording table and the hour-level recording table, so that the storage dimension of the data can be effectively adjusted, the data with proper time dimension can be obtained, and the influence of huge data quantity is avoided;
(2) The second-level record table, the minute-level record table and the hour-level record table created by the method can act simultaneously, so that time dimension-increasing treatment and time dimension-decreasing treatment can be performed in real time, the visual granularity of the page can be controlled in real time, the cache cost is ensured, and the uninterrupted monitoring and rendering for 24 hours are ensured;
(3) The invention can ensure that the acquisition, integration and page rendering of the historical operation data of the equipment can be completed in millisecond level by carrying out logic sub-table recording.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a visual rendering method of page big data applied to a server in embodiment 1 of the present invention.
Fig. 2 is a flow chart of a visual rendering method of page big data applied to a server in embodiment 2 of the present invention.
Fig. 3 is a flow chart of a visual rendering method of page big data applied to a server in embodiment 3 of the present invention.
Fig. 4 is a flow chart of a visual rendering method of page big data applied to a front end in embodiment 4 of the present invention.
Fig. 5 is a schematic structural diagram of a visual rendering system for big page data in embodiment 5 of the present invention.
Fig. 6 is a schematic diagram of a module for integrating minute-level hour-level data in embodiment 5 of the present invention.
In the drawings, the list of components represented by the various numbers is as follows:
5001. creating a time dimension record table module; 5002. creating an event record table module; 5003. writing a second-level data module; 5004. integrating a minute-level hour-level data module; 5005. storing the event information module; 5006. receiving a first time range module; 5007. acquiring a first display data module; 5008. transmitting a first display data module; 5009. the event information sending module; 5010. receiving a scaled graphic instruction module; 5011. a sampling step length module for determining the current time range; 5012. acquiring a second display data module; 5013. transmitting a second display data module; 50041. integrating a minute-level record list unit; 50042. the hour level record table unit is integrated.
Description of the embodiments
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Example 1
As shown in fig. 1, the present embodiment provides a visual rendering method for page big data, which includes:
s101, respectively creating a second-level record table, a minute-level record table and an hour-level record table in a database.
Further, a day level record table is also created in the database. By simultaneously establishing several time dimension logical sub-tables, the dimension increasing processing and the dimension decreasing processing are realized in real time, and a foundation is laid for ensuring the quick completion of the acquisition, integration and page rendering of the historical operating data of the equipment.
S102, monitoring and collecting change data generated by the target equipment every second, and storing the change data every second into a second-level record table.
It will be appreciated that the minimum interval of acquisition time intervals is typically set to 1 second, resulting in a time stamp in seconds. If the minimum interval between acquisition time intervals is set to be smaller, for example, milliseconds or microseconds, a corresponding time dimension logical sub-table needs to be created in the database of step S101.
S103, when the time interval for continuously generating the change data by the target equipment reaches 1 minute, integrating the change data within 1 minute, and storing the integrated data within 1 minute into a minute-level record table.
S104, when the time interval for continuously generating the change data by the target equipment reaches 1 hour, integrating the change data within 1 hour, and storing the integrated data within 1 hour into an hour-level record table.
S105, receiving a user-selected time range collected by the front end, wherein the user-selected time range is a time range selected on a time axis.
Further, the time range selected by the user may be a default optional time range in the front-end page preset information list, the horizontal axis is time, and the vertical axis is data flow, so that the user can directly select the time range, for example, the time range is selected to be the last 10 minutes, the last one hour, the last day, and the operation is convenient and intuitive; the user can also define an optional time range in the page preset information list, so that the user can set the time range according to the needs, and the time range can be selected more flexibly.
S106, determining a sampling step length of the selected time range according to the selected time range, calculating a time dimension to be loaded according to the sampling step length of the selected time range, and acquiring display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table.
It is understood that the sampling step size (Sampling Step Size) refers to the distance or time difference between two adjacent sampling points in a continuous data stream. The unit of sampling step size will vary from application to application. In this embodiment, the time series analysis process is performed, and the unit of the sampling step is time, and the unit may be seconds, minutes, hours, days, or the like.
Further, the step S106 includes:
s1061, acquiring a second-level timestamp T of start time 1 Second-level timestamp T of end time 2 Determining the sampling step length as (T) 2 -T 1 ）；
S1062 according to the sampling step length (T 2 -T 1 ) Calculating the time dimension to be loaded, wherein the calculation formula is as follows: (T) 2 -T 1 ) P=n, wherein P is 10000; n is a variable, when 1<n<24. Time second level record table, 24<n<24×30 minute recording table, 24×30<n takes the hour level record table. In this embodiment, the maximum number of rendering points P in the visual range is 100+.p10000, which can improve the fluency of the operation data and is also helpful for data display.
S107, sending the display data in the selected time range to the front end, and loading and rendering by adopting an Echarts plug-in at the front end.
It should be noted that, according to the rendering tool or the plug-in adopted by the front end, the display data in the selected time range is sent after being packaged into the format of the corresponding rendering tool or plug-in.
According to the technical scheme, the data generated by the target equipment are subjected to logic sub-table recording, and the change data are respectively stored in the second-level recording table, the minute-level recording table and the hour-level recording table, so that the dimension of data storage can be effectively adjusted, the data with proper time dimension can be obtained, and the influence of huge data quantity is avoided.
Example 2
As shown in fig. 2, the present embodiment provides a visual rendering method for page big data, which includes:
s201, respectively creating a second-level record table, a minute-level record table and an hour-level record table in a database;
s202, creating an event record table in a database, wherein the event record table is an automatic label point table and a manual label point table;
s203, monitoring and collecting change data generated by target equipment every second, and storing the change data every second into a second-level record table;
s204, when the time interval for continuously generating the change data by the target equipment reaches 1 minute, integrating the change data within 1 minute, and storing the integrated data within 1 minute into a minute-level record table;
s205, when the time interval for continuously generating the change data by the target equipment reaches 2 hours, integrating the change data within 2 hours, and storing the integrated data within 2 hours into an hour-level record table;
s206, when the target equipment generates an event, capturing event data and event occurrence time, storing the event data and the event occurrence time into an event record table, and generating a time positioning label of the corresponding event in the event record table;
s207, receiving a time range selected by a user and collected by a front end, wherein the time range selected by the user is a default optional time range in a preset information list of a front end page;
s208, determining a sampling step length of a selected time range according to the selected time range, calculating a time dimension to be loaded according to the sampling step length of the selected time range, and acquiring display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table;
s209, sending the display data in the selected time range to the front end, and rendering a first graph by adopting an Echarts plug-in at the front end;
s210, according to the selected time range, sending the time positioning label and event data in the selected time range to the front end, and rendering a current broken line with the time positioning label by adopting an Echarts plug-in at the front end.
Further, analyzing the display data in the selected time range, the time positioning label in the selected time range and the event data, and changing the colors of the current broken line and the time positioning label by combining the analysis result and a preset data quantity warning threshold value. Therefore, the size of the data volume can be more intuitively embodied by changing the colors of the current broken line and the time positioning label, and the user can conveniently check the data volume.
It will be appreciated that the integrated time interval may be set according to the amount of device data required to be rendered for ease of management.
It will be further appreciated that the automatic tag point table represents automatic capturing of generated events, such as downtime events, of a target device system or application, and the automatically captured events are formed into tag points and recorded as a data set.
The manual tag point table is used for manually capturing generated events of a target equipment system or application, such as abnormal attack events and offline events, forming tag points of the manually captured events, and recording the manually captured events as a data set.
According to the technical scheme, by combining the event record table and the line graph, a user can click on the time positioning label on the current line, check the reason of the data change of a specific time point, and provide convenience for the user to further analyze the data, and improve the use experience.
Example 3
As shown in fig. 3, the present embodiment provides a visual rendering method for page big data, which includes:
s301, respectively creating a second-level record table, a minute-level record table and an hour-level record table in a database;
s302, monitoring and collecting change data generated by target equipment every second, and storing the change data every second into a second-level record table;
s303, when the time interval for continuously generating the change data by the target equipment reaches 1 minute, integrating the change data within 1 minute, and storing the integrated data within 1 minute into a minute-level record table;
s304, when the time interval for continuously generating the change data by the target equipment reaches 2 hours, integrating the change data within 2 hours, and storing the integrated data within 2 hours into an hour-level record table;
s305, receiving a time range selected by a user and collected by a front end, wherein the time range selected by the user is a time range selected on a time axis;
s306, determining a sampling step length of a selected time range according to the selected time range, calculating a time dimension to be loaded according to the sampling step length of the selected time range, and acquiring display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table;
s307, sending the display data in the selected time range to the front end, and loading and rendering a second graph by using an Echarts plug-in at the front end;
s308, receiving an instruction of zooming the second graph size input by the current user;
s309, acquiring a current time range according to an instruction for scaling the second graph;
s310, determining a sampling step length of a current time range according to the current time range, calculating a time dimension to be loaded according to the sampling step length of the current time range, and acquiring display data in the current time range from a second-level record table, a minute-level record table or an hour-level record table;
s311, sending the display data in the current time range to the front end, and loading and rendering a third graph by using the Echarts plug-in at the front end.
According to the technical scheme, no matter the user selects the time range by himself or the user operates the mouse wheel or drags the scroll bar to conduct rendering graphic scaling in the visual window range, the device history change data is acquired, loaded and rendered in millisecond level through logic sub-table recording.
Example 4
As shown in fig. 4, in one embodiment, a visual rendering method of page big data is provided, which is applied to a front end, and includes the steps of:
s401, collecting a time range selected by a user;
s402, sending a time range selected by a user to a server;
s403, receiving display data in a selected time range sent by a server, and loading, rendering and displaying the display data in the selected time range; the database of the server is created with a second-level record table, a minute-level record table and an hour-level record table; the server is used for monitoring and collecting change data generated by the target equipment every second, and storing the change data every second into a second-level record table; the server is also used for correspondingly integrating the change data according to the time interval of continuously generating the change data by the target equipment, and correspondingly storing the integrated data into a minute-level record table or an hour-level record table; the server is further used for determining the sampling step length of the selected time range according to the selected time range, calculating the time dimension to be loaded according to the sampling step length of the selected time range, and acquiring the display data in the selected time range from the second-level record table, the minute-level record table or the hour-level record table.
The technical scheme of the embodiment provides a visual rendering method of page big data, which is applied to a front end and is used for rendering a browser page at the front end by receiving display data sent by a server.
Example 5
As shown in fig. 5, in one embodiment, a visual rendering system for big data of a page is provided, which is applied to a server and includes:
a create time dimension record table module 5001 for creating a second level record table, a minute level record table, and an hour level record table in the database, respectively;
an event record table creating module 5002, configured to create an event record table in a database, where the event record table is an automatic tag point table and a manual tag point table;
the writing second-level data module 5003 is configured to monitor and collect change data generated by the target device every second, and store the change data every second into the second-level record table;
the minute-level hour-level data integrating module 5004 is configured to correspondingly integrate the change data according to a time interval when the target device continuously generates the change data, and correspondingly store the integrated data into a minute-level record table or an hour-level record table;
the stored event information module 5005 is configured to capture event data and event occurrence time when an event occurs in the target device, store the event data and the event occurrence time in an event record table, and generate a time positioning tag of a corresponding event in the event record table;
a first time range receiving module 5006, configured to receive a user-selected time range collected by the front end;
the first display data acquisition module 5007 is configured to determine a sampling step size of the selected time range according to the selected time range, calculate a time dimension to be loaded according to the sampling step size of the selected time range, and acquire display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table;
a first display data sending module 5008, configured to send display data in a selected time range to a front end for rendering a graphic;
the event information sending module 5009 is configured to send, according to the selected time range, the time positioning tag and the event data in the selected time range to the front end to render a current polyline with the time positioning tag;
a receive scaled graphics instruction module 5010 for receiving a scaled render graphics size instruction input by a user;
the sampling step size module 5011 is used for determining the current time range and acquiring the current time range in real time according to the instruction of zooming the size of the rendering graph;
the second display data obtaining module 5012 is configured to determine a sampling step length of the current time range according to the current time range, calculate, in real time, a time dimension to be loaded according to the sampling step length of the current time range, and obtain display data in the current time range from a second-level record table, a minute-level record table or an hour-level record table;
and the second display data sending module 5013 is configured to send display data in the current time range to the front end for load rendering.
Further, as shown in fig. 6, the integrated minute-level hour-level data module 5004 includes:
an integrated minute-level record table unit 50041 for integrating the change data within a preset minute when the time interval of the target device continuously generating the change data reaches a preset minute value, and storing the integrated data within the preset minute into a minute-level record table;
and the integrated hour level record table unit 50042 is used for integrating the change data in the preset hours when the time interval of continuously generating the change data by the target equipment reaches the preset hour value, and storing the integrated data in the preset hours into the hour level record table.
In the following, some comparative experiments are provided to further illustrate the present embodiment, specifically as follows:
table 1 experimental environment
| Name of the name | Configuration of |
| Operating system | centos 7.8 |
| Environment configuration | 4 CPUs, 8GB memory |
| Database version | 10.8.3-MariaDB |
| PHP version | 7.4.23 |
| Nginx version | nginx/1.20.1 |
| Echarts version | 5.3 |
| Browser version | Chrome 119.0.6045.124 |
The experiment designs three rendering scenes in total, namely (1) original data loading; (2) original data segment loading; (3) The data of this embodiment is obtained in a sub-table and a segment, which is specifically as follows:
(1) Raw data loading
And (3) obtaining a data scene, namely obtaining a data overview chart generated by uninterrupted operation of the backup system from 10 months 1 in 2023 to 11 months 1 in 2023, wherein the minimum granularity is displayed in a second level, and the total of 2,268,000 display units.
The PHP is used for acquiring corresponding records in the database and analyzing the acquired data, the data are combined into a format required by the Echarts plug-in rendering, the total time is 2.351 seconds, and the total time for the browser to render the acquired data is 31 seconds.
In the test process, multiple browser rendering crashes occur. The rendered data cannot be normally clicked or slid on the scroll wheel to view the data details.
Experimental results: the above data cannot be normally loaded and rendered.
(2) Raw data segment loading
And (3) obtaining a data scene, namely obtaining a data overview chart generated by uninterrupted operation of the backup system from 10 th month 1 of 2023 to 11 th month 1 of 2023, and displaying the data overview chart with minimum granularity to second level, wherein the total data overview chart is 2,268,000 display units.
In view of the situation encountered in test case (1) (i.e., "raw data loading"), the data is now segmented into daily presentations, taking a day of data rendering each time, totaling 86,400 data points per rendering.
The PHP obtains the corresponding records in the database and analyzes the obtained data, the data is combined into a format required by the Echarts plug-in rendering, the total time is 687.18 milliseconds, and the total time of the browser rendering of the obtained data is 757.40 milliseconds.
The browser can normally render data, the rendered data can be clicked to view details or the data range of the data can be enlarged and reduced through the mouse wheel, but the operation process is discontinuous, has a clamping and delay, and the delay time is about 3 seconds.
Experimental results: the obtained data can be normally loaded and rendered, the operation is inconvenient, the segmentation is more, a longer time range cannot be processed, and the whole data overview cannot be acquired.
(3) Data sub-table segment acquisition
And (3) obtaining a data scene, namely obtaining a data overview chart generated by uninterrupted operation of the backup system from 10 th month 1 of 2023 to 11 th month 1 of 2023, and displaying the data overview chart with minimum granularity to second level, wherein the total data overview chart is 2,268,000 display units.
The obtained data were tabulated into a seconds scale (2,268,000 display units), minutes scale (37,800 display units) and hours scale (630 display units).
And dynamically calculating a time dimension table for acquiring data according to the selected loading range according to a time dimension calculation formula, wherein the number of the acquired data points is always less than 10000. The PHP obtains the corresponding records in the database and analyzes the obtained data, the data is combined into a format required by the Echarts plug-in rendering, the total time is 187.18 milliseconds, and the total time for the browser to render the obtained data is 100 milliseconds.
The browser can complete data acquisition and rendering in a millisecond time range, can perform noninductive operation on rendered data, and can acquire a data flow trend chart in the time range from the whole.
Experimental results: the data loading and rendering can be completed in millisecond level, and the data is rendered in noninductive operation. The time range is clear, the segments are few, the data rendering in a longer time range can be processed, and the overall data overview can be intuitively acquired, so that the backup data can be conveniently managed and used.
According to the technical scheme, the second-level recording table storage device generates data sum in each minute, the minute-level recording table storage device generates data sum in each hour, the loading and rendering efficiency of the data are improved integrally, and visual presentation of a large amount of data is achieved.
Example 6
In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the page big data visualization rendering method according to any one of embodiments 1 to 4.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
The computer program code for carrying out operations of the present invention may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++, ruby, go and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (9)
1. The visual rendering method of the page big data is characterized by being applied to a server and comprising the following steps:
s1, respectively creating a second-level record table, a minute-level record table and an hour-level record table in a database;
s2, monitoring and collecting change data generated by target equipment every second, and storing the change data every second into a second-level record table;
s3, according to the time interval of continuously generating the change data by the target equipment, correspondingly integrating the change data, and correspondingly storing the integrated data into a minute-level record table or an hour-level record table;
s4, receiving a time range selected by a user and collected by a front end;
s5, determining a sampling step length of a selected time range according to the selected time range, calculating a time dimension to be loaded according to the sampling step length of the selected time range, and acquiring display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table, wherein the method specifically comprises the following steps:
second-level time stamp T of acquisition start time 1 Second-level timestamp T of end time 2 Determining the productionThe sample step length is (T) 2 -T 1 ) The method comprises the steps of carrying out a first treatment on the surface of the According to the sampling step (T) of the selected time range 2 -T 1 ) Calculating the time dimension to be loaded, wherein the calculation formula is as follows: (T) 2 -T 1 ) P=n, where P is the maximum number of rendering points in the visual range, 100+.p+. 10000, a variable constant; n is a variable, when 1< n <24. Time second level record table, 24< n <24×30 minute recording table, 24×30<n is taken as an hour level record table;
s6, sending the display data in the selected time range to the front end for loading rendering.
2. The visual rendering method of page big data according to claim 1, wherein the step S3 comprises:
when the time interval of continuously generating the change data by the target equipment reaches a preset minute value, integrating the change data in the preset minute, and storing the integrated data in the preset minute into a minute-level record table;
when the time interval of continuously generating the change data by the target equipment reaches a preset hour value, integrating the change data in the preset hour, and storing the integrated data in the preset hour into an hour-level record table.
3. The visual rendering method of page big data according to claim 1, wherein in the step S4, the user-selected time range includes a default selectable time range in a front-end page preset information list, a custom selectable time range in a page preset information list, or a time range selected on a time axis.
4. The visual rendering method of page big data according to claim 1, further comprising step a, before said step S2:
A. creating an event record table in a database, wherein the event record table is an automatic tag point table and/or a manual tag point table;
between step S3 and step S4, further comprising step B:
B. when the target equipment generates an event, capturing event data and event occurrence time, storing the event data and the event occurrence time into an event record table, and generating a time positioning label of a corresponding event in the event record table;
after step S4, step C is further included:
C. and according to the selected time range, sending the time positioning label and the event data in the selected time range to the front end to render the current broken line with the time positioning label.
5. The visual rendering method of page big data according to claim 1, further comprising, after step S6, steps D to G:
D. receiving an instruction of zooming the size of the rendering graph input by a user;
E. acquiring a current time range in real time according to an instruction of zooming the size of the rendering graph;
F. determining a sampling step length of the current time range according to the current time range, calculating the time dimension to be loaded in real time according to the sampling step length of the current time range, and acquiring display data in the current time range from a second-level record table, a minute-level record table or an hour-level record table;
G. and sending the display data in the current time range to the front end for loading rendering.
6. The visual rendering method of the page big data is characterized by being applied to the front end and comprising the following steps:
collecting a time range selected by a user;
the time range selected by the user is sent to the server;
receiving display data in a selected time range sent by a server, and loading, rendering and displaying the display data in the selected time range; the database of the server is created with a second-level record table, a minute-level record table and an hour-level record table; the server is used for monitoring and collecting change data generated by the target equipment every second, and storing the change data every second into a second-level record table; the server is also used for correspondingly integrating the change data according to the time interval of continuously generating the change data by the target equipment, and correspondingly storing the integrated data into a minute-level record table or an hour-level record table; the server is further configured to determine a sampling step size of the selected time range according to the selected time range, and calculate a time dimension to be loaded according to the sampling step size of the selected time range, and obtain presentation data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table, where the presentation data specifically includes:
second-level time stamp T of acquisition start time 1 Second-level timestamp T of end time 2 Determining the sampling step length as (T) 2 -T 1 ) The method comprises the steps of carrying out a first treatment on the surface of the According to the sampling step (T) of the selected time range 2 -T 1 ) Calculating the time dimension to be loaded, wherein the calculation formula is as follows: (T) 2 -T 1 ) P=n, where P is the maximum number of rendering points in the visual range, 100+.p+. 10000, a variable constant; n is a variable, when 1< n <24. Time second level record table, 24< n <24×30 minute recording table, 24×30<n takes the hour level record table.
7. The visual rendering system of page big data is characterized by being applied to a server and comprising the following components:
the time dimension record table creating module is used for creating a second-level record table, a minute-level record table and an hour-level record table in the database respectively;
the second-level data writing module is used for monitoring and collecting the change data generated by the target equipment every second and storing the change data every second into the second-level record table;
the minute-level hour-level data integrating module is used for correspondingly integrating the change data according to the time interval of continuously generating the change data by the target equipment and correspondingly storing the integrated data into a minute-level record table or an hour-level record table;
the first time range receiving module is used for receiving a time range selected by a user and collected by the front end;
the first display data acquisition module is used for determining a sampling step length of a selected time range according to the selected time range, calculating a time dimension to be loaded according to the sampling step length of the selected time range, and acquiring display data in the selected time range from a second-level record table, a minute-level record table or an hour-level record table, wherein the first display data acquisition module is used for acquiring the display data in the selected time range specifically comprises the following steps:
second-level time stamp T of acquisition start time 1 Second-level timestamp T of end time 2 Determining the sampling step length as (T) 2 -T 1 ) The method comprises the steps of carrying out a first treatment on the surface of the According to the sampling step (T) of the selected time range 2 -T 1 ) Calculating the time dimension to be loaded, wherein the calculation formula is as follows: (T) 2 -T 1 ) P=n, where P is the maximum number of rendering points in the visual range, 100+.p+. 10000, a variable constant; n is a variable, when 1< n <24. Time second level record table, 24< n <24×30 minute recording table, 24×30<n is taken as an hour level record table;
the first display data sending module is used for sending display data in a selected time range to the front end for loading rendering.
8. The page big data visualization rendering system of claim 7, further comprising, prior to the write second data module:
the event record table creating module is used for creating an event record table in the database, wherein the event record table is an automatic tag point table and/or a manual tag point table;
between the integrating minute-level hour-level data module and the receiving first time range module, further comprising:
the event information storing module is used for capturing event data and event occurrence time when the target equipment generates an event, storing the event data and the event occurrence time into the event record table, and generating a time positioning label of a corresponding event in the event record table;
after receiving the first time range module, further comprising:
and the event information sending module is used for sending the time positioning label and the event data in the selected time range to the front end to render the current broken line with the time positioning label according to the selected time range.
9. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the page big data visualization rendering method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410058515.0A CN117573953B (en) | 2024-01-16 | 2024-01-16 | Page big data visual rendering method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410058515.0A CN117573953B (en) | 2024-01-16 | 2024-01-16 | Page big data visual rendering method, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117573953A CN117573953A (en) | 2024-02-20 |
| CN117573953B true CN117573953B (en) | 2024-03-22 |
Family
ID=89884879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410058515.0A Active CN117573953B (en) | 2024-01-16 | 2024-01-16 | Page big data visual rendering method, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117573953B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014089460A2 (en) * | 2012-12-07 | 2014-06-12 | Lithium Technologies, Inc. | Device, method and user interface for presenting analytic data |
| CN107948016A (en) * | 2017-12-04 | 2018-04-20 | 科大国创软件股份有限公司 | A kind of visual query of monitor control index data and the method for various dimensions overlay analysis |
| CN108470033A (en) * | 2018-02-01 | 2018-08-31 | 杭州电子科技大学 | A kind of city public bicycle system borrows the visual analysis method of also pattern |
| CN111046306A (en) * | 2019-12-30 | 2020-04-21 | 四川华迪信息技术有限公司 | Method for dynamically displaying employment information analysis result chart by browser |
| US10657099B1 (en) * | 2015-11-02 | 2020-05-19 | Quest Software Inc. | Systems and methods for transformation and analysis of logfile data |
| CN112069200A (en) * | 2020-08-31 | 2020-12-11 | 武汉大学 | Multi-detail-level visualization method for high-dimensional space-time big data |
| CN112732824A (en) * | 2021-01-15 | 2021-04-30 | 绿瘦健康产业集团有限公司 | Index data visualization system |
| CN113128134A (en) * | 2021-06-17 | 2021-07-16 | 中国矿业大学(北京) | Mining area ecological environment evolution driving factor weight quantitative analysis method |
| CN113761428A (en) * | 2020-06-05 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Page rendering method, device and system, storage medium and electronic equipment |
| CN114490840A (en) * | 2021-10-29 | 2022-05-13 | 中远海运科技股份有限公司 | Automatic generation method and system for date dimension table |
| CN116167864A (en) * | 2023-02-27 | 2023-05-26 | 中国建设银行股份有限公司 | Data processing method, apparatus, device, storage medium, and program product |
| CN116450418A (en) * | 2023-06-07 | 2023-07-18 | 成都云祺科技有限公司 | Breakpoint difference synchronization method and system based on volume backup and volume real-time backup method |
-
2024
- 2024-01-16 CN CN202410058515.0A patent/CN117573953B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014089460A2 (en) * | 2012-12-07 | 2014-06-12 | Lithium Technologies, Inc. | Device, method and user interface for presenting analytic data |
| US10657099B1 (en) * | 2015-11-02 | 2020-05-19 | Quest Software Inc. | Systems and methods for transformation and analysis of logfile data |
| CN107948016A (en) * | 2017-12-04 | 2018-04-20 | 科大国创软件股份有限公司 | A kind of visual query of monitor control index data and the method for various dimensions overlay analysis |
| CN108470033A (en) * | 2018-02-01 | 2018-08-31 | 杭州电子科技大学 | A kind of city public bicycle system borrows the visual analysis method of also pattern |
| CN111046306A (en) * | 2019-12-30 | 2020-04-21 | 四川华迪信息技术有限公司 | Method for dynamically displaying employment information analysis result chart by browser |
| CN113761428A (en) * | 2020-06-05 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Page rendering method, device and system, storage medium and electronic equipment |
| CN112069200A (en) * | 2020-08-31 | 2020-12-11 | 武汉大学 | Multi-detail-level visualization method for high-dimensional space-time big data |
| CN112732824A (en) * | 2021-01-15 | 2021-04-30 | 绿瘦健康产业集团有限公司 | Index data visualization system |
| CN113128134A (en) * | 2021-06-17 | 2021-07-16 | 中国矿业大学(北京) | Mining area ecological environment evolution driving factor weight quantitative analysis method |
| CN114490840A (en) * | 2021-10-29 | 2022-05-13 | 中远海运科技股份有限公司 | Automatic generation method and system for date dimension table |
| CN116167864A (en) * | 2023-02-27 | 2023-05-26 | 中国建设银行股份有限公司 | Data processing method, apparatus, device, storage medium, and program product |
| CN116450418A (en) * | 2023-06-07 | 2023-07-18 | 成都云祺科技有限公司 | Breakpoint difference synchronization method and system based on volume backup and volume real-time backup method |
Non-Patent Citations (1)
| Title |
|---|
| 基于数据中台系统的可视化设计与实现;章杰;中国优秀硕士学位论文全文数据库 (信息科技辑);20231015;I138-80 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117573953A (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8842119B2 (en) | Displaying system performance information | |
| Krstajic et al. | Cloudlines: Compact display of event episodes in multiple time-series | |
| CN110990445B (en) | Data processing method, device, equipment and medium | |
| CN111752793A (en) | System abnormity monitoring method and device, computer equipment and storage medium | |
| US8854398B2 (en) | Waveform observing apparatus and waveform observing system | |
| EP1894087A2 (en) | User interface for web analytics tools and methods for automatic generation of calendar notes, targets and alerts | |
| US7908239B2 (en) | System for storing event data using a sum calculator that sums the cubes and squares of events | |
| US10936472B2 (en) | Screen recording preparation method for evaluating software usability | |
| CN107870795B (en) | Method and device for displaying electronic map | |
| CN112035320B (en) | Service monitoring method and device, electronic equipment and readable storage medium | |
| CN111651966A (en) | Data report file generation method and device and electronic equipment | |
| US20090106290A1 (en) | Method of analyzing manufacturing process data | |
| CN110781063B (en) | Method, device, equipment and storage medium for measuring first screen time of web page | |
| CN117573953B (en) | Page big data visual rendering method, system and storage medium | |
| US8588587B2 (en) | Navigation interface with ghost region | |
| CN111784176A (en) | Data processing method, device, server and medium | |
| US8660887B2 (en) | Method for visual presentation of key performance indicators of a business utilizing a squarified tree map including selectively displaying additional node data | |
| CN112700511B (en) | Data drawing method and device, electronic equipment and storage medium | |
| CN107885807B (en) | File saving method and device, intelligent tablet and storage medium | |
| CN111047433B (en) | Analysis method, analysis device, server and storage medium for user anomaly reasons | |
| CN110096601B (en) | Method, device, medium, server and user terminal for determining chart key area | |
| US8929712B2 (en) | Display for displaying progress | |
| CN112000549B (en) | Capacity analysis method of storage equipment and related device | |
| CN112291297B (en) | Information data processing method, device, storage medium and electronic equipment | |
| CN115471041B (en) | Identification method, device, equipment and storage medium of black product account |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |