CN110618814A - Data visualization method and device, electronic equipment and computer readable storage medium - Google Patents
Data visualization method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN110618814A CN110618814A CN201910882445.XA CN201910882445A CN110618814A CN 110618814 A CN110618814 A CN 110618814A CN 201910882445 A CN201910882445 A CN 201910882445A CN 110618814 A CN110618814 A CN 110618814A
- Authority
- CN
- China
- Prior art keywords
- data
- visualization
- data source
- agent
- predetermined
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000013079 data visualisation Methods 0.000 title claims abstract description 25
- 238000012800 visualization Methods 0.000 claims abstract description 149
- 238000012545 processing Methods 0.000 claims abstract description 37
- 230000008569 process Effects 0.000 claims description 31
- 230000000007 visual effect Effects 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012549 training Methods 0.000 description 28
- 238000013135 deep learning Methods 0.000 description 18
- 238000002474 experimental method Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000005034 decoration Methods 0.000 description 2
- 235000019800 disodium phosphate Nutrition 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/38—Creation or generation of source code for implementing user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application discloses a data visualization method, a data visualization device, electronic equipment and a computer-readable storage medium, wherein the data visualization method comprises the following steps: acquiring data of a corresponding data source through at least one of a visualization tool Tensorb and a Tensorb agent of a visualization system, wherein the data stored in a local data file of the visualization system is acquired through the Tensorb agent, and the data stored in a predetermined system except the visualization system is acquired through the Tensorb agent; and respectively carrying out visualization processing on the data of the corresponding data source through the Tensoboard to generate a visualization chart corresponding to the data of the corresponding data source. The method of the embodiment of the application realizes cross-system use of the visualization system, can dynamically load or delete the corresponding data source according to needs, and realizes dynamic adjustment of the data to be visualized.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a data visualization method and device, electronic equipment and a computer-readable storage medium.
Background
In the training process of the deep learning network, the visualization of data is very important, and a developer can understand and timely find problems existing in the training process. The developer can install a visualization tool locally and view experimental data generated by the local deep learning network in the training process through the visualization tool.
However, in the specific implementation process, the inventor of the present application finds that: the current visualization tool needs a developer to independently install in each container or virtual machine of a training deep learning network, cross-container use of the visualization tool cannot be realized, experimental data to be checked needs to be specified in advance when the experimental data are presented through the visualization tool, the experimental data to be checked cannot be dynamically adjusted, and great inconvenience is brought to the developer.
Disclosure of Invention
The purpose of the embodiments of the present application is to solve at least one of the above technical drawbacks, and to provide the following technical solutions:
in one aspect, a data visualization method is provided, including:
acquiring data of a corresponding data source through at least one of a visualization tool Tensorb and a Tensorb agent of a visualization system, wherein the data stored in a local data file of the visualization system is acquired through the Tensorb agent, and the data stored in a predetermined system except the visualization system is acquired through the Tensorb agent;
and respectively carrying out visualization processing on the data of the corresponding data source through the Tensoboard to generate a visualization chart corresponding to the data of the corresponding data source.
In a possible implementation manner, after obtaining data stored in a predetermined system except the visualization system through the tensorbard agent, the method further comprises the following steps:
and storing the acquired data stored in the predetermined system into a local data file.
In a possible implementation manner, the visualizing the data of the corresponding data source through the tensorbard includes:
and carrying out visualization processing on the data stored in the local data file through the Tensoboard.
In one possible implementation, the data stored in the predetermined system except the visualization system is obtained by the tensorbard agent, and the data comprises any one of the following items:
determining configuration information of a data source of a predetermined system through a Tensoboard agent, and acquiring corresponding data according to the configuration information, wherein the configuration information comprises a link address or a directory address of the data source;
and sending a data acquisition request to the predetermined system at a predetermined time interval through the TensorBoard agent so as to acquire data of a data source of the predetermined system.
In a possible implementation manner, acquiring corresponding data according to the configuration information includes:
detecting whether a link address or a directory address of a data source meets a preset condition;
and if the preset conditions are met, establishing a corresponding data acquisition process, and acquiring the data of the data source according to the link address or the directory address of the data source.
In one possible implementation, the predetermined condition includes at least one of:
the link address or directory address of the data source belongs to a predetermined format;
data specified by the link address or directory address of the data source exists.
In a possible implementation manner, after generating the visual charts corresponding to the data of the respective data sources, the method further includes:
receiving a display request for a visualization chart;
and displaying the generated visual chart according to the display request.
In one aspect, a data visualization apparatus is provided, including:
the acquisition module is used for acquiring data of a corresponding data source through at least one of a visualization tool Tenscoreboard and a Tenscoreboard agent of the visualization system, wherein the data stored in a local data file of the visualization system is acquired through the Tenscoreboard, and the data stored in a predetermined system except the visualization system is acquired through the Tenscoreboard agent;
and the processing module is used for respectively carrying out visualization processing on the data of the corresponding data source through the Tensoboard and generating a visualization chart corresponding to the data of the corresponding data source.
In a possible implementation manner, the system further comprises a storage module;
and the storage module is used for storing the acquired data stored in the predetermined system into a local data file.
In a possible implementation manner, the processing module is specifically configured to perform visualization processing on data stored in the local data file through a tensorbard.
In one possible implementation, the obtaining module is configured to perform at least one of:
determining configuration information of a data source of a predetermined system through a Tensoboard agent, and acquiring corresponding data according to the configuration information, wherein the configuration information comprises a link address or a directory address of the data source;
and sending a data acquisition request to the predetermined system at a predetermined time interval through the TensorBoard agent so as to acquire data of a data source of the predetermined system.
In a possible implementation manner, when the obtaining module obtains the corresponding data according to the configuration information, the obtaining module is specifically configured to:
detecting whether a link address or a directory address of a data source meets a preset condition;
and if the preset conditions are met, establishing a corresponding data acquisition process, and acquiring the data of the data source according to the link address or the directory address of the data source.
In one possible implementation, the predetermined condition includes at least one of:
the link address or directory address of the data source belongs to a predetermined format;
data specified by the link address or directory address of the data source exists.
In a possible implementation manner, the system further comprises a display module;
the display module is used for receiving a display request aiming at the visual chart; and the display module is used for displaying the generated visual chart according to the display request.
In one aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the data visualization method is implemented.
In one aspect, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the data visualization method described above.
According to the data visualization method provided by the embodiment of the application, data stored in a local data file of a visualization system is obtained through a Tenscoreboard, and data stored in a predetermined system except the visualization system is obtained through a Tenscoreboard agent, so that the visualization system can generate a visualization chart from the data stored in the local data file and generate a visualization chart from the data stored in the predetermined system except the visualization system, cross-system use of the visualization system is realized, and meanwhile, the Tenscoreboard agent can dynamically load or delete corresponding data sources according to needs, thereby realizing dynamic adjustment of the data needing visualization, and greatly improving convenience.
Additional aspects and advantages of embodiments of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of embodiments of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow chart of a data visualization method according to an embodiment of the present application;
FIG. 2 is a process diagram of data visualization according to an embodiment of the present application;
fig. 3 is a schematic diagram of a basic structure of a data visualization apparatus according to an embodiment of the present application;
fig. 4 is a detailed structural diagram of a data visualization apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.
The following describes in detail the technical solutions of the embodiments of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
One embodiment of the present application provides a data visualization method, which is executed by a computer device, and the computer device may be a terminal or a server. The terminal may be a desktop device or a mobile terminal. The servers may be individual physical servers, clusters of physical servers, or virtual servers. As shown in fig. 1, the method includes:
and step S110, acquiring data of a corresponding data source through at least one of a visualization tool Tenscoreboard and a Tenscoreboard agent of the visualization system, wherein the data stored in a local data file of the visualization system is acquired through the Tenscoreboard, and the data stored in a predetermined system except the visualization system is acquired through the Tenscoreboard agent.
Specifically, the TensorBoard is a set of visualization tools provided by tensorb flow (a symbolic mathematical system based on data flow programming), and can help developers to conveniently understand, debug and optimize the tensorb flow program. The TensorBoard is a common visual tool in the training process of the deep learning network, and can be used for conveniently observing process information, network structures and the like in the training process of the deep learning network.
Specifically, the TensorBoard can perform visualization processing on the following data in the deep learning network training process: (1) scalar data such as a loss function or a learning rate, for example, a change curve of the scalar data such as the loss function or the learning rate is generated; (2) weights for model training, such as generating a histogram of the weights for model training; (3) pictures in the training process, such as presenting the pictures in the training process to the developer; (4) the hyper-parameters used by the model training, such as generating the change curve of the hyper-parameters; (5) the model structure used for model training, such as generating a relational graph of the model structure.
Specifically, the Tensoboard agent is an extension component of the Tensoboard, and forms a visualization system together with the Tensoboard, and the visualization system can perform visualization processing on visualization data in the deep learning network training process. Wherein, the TensorBoard can obtain the data stored in the local data file of the visualization system, and the TensorBoard agent can obtain the data stored in the predetermined system except the visualization system.
In other words, the Tensorboard agent expands the function of the Tensorboard, so that the visualization system can not only visualize the locally stored data, but also visualize the data stored in a predetermined system except the visualization system, thereby realizing the cross-system use of the visualization system. The predetermined system includes, but is not limited to, a certain virtual machine or a certain container running the deep learning network, an experiment training platform running the deep learning network, a storage system storing experiment data generated during training of the deep learning network, and the like. The Tensoborker agent can dynamically load or delete the corresponding data source according to the requirement, such as loading data in a certain virtual machine running the deep learning network, deleting data in an experiment training platform running the deep learning network, and loading experiment data in a storage system, thereby realizing the dynamic adjustment of the data.
And step S120, respectively carrying out visualization processing on the data of the corresponding data source through the Tensoboard, and generating a visualization chart corresponding to the data of the corresponding data source.
Specifically, after the data of the corresponding data source is acquired through at least one of the visualization tools tensorbard and tensorbard agents of the visualization system, the data of the corresponding data source can be respectively visualized through the tensorbard of the visualization system, and a visualization chart corresponding to the data of the corresponding data source is generated. The visualization chart includes, but is not limited to, a graph, a histogram, a split line graph, and the like. For example, the data stored in the local data file is visualized through the Tensorboard, and the data stored in a predetermined system acquired by the Tensorboard agent is visualized through the Tensorboard.
According to the data visualization method provided by the embodiment of the application, data stored in a local data file of a visualization system is obtained through a Tenscoreboard, and data stored in a predetermined system except the visualization system is obtained through a Tenscoreboard agent, so that the visualization system can generate a visualization chart from the data stored in the local data file and generate a visualization chart from the data stored in the predetermined system except the visualization system, cross-system use of the visualization system is realized, and meanwhile, the Tenscoreboard agent can dynamically load or delete corresponding data sources according to needs, thereby realizing dynamic adjustment of the data needing visualization, and greatly improving convenience.
In a possible implementation manner, after the data stored in the predetermined system except the visualization system is acquired through the tensorbard agent, the acquired data stored in the predetermined system can be stored in the local data file.
Specifically, after the tensorbard agent obtains the data of the corresponding data source, that is, after obtaining the corresponding data from the predetermined system, the obtained data may be stored in a local data file of the visualization system, so that the obtained data may be visualized through the tensorbard agent. The TenSorboard agent stores the acquired data into a local data file, the TenSorboard agent can be better compatible with a processing process of the TenSorboard, an additional development visualization processing process is not needed, and the workload of a developer is greatly reduced.
Specifically, in the process of performing visualization processing on the data of the corresponding data source through the tensorbard, the data stored in the local data file may be subjected to visualization processing through the tensorbard. When the data stored in the local data file is subjected to visual processing through the Tensorboard, on one hand, the data stored in the local data file of the visual system acquired through the Tensorboard can be subjected to visual processing, on the other hand, the data acquired from the predetermined system through the Tensorboard agent and stored in the local data file of the visual system can be subjected to visual processing, so that the data of various data sources can be conveniently and quickly subjected to visual processing on the basis of keeping the original visual architecture, and the change of the original visual architecture is small.
In a possible implementation manner, when data stored in a predetermined system except a visualization system is acquired through a Tenscoreboard agent, configuration information of a data source of the predetermined system can be determined through the Tenscoreboard agent, and corresponding data is acquired according to the configuration information, wherein the configuration information comprises a link address or a directory address of the data source; data acquisition requests can also be sent to the predetermined system at predetermined time intervals through the Tensoboard agent to acquire data of the data source of the predetermined system.
Specifically, the developer may configure each data source to be visualized in the predetermined system according to actual requirements, that is, determine each data to be visualized, so as to perform visualization processing on each data corresponding to each data source. For example, configuring data sources a _1, a _2, …, a _ N, etc. in the virtual machine a so as to perform visualization processing on data corresponding to the data sources a _1, a _2, …, a _ N, etc.; for another example, the data sources B _1, B _2, …, B _ N, etc. in the container B are configured so as to perform visualization processing on the data corresponding to the data sources B _1, B _2, …, B _ N, etc.; for another example, the data sources C _1, C _2, …, C _ N, etc. in the experimental training platform C are configured so as to perform visualization processing on the data corresponding to the data sources C _1, C _2, …, C _ N, etc., respectively; for another example, the data sources D _1, D _2, …, D _ N, etc. in the storage system D are configured to perform visualization processing on the data corresponding to the data sources D _1, D _2, …, D _ N, etc.
Specifically, when configuring each data source to be visualized in the predetermined system, the developer may configure each data source by specifying a link address or a directory address of each data source. The following examples show:
a. directory addresses of several data sources in a certain container B are specified, for example:
i.ws://my_workspace/my_folder/my_experiment/log/
ii.ws://my_workspacce2/my_folder/my_experiment/log/
b. specifying the directory address of a data source in a storage system for experimental data, for example:
i.oss://my_bucket/my_experiment/log/
c. specifying a directory address for a data source in a full-scale experimental training platform, for example:
i.exphub://my_experiment_id/data_bundle_id/log/
d. specifying the directory address of a data source of a virtual machine a that is running a deep learning network, for example:
i.vm://my_experiment_vm/exp_id/run_id/tfevents/
e. specifying a directory address of a running experimental data source on an experimental training platform, for example:
i.exphub://my_experiment_id/run_id/tfevents/
f. specifying a directory address of a certain data source of storage in the storage system of experimental data, for example:
i.oss://my_experiment_bucket/run_id/tfevents/
g. specifying the link address of a certain data source stored in the storage system of experimental data, for example:
i.http://oss.com/my_experiment_bucket/run_id/tfevents/
specifically, after configuring each data source to be visualized in the predetermined system, the developer may send a corresponding configuration request to the tensorbard agent, where the configuration request carries the configuration information of each configured data source, that is, the developer sends the configuration information of each data source to the tensorbard agent, and the configuration information includes a link address or a directory address of the data source. Correspondingly, the Tensorboard agent receives the configuration information, wherein after receiving the configuration information, the Tensorboard agent can acquire corresponding data according to the configuration information, namely, the Tensorboard agent acquires data specified by the link address or the directory address according to the link address or the directory address of each data source.
Specifically, in the process of acquiring corresponding data according to the received configuration information, the tensorbard agent may first detect whether a link address or a directory address of a data source in the configuration information meets a predetermined condition, and establish a corresponding data acquisition process when the link address or the directory address of the data source meets the predetermined condition, so as to acquire data of the data source according to the link address or the directory address of the data source.
Wherein the predetermined condition comprises at least one of:
the link address or directory address of the data source belongs to a predetermined format;
data specified by the link address or directory address of the data source exists.
Specifically, the Tensobord agent can detect whether the link address or directory address of the data source belongs to the link address or directory address of the predetermined format, and can also detect whether the data specified by the link address or directory address of the data source exists. For example, when the link address or the directory address of the data source is determined to belong to the link address or the directory address of the predetermined format, determining that the link address or the directory address of the data source meets the predetermined condition; for another example, when the detection determines that the data specified by the link address or the directory address of the data source exists, it is determined that the link address or the directory address of the data source satisfies the predetermined condition; for another example, when the link address or the directory address of the data source is determined to belong to the link address or the directory address of the predetermined format and the data specified by the link address or the directory address of the data source exists, it is determined that the link address or the directory address of the data source satisfies the predetermined condition.
Specifically, after determining that the link address or directory address of the data source meets the predetermined condition, corresponding data acquisition processes may be respectively established according to each data source to acquire data of each data source. When the link address or the directory address of the data source is determined not to meet the preset condition, corresponding error prompt information can be returned.
Specifically, in the process of respectively establishing corresponding data acquisition processes according to each data source to acquire data of each data source, different data acquisition processes may be established for different data sources, that is, corresponding data acquisition processes are respectively established for each data source to acquire data of each data source. For example, when the predetermined system is a virtual machine a running a deep learning network, that is, when the data source is a directory address or a link address in the virtual machine a, the directory address or the link address of the virtual machine a may be incrementally synchronized into a local data file of the visualization system by rsync (data mirror backup tool under linux system) based on ssh (Secure Shell); for another example, when the predetermined system is an experimental training platform C running a deep learning network, data generated in the training process may be incrementally downloaded to a local data file of the visualization system through an HTTP (Hyper text transfer Protocol) interface provided by the experimental training platform C; for another example, when the predetermined system is a storage system D for storing experimental data generated during deep learning network training, data generated during training may be incrementally downloaded to a local data file of the visualization system through a standard HTTP interface provided by the storage system.
Specifically, after the developer designates each data source to be visualized in the predetermined system, the tensorbard agent may send a data acquisition request to the predetermined system at a predetermined time interval to acquire data of the designated each data source. For example, the tensorbard agent sends a data acquisition request to a predetermined system at time intervals of 30 seconds, 2 minutes, 5 minutes, and the like, so as to dynamically acquire data of each specified data source, and especially when the data of each specified data source is updated (for example, modified, added, deleted, and the like) within the time intervals, the tensorbard agent can be ensured to dynamically acquire the updated data, and dynamic loading, updating, deleting, and the like of the data are realized.
Specifically, after generating the visual charts corresponding to the data of the corresponding data sources, a display request for the visual charts may be further received, and the generated visual charts are displayed according to the display request. As an optional mode, after the visualization system generates the visualization charts corresponding to the data of each data source, the visualization system may load the generated visualization charts onto corresponding web pages, and when a developer needs to view the generated visualization charts, the web pages corresponding to the visualization system may be opened to view all the visualization charts generated by the visualization system.
The method of the embodiment of the present application is described below by way of a specific example, as shown in fig. 2:
step S1, the developer is provided with a visualization system management interface on which the developer can click "create" to automatically create a visualization system in which the tensorber and tensorber Proxy (i.e., tensorber-Proxy) are running. The Tensoboard is mainly responsible for reading data in a local data file of a visualization system and then drawing the data on a webpage through various visualization charts such as a line-splitting chart, a histogram and the like. The Tensoborard-Proxy is mainly responsible for externally receiving a configuration request of a predetermined system (such as an experiment training platform, an experiment data storage system, a virtual machine and the like) and pulling data of a data source to a local data file in real time.
In step S2, the developer may view a list including all the visualization systems through the visualization system management interface, and the developer may optionally select one visualization system for data source configuration, and specifically may configure a link address or a directory address of the data source, for example:
(1) specifying the directory address of a data source of a virtual machine a that is running a deep learning network, such as: vm:// my _ experience _ vm/exp _ id/run _ id/tfovents/;
(2) specifying a directory address of a running experimental data source on an experimental training platform, such as: expubb:// my _ experience _ id/run _ id/tfovents/;
(3) specifying a directory address of a data source of storage in a storage system for experimental data, such as: oss:// my _ experience _ bucket/run _ id/tfevents/.
Step S3, after the developer configures the data source, the click determination will send the configuration request to the Tensoborard-Proxy, after receiving the configuration request, the Tensoborard-Proxy processes the following steps:
step S3_ 1: checking whether the configured data source is valid, such as checking whether the configured data source is in a supported data source format, such as checking whether data specified by the data source exists, returning an error message if the configured data source is invalid, and executing the following step S3_2 if the configured data source is valid;
step S3_ 2: a corresponding data acquisition process is created according to the configured data source, and in the data acquisition process, different data acquisition modes can be determined for different data sources, for example:
(1) when the data source comes from a virtual machine running a deep learning network, data of the virtual machine can be incrementally synchronized into a local data file of a visualization system through rsync based on ssh;
(2) when the data source comes from the experiment training platform, data generated by the experiment under the data source can be incrementally downloaded to a local data file of the visualization system through an HTTP interface provided by the experiment platform;
(3) when the data source comes from the storage system of the experimental data, the data generated by the experiment under the data source can be incrementally downloaded to the local data file of the visualization system through the standard HTTP interface provided by the storage system.
In step S4, the tensorbard-Proxy may update the data of the data source of the predetermined system to the local data file of the visualization system in real time at a set frequency.
Step S5, when the data in the local data file of the visualization system is updated, the Tenboard dynamically loads the data in the local data file, generates a corresponding visualization chart, and updates and displays the generated visualization chart on a corresponding webpage;
in step S6, the developer can arbitrarily select one of the visualization systems from the list including the respective visualization systems, and then click on the web page that opens the selected visualization system to view all the visualization charts generated by the visualization system.
Fig. 3 is a schematic structural diagram of a data visualization apparatus according to another embodiment of the present application, and as shown in fig. 3, the apparatus 30 may include: an obtaining module 31 and a processing module 32, wherein:
an obtaining module 31, configured to obtain data of a corresponding data source through at least one of a visualization tool tensorb and a tensorb agent of a visualization system, where the data stored in a local data file of the visualization system is obtained through the tensorb, and the data stored in a predetermined system other than the visualization system is obtained through the tensorb agent;
and the processing module 32 is configured to perform visualization processing on the data of the corresponding data source through the tensorbard, and generate a visualization chart corresponding to the data of the corresponding data source.
The device that this application embodiment provided, acquire the data of storage in the local data file of visual system through the Tensorboard, acquire the data of storage in the predetermined system except visual system through the Tensorboard agent, make visual system not only can generate visual chart with the data of storage in the local data file, and can generate visual chart with the data of storage in the predetermined system except visual system, thereby the cross-system use of visual system has been realized, simultaneously Tensorboard agent can be as required dynamic loading or delete corresponding data source, thereby the dynamic adjustment of the data that need visualize has been realized, greatly improve the convenience.
Fig. 4 is a detailed structural schematic diagram of a data visualization apparatus according to still another embodiment of the present application, and as shown in fig. 4, the apparatus 40 may include an obtaining module 41, a processing module 42, a storage module 43, and a display module 44, where functions implemented by the obtaining module 41 in fig. 4 are the same as those implemented by the obtaining module 31 in fig. 3, and functions implemented by the processing module 42 in fig. 4 are the same as those implemented by the processing module 32 in fig. 3, and are not repeated herein. The data visualization device shown in fig. 4 is described in detail below:
in a possible implementation manner, the system further includes a storage module 43, where:
and the storage module 43 is configured to store the acquired data stored in the predetermined system into a local data file.
In a possible implementation manner, the processing module 42 is specifically configured to perform visualization processing on data stored in the local data file through a tensorbard.
In one possible implementation, the obtaining module 41 is configured to perform at least one of the following:
determining configuration information of a data source of a predetermined system through a Tensoboard agent, and acquiring corresponding data according to the configuration information, wherein the configuration information comprises a link address or a directory address of the data source;
and sending a data acquisition request to the predetermined system at a predetermined time interval through the TensorBoard agent so as to acquire data of a data source of the predetermined system.
In a possible implementation manner, when the obtaining module 41 obtains the corresponding data according to the configuration information, it is specifically configured to:
detecting whether a link address or a directory address of a data source meets a preset condition;
and if the preset conditions are met, establishing a corresponding data acquisition process, and acquiring the data of the data source according to the link address or the directory address of the data source.
In one possible implementation, the predetermined condition includes at least one of:
the link address or directory address of the data source belongs to a predetermined format;
data specified by the link address or directory address of the data source exists.
In a possible implementation, the system further includes a display module 44, wherein:
the display module 44 is configured to receive a display request for a visualization chart; and the display module is used for displaying the generated visual chart according to the display request.
It should be noted that the present embodiment is an apparatus embodiment corresponding to the method embodiment described above, and the present embodiment can be implemented in cooperation with the method embodiment described above. The related technical details mentioned in the above method embodiments are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the above-described method item embodiments.
Another embodiment of the present application provides an electronic device, as shown in fig. 5, an electronic device 500 shown in fig. 5 includes: a processor 501 and a memory 503. Wherein the processor 501 is coupled to the memory 503, such as via the bus 502. Further, the electronic device 500 may also include a transceiver 504. It should be noted that the transceiver 504 is not limited to one in practical applications, and the structure of the electronic device 500 is not limited to the embodiment of the present application.
The processor 501 is applied in the embodiment of the present application, and is used to implement the functions of the obtaining module and the processing module shown in fig. 3 and fig. 4. The transceiver 504 includes a receiver and a transmitter, and the transceiver 504 is applied in the embodiment of the present application to realize the functions of the storage module and the display module shown in fig. 4.
The processor 501 may be a CPU, general purpose processor, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 501 may also be a combination of implementing computing functionality, e.g., comprising one or more microprocessors, a combination of DSPs and microprocessors, and the like.
Bus 502 may include a path that transfers information between the above components. The bus 502 may be a PCI bus or an EISA bus, etc. The bus 502 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.
The memory 503 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, a CD-ROM or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
The memory 503 is used for storing application program codes for executing the scheme of the application, and the processor 501 controls the execution. The processor 501 is configured to execute application program codes stored in the memory 503 to implement the actions of the data visualization apparatus provided by the embodiment shown in fig. 3 or fig. 4.
The electronic device provided by the embodiment of the application comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the electronic device can realize that: the data stored in the local data file of the visualization system is acquired through the Tensorboard, the data stored in the predetermined system except the visualization system is acquired through the Tensorboard agent, so that the visualization system can generate a visualization chart from the data stored in the local data file and can generate the visualization chart from the data stored in the predetermined system except the visualization system, cross-system use of the visualization system is realized, meanwhile, the Tensorboard agent can dynamically load or delete corresponding data sources according to needs, dynamic adjustment of the data needing visualization is realized, and convenience is greatly improved.
The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the method shown in the first embodiment. Wherein: the data stored in the local data file of the visualization system is acquired through the Tensorboard, the data stored in the predetermined system except the visualization system is acquired through the Tensorboard agent, so that the visualization system can generate a visualization chart from the data stored in the local data file and can generate the visualization chart from the data stored in the predetermined system except the visualization system, cross-system use of the visualization system is realized, meanwhile, the Tensorboard agent can dynamically load or delete corresponding data sources according to needs, dynamic adjustment of the data needing visualization is realized, and convenience is greatly improved.
The computer-readable storage medium provided by the embodiment of the application is suitable for any embodiment of the method.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.
Claims (10)
1. A method of data visualization, comprising:
acquiring data of a corresponding data source through at least one of a visualization tool Tensorb and a Tensorb agent of a visualization system, wherein the Tensorb acquires data stored in a local data file of the visualization system, and the Tensorb agent acquires data stored in a predetermined system except the visualization system;
and respectively carrying out visualization processing on the data of the corresponding data source through the Tensoboard to generate a visualization chart corresponding to the data of the corresponding data source.
2. The method according to claim 1, further comprising, after said obtaining by said Tensioboard agent data stored in a predetermined system other than said visualization system:
and storing the acquired data stored in the predetermined system into the local data file.
3. The method according to claim 2, wherein the visualizing the data of the corresponding data source through the Tensioboard respectively comprises:
and carrying out visualization processing on the data stored in the local data file through the Tenscoreboard.
4. The method according to claim 1, wherein said obtaining, by said Tensioboard agent, data stored in a predetermined system other than said visualization system comprises any of:
determining configuration information of a data source of the predetermined system through the Tensoboard agent, and acquiring corresponding data according to the configuration information, wherein the configuration information comprises a link address or a directory address of the data source;
sending a data acquisition request to the predetermined system at a predetermined time interval through the TensorBoard agent so as to acquire data of a data source of the predetermined system.
5. The method according to claim 4, wherein the obtaining the corresponding data according to the configuration information comprises:
detecting whether the link address or the directory address of the data source meets a preset condition;
and if the preset conditions are met, establishing a corresponding data acquisition process, and acquiring the data of the data source according to the link address or the directory address of the data source.
6. The method of claim 5, wherein the predetermined condition comprises at least one of:
the link address or directory address of the data source belongs to a predetermined format;
the data specified by the link address or directory address of the data source exists.
7. The method according to any one of claims 1-6, further comprising, after the generating of the visual charts corresponding to the data of the respective data sources, respectively:
receiving a display request for the visualization chart;
and displaying the generated visual chart according to the display request.
8. A data visualization device, comprising:
the acquisition module is used for acquiring data of a corresponding data source through at least one of a visualization tool Tenscoreboard and a Tenscoreboard agent of a visualization system, wherein the data stored in a local data file of the visualization system is acquired through the Tenscoreboard, and the data stored in a predetermined system except the visualization system is acquired through the Tenscoreboard agent;
and the processing module is used for respectively carrying out visualization processing on the data of the corresponding data source through the Tensoboard and generating a visualization chart corresponding to the data of the corresponding data source.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the data visualization method of any one of claims 1 to 7 when executing the program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the data visualization method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910882445.XA CN110618814A (en) | 2019-09-18 | 2019-09-18 | Data visualization method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910882445.XA CN110618814A (en) | 2019-09-18 | 2019-09-18 | Data visualization method and device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110618814A true CN110618814A (en) | 2019-12-27 |
Family
ID=68923629
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910882445.XA Pending CN110618814A (en) | 2019-09-18 | 2019-09-18 | Data visualization method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110618814A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463872A (en) * | 2020-11-04 | 2021-03-09 | 中国建设银行股份有限公司 | Visual index data acquisition method, device, system and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160070430A1 (en) * | 2014-09-08 | 2016-03-10 | Tableau Software Inc. | Systems and Methods for Providing Drag and Drop Analytics in a Dynamic Data Visualization Interface |
| CN108334948A (en) * | 2018-02-09 | 2018-07-27 | 武汉理工大学 | A kind of mechanical bearing fault diagnosis technology based on wide residual error network learning model |
| CN109254830A (en) * | 2018-09-04 | 2019-01-22 | 郑州云海信息技术有限公司 | Visual management method and device in deep learning system |
| CN109388479A (en) * | 2018-11-01 | 2019-02-26 | 郑州云海信息技术有限公司 | The output method and device of deep learning data in mxnet system |
| CN110032590A (en) * | 2019-04-15 | 2019-07-19 | 重庆天蓬网络有限公司 | A kind of data visualisation system, method, server and storage medium |
-
2019
- 2019-09-18 CN CN201910882445.XA patent/CN110618814A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160070430A1 (en) * | 2014-09-08 | 2016-03-10 | Tableau Software Inc. | Systems and Methods for Providing Drag and Drop Analytics in a Dynamic Data Visualization Interface |
| CN108334948A (en) * | 2018-02-09 | 2018-07-27 | 武汉理工大学 | A kind of mechanical bearing fault diagnosis technology based on wide residual error network learning model |
| CN109254830A (en) * | 2018-09-04 | 2019-01-22 | 郑州云海信息技术有限公司 | Visual management method and device in deep learning system |
| CN109388479A (en) * | 2018-11-01 | 2019-02-26 | 郑州云海信息技术有限公司 | The output method and device of deep learning data in mxnet system |
| CN110032590A (en) * | 2019-04-15 | 2019-07-19 | 重庆天蓬网络有限公司 | A kind of data visualisation system, method, server and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| NIKHIL BUDUMA, 东南大学出版社 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463872A (en) * | 2020-11-04 | 2021-03-09 | 中国建设银行股份有限公司 | Visual index data acquisition method, device, system and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3667500B1 (en) | Using a container orchestration service for dynamic routing | |
| US8375383B2 (en) | Rolling upgrades in distributed applications | |
| CN108563579B (en) | White box testing method, device and system and storage medium | |
| JP2020091835A (en) | Method and device for processing information | |
| CN107608901B (en) | Jmeter-based testing method and device, storage medium and electronic equipment | |
| CN107357890B (en) | Webpage file processing method, device and system and storage equipment | |
| US10303444B2 (en) | Composable application session parameters | |
| US20190258534A1 (en) | Message oriented middleware with integrated rules engine | |
| CN110647332A (en) | Software deployment method and device based on container cloud | |
| US20180159724A1 (en) | Automatic task tracking | |
| CN111010364A (en) | System for offline object-based storage and simulation of REST responses | |
| CN113419818B (en) | Basic component deployment method, device, server and storage medium | |
| CN112887440A (en) | IP address management method and device | |
| CN109299124B (en) | Method and apparatus for updating a model | |
| CN110618814A (en) | Data visualization method and device, electronic equipment and computer readable storage medium | |
| US8245182B2 (en) | Class selectable design sharing | |
| US10073689B2 (en) | Managing application lifecycles within a federation of distributed software applications | |
| CN117149248A (en) | Micro front end construction method, device, equipment and storage medium | |
| CN111767062A (en) | Application configuration data updating method, device, equipment and storage medium | |
| US20130138690A1 (en) | Automatically identifying reused model artifacts in business process models | |
| CN111984543B (en) | Instruction testing method, device and system | |
| CN113407229B (en) | Method and device for generating offline scripts | |
| CN114090242A (en) | Model training method, device, equipment, storage medium and program product | |
| CN113253991A (en) | Task visualization processing method and device, electronic equipment and storage medium | |
| JP6760816B2 (en) | Application development support device and application development support method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191227 |
|
| RJ01 | Rejection of invention patent application after publication |