WO2022007434A1 - Procédé de visualisation et dispositif associé - Google Patents

Procédé de visualisation et dispositif associé Download PDF

Info

Publication number
WO2022007434A1
WO2022007434A1 PCT/CN2021/082348 CN2021082348W WO2022007434A1 WO 2022007434 A1 WO2022007434 A1 WO 2022007434A1 CN 2021082348 W CN2021082348 W CN 2021082348W WO 2022007434 A1 WO2022007434 A1 WO 2022007434A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
visualization
target
training data
data
Prior art date
Application number
PCT/CN2021/082348
Other languages
English (en)
Chinese (zh)
Inventor
朱雁博
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Priority to JP2021570729A priority Critical patent/JP2022543180A/ja
Priority to KR1020217039065A priority patent/KR20220011134A/ko
Publication of WO2022007434A1 publication Critical patent/WO2022007434A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/904—Browsing; Visualisation therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Definitions

  • the present application relates to the field of computer technology, and in particular, to a visualization method and related equipment.
  • deep learning models have been widely used in the field of image and video processing.
  • deep learning models are obtained through training.
  • the training process of deep learning models involves complex and esoteric calculations, and needs to be implemented through multiple iterations.
  • the training time is long, and developers can only evaluate the performance of the deep learning model after training.
  • Embodiments of the present application provide a visualization method and related equipment, which are used to visualize training data of a deep learning model.
  • an embodiment of the present application provides a visualization method, which is applied to a user equipment, and the method includes:
  • the visualization information is displayed.
  • an embodiment of the present application provides a visualization apparatus, which is applied to a user equipment, and the apparatus includes:
  • a sending unit configured to send a visualization request to the cloud server, where the visualization request is used to request visualization of the target training data of the target deep learning model
  • a receiving unit configured to receive a visualization response sent by the cloud server, where the visualization response carries the visualization information of the target training data
  • a display unit for displaying the visual information.
  • an embodiment of the present application provides a visualization method, which is applied to a cloud server, and the method includes:
  • a visualization response is sent to the user equipment, and the visualization response carries the visualization information.
  • an embodiment of the present application provides a visualization device, which is applied to a cloud server, and the device includes:
  • a receiving unit configured to receive a visualization request from the user equipment, where the visualization request is used to request visualization of the target training data of the target deep learning model;
  • a sending unit configured to send a visualization response to the user equipment, where the visualization response carries the visualization information.
  • the present application provides a computer device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured by
  • the above-mentioned processor is executed, and the above-mentioned program includes instructions for executing the steps in the method described in the first aspect or the third aspect of the embodiments of the present application.
  • an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the computer program as described in the first embodiment of the present application.
  • an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute as implemented in the present application.
  • the computer program product may be a software installation package.
  • the user equipment first sends a visualization request to the cloud server, then receives the visualization response sent by the cloud server, the visualization response carries the visualization information of the target training data, and finally displays the visualization information.
  • the visualization information of the data can intuitively understand the training status of the deep learning model, which helps to judge the feasibility of the current training strategy in time, and provides a basis for decision-making such as early stopping.
  • FIG. 1 is a schematic diagram of the architecture of a visualization system provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a visualization method provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a visualization device provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of another visualization apparatus provided by an embodiment of the present application.
  • FIG. 1 is a schematic diagram of the architecture of a visualization system provided by an embodiment of the application, where the visualization system includes a supercomputing cluster, a cloud server, a cloud storage, and user equipment.
  • the visualization system can realize the visualization of multi-dimensional and multi-class intermediate training data based on javascript and svg technology, and supports fast and efficient visualization rendering of a large amount of intermediate training data.
  • the visualization system is based on the kubernetes container orchestration system, which provides stable visualization system deployment at the production level and provides dynamic and controllable service capabilities.
  • the kubernetes container is used to manage containerized applications on multiple hosts in the cloud service.
  • the cloud server provides visualization services, and the visualization services are used to perform data operations such as preprocessing on the acquired intermediate training data, and visualize and render the intermediate training data after data operations to obtain visualization information.
  • the cloud server may further provide at least one of the following services: a service gateway, a training management service, a data storage service, and a permission service.
  • the service gateway can be used as the entrance of at least one of the following services: the entrance of training management service, data storage service, data visualization service and authority service.
  • the service gateway can be an application.
  • the service gateway may have a current limiting function.
  • the training management service can provide at least one of the following services: monitoring whether the training tasks of the deep learning model are completed, recording, indexing and searching for abnormal training tasks, comparing the training process of multiple training tasks of the same deep learning model, and Share the training results of deep learning models.
  • the data storage service can provide at least one of the following services: data persistence, using data threads to store intermediate training data generated by deep learning model training, and using cache to store intermediate training data generated by deep learning model training.
  • the rights management service is used to ensure the security of the visualization system, and the visualization system can only be used after passing the rights management service.
  • shared training is also authenticated through the rights management service, which supports unified authentication account login and/or ordinary There are two types of login methods to register.
  • the cloud server can interact with the developer toolkit through HyperText Transfer Protocol (HTTP), Google Remote Procedure Call (GRPC) protocol or other protocols.
  • HTTP HyperText Transfer Protocol
  • GRPC Google Remote Procedure Call
  • the cloud server can directly store the training data in the cloud storage.
  • the supercomputing cluster provides development tools such as python and other programming language toolkits by running the developer toolkit for users to use with tensorflow, pytorch, caffe frameworks, etc., and/or open a new thread as a dedicated thread for data uploading, and
  • the intermediate training data generated by the training of the deep learning model is uploaded to the cloud server through the HTTP communication protocol and the GRPC protocol on the dedicated thread for data uploading.
  • the uploaded intermediate training data types can include vectors, scalars, pictures, videos, audios, etc. Data is stored in the developer kit's data cache.
  • opening a new thread as a dedicated thread for data uploading can ensure the computing speed of the deep learning model.
  • cloud storage can optionally be distributed storage, which is used for unified management of intermediate training data generated by deep learning models, providing massive storage functions for intermediate training data, and as the amount of intermediate training data increases, it can dynamically
  • the capacity can be expanded to meet the storage requirements of intermediate training data and provide follow-up guarantee for the growth of visualization business.
  • the cloud storage may also be data block-level cloud storage, file-level cloud storage, object-level cloud storage and/or other forms of cloud storage.
  • the user equipment may communicate with the cloud server through preset application software, or communicate with the cloud server through a preset network address.
  • the user equipment may include various handheld devices with communication functions, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to the wireless modem, and various forms of user equipment (User Equipment, UE), Mobile station (Mobile Station, MS), terminal device (terminal device) and so on.
  • UE User Equipment
  • MS Mobile Station
  • terminal device terminal device
  • FIG. 2 is a schematic flowchart of a visualization method provided by an embodiment of the present application, which is applied to the above-mentioned user equipment and cloud server, and specifically includes the following steps:
  • Step 201 The user equipment sends a visualization request to the cloud server, where the visualization request is used to request visualization of the target training data of the target deep learning model.
  • Step 202 The cloud server receives a visualization request from the user equipment, where the visualization request is used to request visualization of the target training data of the target deep learning model.
  • the visualization request carries the training task identifier of the target deep learning model.
  • the cloud server can obtain the relevant training data of the training task based on the training task identifier of the target deep learning model.
  • the stored training data related to the training tasks of each deep learning model may be associated with the training task identifier of the deep learning model.
  • training data may be stored according to training tasks, and relevant training data of different training tasks are stored in different locations.
  • training data may be stored according to deep learning models or data types.
  • the data type can be a two-dimensional heat map, a three-dimensional histogram, a feature map, a call sequence diagram, a scalar line graph, a directed acyclic graph, and so on.
  • the training tasks are all located on the supercomputing cluster, and the training of the deep learning model can be accelerated by using the central processing unit of the supercomputing cluster.
  • the training data may be stored in association with the task identifier of the training task to which the training data belongs, and so on, which is not limited in this embodiment of the present disclosure.
  • the visualization request carries the training task identification of the target deep learning model and the index information of the target training data, and the index information is used to find the target training data to determine the storage location of the target training data.
  • the index information may be implemented in multiple ways.
  • the index information may include any one or more of data creation time, data type, and data label.
  • the index information includes data type and data label.
  • the same data type can have multiple data labels, and the data labels can be customized.
  • the data labels can be exact values, loss values, and so on.
  • the visualization request may be a request to sample all the data in the intermediate data set corresponding to the index information or randomly sample the data in the intermediate data set according to the training task identifier and the index information.
  • the target deep learning model includes at least one deep neural network, and the target training data may be intermediate data generated by one of the deep neural networks in the target deep learning model, or may be intermediate data generated by multiple deep neural networks in the target deep learning model. .
  • the visualization request may be a request to visualize the intermediate data generated by the target deep learning model in the first period of time, wherein the duration of the first period of time may be other values such as 3min, 5min, 9min, and 15min, and the termination time of the first moment may be is the current time, it can be earlier than the current time, or it can be later than the current time.
  • the target training data is stored in cloud storage.
  • the target training data includes at least one of the following: model performance change trend information, model loss trend information, model parameter distribution information, model processing intermediate results, model structure information, current progress information of model training, Comparing information for different trainings of the same model, and time series of scheduling time information for various processes.
  • the category of information is a scalar line chart.
  • the target training data is the model performance change trend information
  • the target training data is the model loss trend information
  • the target training data is the current progress information of model training
  • the target training data is the comparison information of different trainings of the same model
  • the category of visualization information is a three-dimensional histogram, and it can be determined whether the model parameter distribution is abnormal through the three-dimensional histogram. If the model parameter distribution is abnormal, the model training is stopped.
  • the category of the visual information is the feature map, and through the feature map, it can be determined whether the features extracted by the model are correct, and if the extracted features are wrong, rebuild the model.
  • the category of the visualization information is a directed acyclic graph
  • the directed acyclic graph can determine whether the structure of the model is correct, and if the structure of the model is wrong, rebuild the The structure of the model.
  • the architecture of the deep learning model supported by the directed acyclic graph can be a standard architecture such as Open Neural Network Exchange (ONNX), or other types of architectures, in which ONNX does not need to perform intermediate training data.
  • ONNX Open Neural Network Exchange
  • the types of visualization information are the calling sequence diagrams of various processes, and the time-consuming situation of each operator can be determined through the calling sequence diagram.
  • the time consumption is greater than the preset time, the operators whose time consumption is greater than the preset time are optimized.
  • the category of the visualization information is a two-dimensional heat map. Through the heat map, it can be determined whether the features extracted by the model are correct. Build the model.
  • the feature map and the heat map are different representations of the target training data for the model to process the intermediate results.
  • the types of target training data include at least one of the following: scalar, vector, picture, video, and audio.
  • Step 203 The cloud server obtains target training data corresponding to the visualization request.
  • the acquiring target training data corresponding to the visualization request includes:
  • the target training data stored in the cloud storage is acquired based on the storage location information of the target training data.
  • a list of intermediate training data is obtained based on the training task identification and index information, and the intermediate training data is obtained in cloud storage based on the list of intermediate training data.
  • the visualization request carries the identification information of the target training data of the target deep learning model, and the identification information is used by the cloud server to obtain the target training data from the cloud storage; wherein, the identification information is preset, and the identification Information is unique in the cloud storage.
  • the training tasks of the target deep learning model include training task A and training task B
  • the training data generated by training task A includes training data A1 and training data A2
  • the training data generated by training task B includes training data B1 and training data B2
  • training data B1 and training data B2 are all stored in the cloud platform
  • the identification information of training data A1 is 1, the identification information of training data A2 is 2
  • the identification information of training data B1 is 3
  • the identification information of the training data B2 is 4, if the identification information is 1, the target training data obtained by the cloud server from the cloud storage is the training data A1; if the identification information is 2, the target training data obtained by the cloud server from the cloud storage
  • the data is training data A2; if the identification information is 3, the target training data obtained by the cloud server from the cloud storage is training data B1; if the identification information is 4, the target training data obtained by the cloud server from the cloud storage is training data B2.
  • Step 204 The cloud server preprocesses the target training data to obtain visualization information.
  • Step 205 The cloud server sends a visualization response to the user equipment, where the visualization response carries the visualization information.
  • Step 206 The user equipment receives a visualization response sent by the cloud server, where the visualization response carries the visualization information of the target training data.
  • Step 207 The user equipment displays the visualized information.
  • the visualization information includes at least one of the following categories: directed acyclic graphs, three-dimensional histograms, call sequence diagrams of various processes, feature maps, two-dimensional heat maps, and scalar line graphs.
  • the visualization information includes the following information: topology information of at least a part of the target deep learning model, wherein at least a part of the target deep learning model includes: multiple modules and/or multiple operators of the target deep learning model; resource occupation information of each operator in at least one operator included in the target deep learning model.
  • the topology information of multiple modules includes at least one of the following: identification information of the multiple modules, dependencies between the multiple modules, data size of each module in the multiple modules, Information of at least one operator included in each of the modules.
  • the identification information of the module is unique and can be preset.
  • the information of the operators includes at least one of the following: identification information of the operators, dependencies between operators, and data size of the operators.
  • the dependency relationship can be a sequential relationship or a parallel relationship.
  • the resource occupation information is determined by the cloud server based on at least one of the data type of the operator, the input data information of the operator, and the output data information of the operator.
  • the input of the operator can be a picture, video, audio, scalar, vector, etc., and different inputs correspond to different dimensions.
  • a picture is equivalent to a two-dimensional matrix
  • an audio is equivalent to a one-dimensional matrix.
  • the data type of the operator can be a double-precision type or a single-precision type.
  • the operator can be at least one of the following: convolution, batch normalization (BatchNorm, BN), full link, pooling, matrix multiplication and division, dropout (DropOut), activation, etc.
  • the input data information may be the size of the input data
  • the output data information may be the size of the output data
  • the visualization information includes the performance change trend information of the target deep model, the loss trend information of the target deep model, the training progress information of the target deep learning model, and the difference in the target deep learning model. At least one of the training comparison information.
  • the visualization information includes features extracted by the target deep learning model.
  • the visualization information includes the parameter distribution of each operator in the plurality of operators.
  • the visualization information includes the running time of each operator in at least one process of the multiple operators.
  • the process may be at least one of an interpreted language process, a local process, and an AI chip process.
  • the interpreted language process, the local process, and the AI chip process alternately execute the calling sequence diagram according to time.
  • the user equipment first sends a visualization request to the cloud server, then receives the visualization response sent by the cloud server, the visualization response carries the visualization information of the target training data, and finally displays the visualization information.
  • the visual information of the data can intuitively understand the training status of the deep learning model, which helps to improve the feasibility of timely judgment of the current training strategy, and provides a basis for decision-making such as early stopping.
  • the displaying the visualized information includes:
  • a module in the target deep learning model is used as the minimum display unit, wherein the module includes at least one operator; and/or
  • the operator in the target deep learning model is used as the minimum display unit.
  • the visualization information when the category of the visualization information is a directed acyclic graph, the visualization information can be displayed in the collapsed display mode, and the visualization information can also be displayed in the expanded display mode. , which displays visual information by expanding the display mode.
  • the expanded display mode may be an operator in the expanded display module after the module receives an operation instruction.
  • the expanded display mode may be to directly display the parameter distribution of each operator in each module.
  • the expanded display mode may be to directly display the running time of each operator in each module in at least one process.
  • the target deep learning model includes 2 modules (module A and module B), and module A includes 3 operators (A1, A2, and A3), the model B includes 2 operators (B1 and B2). If the visualized information is displayed in the folded display mode, module A and module B are displayed; if the visualized information is displayed in the expanded display mode, after the module A receives the operation instruction, the operators A1, A2 and A3 in the module A are displayed, and the module B receives the operation After the instruction, the operators B1 and B2 in module B are displayed.
  • the target deep learning model includes 2 modules (module A and module B), module A includes 3 operators (A1, A2, and A3), and model B includes Including two operators (B1 and B2), module A corresponds to 3D histogram 1, and module B corresponds to 3D histogram B, then 3D histogram 1 includes the parameter distribution of A1, the parameter distribution of A2 and the parameter distribution of A3
  • 3D histogram 1 includes the parameter distribution of A1, the parameter distribution of A2 and the parameter distribution of A3
  • the three-dimensional histogram 2 includes the parameter distribution of B1 and the parameter distribution of B2.
  • the target deep learning model includes 2 modules (module A and module B) and 2 processes (process C1 and C2), and module A includes 2 operators ( A1 and A2), model B includes 2 operators (B1 and B2), module A corresponds to call sequence diagram 1, and module B corresponds to call sequence diagram B, then the call sequence diagram 1 includes the running time of A1 in process C1 , the running duration of A1 in process C2, including the running duration of A2 in process C1 and the running duration of A2 in process C2; the calling sequence diagram 2 includes the running duration of B1 in process C1 and the running duration of B1 in process C2 The running duration includes the running duration of B2 in process C1 and the running duration of B2 in process C2.
  • a display mode of visualized information may be preset, which may be to display the visualized information in a folded display mode first, and then display the visualized information in an expanded display mode after a first duration, which may be preset.
  • the type of visualization information is a scalar line graph
  • the expandable display mode may be to directly display the features extracted by the target deep learning model.
  • the user equipment displays visual information in different ways, which is beneficial to improve the application scope of the user equipment.
  • the method before the sending the visualization request to the cloud server, the method further includes:
  • the intermediate training data of the target deep learning model is sent to the cloud server through a data upload thread, and the intermediate training data is used by the cloud server to obtain the index information and training task identifier of the target training data, and based on the
  • the training task identifier stores the index information in a database, and stores the target training data in cloud storage based on the index information.
  • the data uploading thread is a newly opened thread.
  • the training task identification is unique, and the training task identification may be determined after the intermediate training data is generated, or may be determined at the beginning of training.
  • the intermediate training data includes target training data, training task identifiers, and index information.
  • the intermediate training data of the target deep learning model is sent to the cloud server through a data upload thread, and the intermediate data is used for the cloud server to obtain the identification information of the target training data, and to store the identification information in a database.
  • the target training data is stored in cloud storage based on the identification information.
  • the identification information is unique.
  • the method before sending the intermediate training data of the target deep learning model to the cloud server through a data upload thread, the method further includes:
  • the training task identifier and the index information of the target training data determine the training task identifier and the index information of the target training data, and carry the training task information and the index information of the target training data in the intermediate training data;
  • the identification information of the target training data is determined, and the identification information is carried on the intermediate training data.
  • the identification information may be stored in the first buffer area of the database, and the target training data may be stored in the second buffer area of the database based on the identification information.
  • the target data is stored in the cloud storage by the cloud server, which is beneficial for the cloud server to quickly obtain the target training data after receiving the visualization request.
  • the preprocessing of the target training data to obtain visualization information includes:
  • the preprocessing of the picture or video includes one or more of the following processes: insensitive area removal processing, image precision enhancement processing, image noise reduction processing and image processing Binarization processing.
  • the text preprocessing includes one or more of the following processes: document segmentation, text segmentation, and removal of stop words (including punctuation, numbers, monads, and other Meaningless words) processing, text feature extraction, word frequency statistics processing and text vectorization processing.
  • stop words including punctuation, numbers, monads, and other Meaningless words
  • the visual rendering is to assemble the preprocessed data into a hypertext markup language (Hypertext Markup Language, HTML).
  • HTML Hypertext Markup Language
  • the type of visualization information obtained by preprocessing is non-directed acyclic graph.
  • the target training data is preprocessed to obtain preprocessed data, and then the preprocessed data is rendered to obtain visual information, which is beneficial to the target deep learning model and analysis based on the visual information.
  • the preprocessing of the target training data to obtain visualization information includes:
  • the target training data is parsed through a binary tree to obtain the visualization information.
  • the type of the target deep learning model that processes the target training data through the binary tree is parrots.
  • the type of visualization information obtained by parsing the target training data through a binary tree is a directed acyclic graph.
  • the visualization information is obtained by analyzing the target training data, which is beneficial to analyze and analyze the target deep learning model based on the visualization information.
  • FIG. 3 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the computer device includes a processor, a memory, a communication interface, and one or more programs, wherein one or more of the above A plurality of programs are stored in the above-mentioned memory, and are configured to be executed by the above-mentioned processor.
  • the computer device is user equipment
  • the above program includes instructions for performing the following steps:
  • the visualization information is displayed.
  • the visualization information includes at least one of the following categories: directed acyclic graph, three-dimensional histogram, call sequence diagram of various processes, feature map, two-dimensional heat map, and scalar line graph; and/or
  • the target training data includes at least one of the following: model performance change trend information, model loss trend information, model parameter distribution information, model processing intermediate results, model structure information, current progress information of model training, and information on different trainings for the same model. Comparison information, scheduling time information of various processes.
  • the visualization information includes at least one of the following information:
  • topology information of at least a part of the target deep learning model wherein at least a part of the target deep learning model includes: multiple modules and/or multiple operators of the target deep learning model;
  • the topology information of the multiple modules includes at least one of the following:
  • the identification information of the multiple modules the dependencies between the multiple modules, the data size of each module in the multiple modules, and the information of at least one operator included in each of the modules.
  • the resource occupation information is determined by the cloud server based on at least one of the data type of the operator, the input data information of the operator, and the output data information of the operator.
  • the visualization request carries a training task identifier of the target deep learning model and index information of the target training data, where the index information includes: data type and data label.
  • the above-mentioned program includes instructions for executing the following steps:
  • the display of visual data in different modes is supported, wherein, in some embodiments, the visual information may be displayed in a folding display mode or a presentation display mode, or other display modes may also be defined. Embodiments are not limited thereto.
  • a module in the target deep learning model is used as the smallest display unit, wherein the module includes at least one operator.
  • the modules here may be divided in a default division manner, or may also be set by a user, which is not limited in this embodiment of the present disclosure.
  • the operator in the target deep learning model is used as the minimum display unit. At this point, the information of all operators in the model can be displayed.
  • it can also be displayed in a mixed display mode, that is, some modules are folded and displayed, and other modules are displayed in an expanded manner, which can optionally be displayed based on user settings, which is not covered by the embodiments of the present disclosure. Do limit.
  • the above program before sending the visualization request to the cloud server, the above program includes an instruction for performing the following steps:
  • the intermediate training data of the target deep learning model is sent to the cloud server through a data upload thread, and the intermediate training data is used by the cloud server to obtain the index information and training task identifier of the target training data, and based on the
  • the training task identifier stores the index information in a database, and stores the target training data in cloud storage based on the index information.
  • the computer device is a cloud server
  • the above program includes instructions for performing the following steps:
  • a visualization response is sent to the user equipment, and the visualization response carries the visualization information.
  • the visualization information includes at least one of the following categories: directed acyclic graph, three-dimensional histogram, call sequence diagram of various processes, feature map, two-dimensional heat map, and scalar line graph; and/or
  • the target training data includes at least one of the following: model performance change trend information, model loss trend information, model parameter distribution information, model processing intermediate results, model structure information, current progress information of model training, and information on different trainings for the same model. Comparison information, scheduling time information of various processes.
  • the visualization information includes at least one of the following information:
  • topology information of at least a part of the target deep learning model wherein at least a part of the target deep learning model includes: multiple modules and/or multiple operators of the target deep learning model;
  • the topology information of the multiple modules includes at least one of the following:
  • the identification information of the multiple modules the dependencies between the multiple modules, the data size of each module in the multiple modules, and the information of at least one operator included in each of the modules.
  • the above program includes instructions for executing the following steps:
  • the visualization request carries a training task identifier of the target deep learning model and index information of the target training data, where the index information includes: data type and data label;
  • the above-mentioned program includes an instruction for executing the following steps: searching an index database based on the training task identifier and index information carried in the visualization request, so as to obtain the target training data. storage location information; based on the storage location information of the target training data, obtain the target training data stored in the cloud storage.
  • the above program includes instructions for executing the following steps:
  • the above program includes instructions for executing the following steps:
  • the target training data is parsed through a binary tree to obtain the visualization information.
  • FIG. 4 is a visualization apparatus provided by an embodiment of the present application, applied to user equipment, and the apparatus includes:
  • a sending unit 401 configured to send a visualization request to a cloud server, where the visualization request is used to request visualization of target training data of a target deep learning model;
  • a receiving unit 402 configured to receive a visualization response sent by the cloud server, where the visualization response carries the visualization information of the target training data;
  • the display unit 403 is configured to display the visualized information.
  • the visualization information includes at least one of the following categories: directed acyclic graph, three-dimensional histogram, call sequence diagram of various processes, feature map, two-dimensional heat map, and scalar polyline Figures; and/or
  • the target training data includes at least one of the following: model performance change trend information, model loss trend information, model parameter distribution information, model processing intermediate results, model structure information, current progress information of model training, and information on different trainings for the same model. Comparison information, scheduling time information of various processes.
  • the visualized information includes at least one of the following information:
  • topology information of at least a part of the target deep learning model wherein at least a part of the target deep learning model includes: multiple modules and/or multiple operators of the target deep learning model;
  • the topology information of the multiple modules includes at least one of the following:
  • the identification information of the multiple modules the dependencies between the multiple modules, the data size of each module in the multiple modules, and the information of at least one operator included in each of the modules.
  • the resource occupation information is at least one of the cloud server based on the data type of the operator, the input data information of the operator, and the output data information of the operator definite.
  • the visualization request carries a training task identifier of the target deep learning model and index information of the target training data, where the index information includes: data type and data label.
  • the above-mentioned display unit 403 is specifically used for:
  • a module in the target deep learning model is used as the minimum display unit, wherein the module includes at least one operator; and/or
  • the operator in the target deep learning model is used as the minimum display unit.
  • the above-mentioned sending unit 401 is specifically further configured to send the intermediate training data of the target deep learning model to the cloud server through a data upload thread before sending the visualization request to the cloud server, and the The intermediate training data is used for the cloud server to obtain the index information and training task identifier of the target training data;
  • the apparatus further includes: a data storage unit 404, configured to store the index information in a database based on the training task identifier, and store the target training data in cloud storage based on the index information.
  • a data storage unit 404 configured to store the index information in a database based on the training task identifier, and store the target training data in cloud storage based on the index information.
  • the sending unit 401 and the receiving unit 402 can be implemented through a communication interface
  • the display unit 403 can be implemented through a display screen
  • the data storage unit 404 can be implemented through a processor screen.
  • FIG. 5 is a visualization device provided by an embodiment of the present application, applied to a cloud server, and the device includes:
  • a receiving unit 501 configured to receive a visualization request from a user equipment, where the visualization request is used to request visualization of target training data of a target deep learning model;
  • an obtaining unit 502 configured to obtain target training data corresponding to the visualization request
  • Determining unit 503, configured to preprocess the target training data to obtain visualization information
  • the sending unit 504 is configured to send a visualization response to the user equipment, where the visualization response carries the visualization information.
  • the visualization information includes at least one of the following categories: directed acyclic graph, three-dimensional histogram, call sequence diagram of various processes, feature map, two-dimensional heat map, and scalar polyline Figures; and/or
  • the target training data includes at least one of the following: model performance change trend information, model loss trend information, model parameter distribution information, model processing intermediate results, model structure information, current progress information of model training, and information on different trainings for the same model. Comparison information, scheduling time information of various processes.
  • the visualized information includes at least one of the following information:
  • topology information of at least a part of the target deep learning model wherein at least a part of the target deep learning model includes: multiple modules and/or multiple operators of the target deep learning model;
  • the topology information of the multiple modules includes at least one of the following:
  • the identification information of the multiple modules the dependencies between the multiple modules, the data size of each module in the multiple modules, and the information of at least one operator included in each of the modules.
  • the above program includes instructions for executing the following steps:
  • the visualization request carries a training task identifier of the target deep learning model and index information of the target training data, where the index information includes: data type and data label;
  • the above acquiring unit 502 is specifically used for:
  • the target training data stored in the cloud storage is acquired based on the storage location information of the target training data.
  • the above determining unit 503 is specifically configured to:
  • the above determining unit 503 is specifically configured to:
  • the target training data is parsed through a binary tree to obtain the visualization information.
  • the receiving unit 501 and the sending unit 504 may be implemented by a communication interface
  • the acquiring unit 502 and the determining unit 503 may be implemented by a processor.
  • Embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the electronic Some or all of the steps described by the device or cloud server.
  • Embodiments of the present application further provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause the computer to execute the electronic method as described above. Some or all of the steps described by the device or cloud server.
  • the computer program product may be a software installation package.
  • the steps of the method or algorithm described in the embodiments of the present application may be implemented in a hardware manner, or may be implemented in a manner in which a processor executes software instructions.
  • Software instructions can be composed of corresponding software modules, and software modules can be stored in random access memory (Random Access Memory, RAM), flash memory, read only memory (Read Only Memory, ROM), erasable programmable read only memory ( Erasable Programmable ROM, EPROM), Electrically Erasable Programmable Read-Only Memory (Electrically EPROM, EEPROM), registers, hard disk, removable hard disk, CD-ROM, or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor, such that the processor can read information from, and write information to, the storage medium.
  • the storage medium can also be an integral part of the processor.
  • the processor and storage medium may reside in an ASIC.
  • the ASIC may reside in access network equipment, target network equipment or core network equipment.
  • the processor and the storage medium may also exist in the access network device, the target network device or the core network device as discrete components.
  • the functions described in the embodiments of the present application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server or data center Transmission to another website site, computer, server or data center via wired (eg coaxial cable, optical fiber, Digital Subscriber Line, DSL) or wireless (eg infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media.
  • the available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, Digital Video Disc (DVD)), or semiconductor media (eg, Solid State Disk (SSD)) )Wait.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electrically Operated Instructional Devices (AREA)

Abstract

Procédé de visualisation et dispositif associé. Le procédé comprend : l'envoi, par un équipement utilisateur, d'une demande de visualisation à un serveur en nuage, la demande de visualisation étant utilisée pour demander la visualisation de données de formation cibles d'un modèle d'apprentissage profond cible ; la réception, par l'équipement utilisateur, d'une réponse de visualisation envoyée par le serveur en nuage, la réponse de visualisation transportant des informations de visualisation des données de formation cibles ; et l'affichage, par l'équipement utilisateur, des informations de visualisation. Au moyen du présent procédé, des données de formation d'un modèle d'apprentissage profond peuvent être visualisées, ce qui facilite la détermination opportune de la faisabilité de la stratégie de formation actuelle, et apporte un fondement pour des décisions telles que l'arrêt précoce.
PCT/CN2021/082348 2020-07-09 2021-03-23 Procédé de visualisation et dispositif associé WO2022007434A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2021570729A JP2022543180A (ja) 2020-07-09 2021-03-23 可視化方法及び関連デバイス
KR1020217039065A KR20220011134A (ko) 2020-07-09 2021-03-23 가시화 방법 및 관련 기기

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010656553.8 2020-07-09
CN202010656553.8A CN111797294A (zh) 2020-07-09 2020-07-09 可视化方法及相关设备

Publications (1)

Publication Number Publication Date
WO2022007434A1 true WO2022007434A1 (fr) 2022-01-13

Family

ID=72810505

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/082348 WO2022007434A1 (fr) 2020-07-09 2021-03-23 Procédé de visualisation et dispositif associé

Country Status (4)

Country Link
JP (1) JP2022543180A (fr)
KR (1) KR20220011134A (fr)
CN (1) CN111797294A (fr)
WO (1) WO2022007434A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114707654A (zh) * 2022-06-06 2022-07-05 浙江大学 基于人工智能框架的算法训练推理性能可视化方法及装置

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111797294A (zh) * 2020-07-09 2020-10-20 上海商汤智能科技有限公司 可视化方法及相关设备
CN112199345A (zh) * 2020-10-28 2021-01-08 上海大学 深度学习训练实时可视化方法及装置
CN112733892A (zh) * 2020-12-28 2021-04-30 北京聚云科技有限公司 一种用于模型训练的数据交互方法及装置
CN113326113B (zh) * 2021-05-25 2022-11-22 北京市商汤科技开发有限公司 任务处理方法及装置、电子设备和存储介质
CN113658289A (zh) * 2021-07-30 2021-11-16 国网天津市电力公司 一种二次设备编辑方法
CN114357253A (zh) * 2021-12-27 2022-04-15 上海商汤科技开发有限公司 数据可视化方法、装置、电子设备和存储介质
CN116360759A (zh) * 2023-03-10 2023-06-30 青软创新科技集团股份有限公司 一种人工智能算法的可视化系统和方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012012194A2 (fr) * 2010-07-21 2012-01-26 Microsoft Corporation Valeurs par défaut intelligentes pour des visualisations de données
CN104615735A (zh) * 2015-02-11 2015-05-13 中科九度(北京)空间信息技术有限责任公司 一种基于地理信息空间系统的时空信息可视化方法
CN109241141A (zh) * 2018-09-04 2019-01-18 北京百度网讯科技有限公司 深度学习的训练数据处理方法和装置
CN111797294A (zh) * 2020-07-09 2020-10-20 上海商汤智能科技有限公司 可视化方法及相关设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101719278B1 (ko) * 2015-04-14 2017-04-04 (주)한국플랫폼서비스기술 비주얼 콘텐츠기반 영상 인식을 위한 딥러닝 프레임워크 및 영상 인식 방법
CN111079892A (zh) * 2019-10-30 2020-04-28 华为技术有限公司 深度学习模型的训练方法、装置及系统
CN111190805A (zh) * 2019-12-29 2020-05-22 广东浪潮大数据研究有限公司 深度学习训练平台的训练日志可视化系统、方法及设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012012194A2 (fr) * 2010-07-21 2012-01-26 Microsoft Corporation Valeurs par défaut intelligentes pour des visualisations de données
CN104615735A (zh) * 2015-02-11 2015-05-13 中科九度(北京)空间信息技术有限责任公司 一种基于地理信息空间系统的时空信息可视化方法
CN109241141A (zh) * 2018-09-04 2019-01-18 北京百度网讯科技有限公司 深度学习的训练数据处理方法和装置
CN111797294A (zh) * 2020-07-09 2020-10-20 上海商汤智能科技有限公司 可视化方法及相关设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114707654A (zh) * 2022-06-06 2022-07-05 浙江大学 基于人工智能框架的算法训练推理性能可视化方法及装置
CN114707654B (zh) * 2022-06-06 2022-08-23 浙江大学 基于人工智能框架的算法训练推理性能可视化方法及装置

Also Published As

Publication number Publication date
JP2022543180A (ja) 2022-10-11
CN111797294A (zh) 2020-10-20
KR20220011134A (ko) 2022-01-27

Similar Documents

Publication Publication Date Title
WO2022007434A1 (fr) Procédé de visualisation et dispositif associé
US20210256320A1 (en) Machine learning artificialintelligence system for identifying vehicles
CN111931064B (zh) 基于大数据和人工智能的信息解析方法及云服务信息平台
CN107766940B (zh) 用于生成模型的方法和装置
US11132114B2 (en) Method and apparatus for generating customized visualization component
CN109684047A (zh) 事件处理方法、装置、设备和计算机存储介质
CN109862100B (zh) 用于推送信息的方法和装置
WO2017181866A1 (fr) Réalisation de requêtes de motifs de graphe limités dans des grands graphes
CN108885633A (zh) 用于自动发现和连接到rest接口的技术
CN113807926A (zh) 推荐信息生成方法、装置、电子设备和计算机可读介质
US11676345B1 (en) Automated adaptive workflows in an extended reality environment
CN112883088B (zh) 一种数据处理方法、装置、设备及存储介质
CN117093619A (zh) 一种规则引擎处理方法、装置、电子设备及存储介质
CN116956117A (zh) 一种标签识别的方法、装置、设备、存储介质及程序产品
CN111814044A (zh) 一种推荐方法、装置、终端设备及存储介质
CN113808134B (zh) 油罐布局信息生成方法、装置、电子设备、介质
WO2021232860A1 (fr) Procédé, appareil et système de communication
WO2015078124A1 (fr) Procédé et dispositif de traitement de données réseau
US10817313B2 (en) Machine learning techniques for adaptation of initial workspace
CN112905858A (zh) 节点关系图谱显示方法及装置、计算机设备和存储介质
CN112379967A (zh) 模拟器检测方法、装置、设备及介质
CN112394809A (zh) 一种实现增强现实的方法、装置及通信设备
US20240127406A1 (en) Image quality adjustment method and apparatus, device, and medium
CN112783986B (zh) 基于标签的对象分组编制方法及装置、存储介质、终端
CN117687550A (zh) 卡牌交互方法、卡牌组合分类方法、装置、介质及设备

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021570729

Country of ref document: JP

Kind code of ref document: A

Ref document number: 20217039065

Country of ref document: KR

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21837004

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21837004

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 03.07.20023)

122 Ep: pct application non-entry in european phase

Ref document number: 21837004

Country of ref document: EP

Kind code of ref document: A1