CN113836291A - Data processing method, device, equipment and storage medium - Google Patents
Data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113836291A CN113836291A CN202111154874.9A CN202111154874A CN113836291A CN 113836291 A CN113836291 A CN 113836291A CN 202111154874 A CN202111154874 A CN 202111154874A CN 113836291 A CN113836291 A CN 113836291A
- Authority
- CN
- China
- Prior art keywords
- data
- offline
- online
- recommendation model
- difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/335—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3346—Query execution using probabilistic model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/34—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Debugging And Monitoring (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The present disclosure provides a data processing method, apparatus, device and storage medium, which relate to the technical field of computers, and in particular to the artificial intelligence fields of intelligent recommendation, natural language processing, deep learning, and the like. The data processing method comprises the following steps: responding to the difference between an online recommendation model and an offline recommendation model, and acquiring data to be compared based on the online recommendation model and the offline recommendation model; acquiring difference data of the difference based on the data to be compared; performing processing operations based on the difference data, the processing operations comprising: displaying a visual image-text corresponding to the difference data, and/or positioning the reason of the difference based on the difference data. The method and the device can realize automatic processing of the difference between the offline recommendation model and the online recommendation model.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to the field of artificial intelligence, such as intelligent recommendation, natural language processing, and deep learning, and in particular, to a data processing method, apparatus, device, and storage medium.
Background
With the development of intelligence, a suitable recommended resource, such as an article, can be recommended for a user based on a recommendation model. The recommendation models may include an offline recommendation model and an online recommendation model for the same batch of samples. An offline prediction value can be obtained based on the offline recommendation model, and an online prediction value can be obtained based on the online recommendation model.
In the related art, when the off-line estimated value is inconsistent with the on-line estimated value, the difference between the off-line recommendation model and the on-line recommendation model is checked in a manual mode.
Disclosure of Invention
The disclosure provides a data processing method, apparatus, device and storage medium.
According to an aspect of the present disclosure, there is provided a data processing method including: responding to the difference between an online recommendation model and an offline recommendation model, and acquiring data to be compared based on the online recommendation model and the offline recommendation model; acquiring difference data of the difference based on the data to be compared; performing processing operations based on the difference data, the processing operations comprising: displaying a visual image-text corresponding to the difference data, and/or positioning the reason of the difference based on the difference data.
According to another aspect of the present disclosure, there is provided a data processing apparatus including: the first obtaining module is used for responding to the difference between an online recommendation model and an offline recommendation model and obtaining data to be compared based on the online recommendation model and the offline recommendation model; a second obtaining module, configured to obtain difference data of the difference based on the data to be compared; a processing module to perform processing operations based on the difference data, the processing operations comprising: displaying a visual image-text corresponding to the difference data, and/or positioning the reason of the difference based on the difference data.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the above aspects.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method according to any one of the above aspects.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of the above aspects.
According to the technical scheme of the disclosure, the automatic processing of the difference between the offline recommendation model and the online recommendation model can be realized.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure;
FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure;
FIG. 3 is a schematic diagram according to a third embodiment of the present disclosure;
FIG. 4 is a schematic diagram according to a fourth embodiment of the present disclosure;
FIG. 5 is a schematic diagram according to a fifth embodiment of the present disclosure;
FIG. 6 is a schematic diagram according to a sixth embodiment of the present disclosure;
FIG. 7 is a schematic diagram according to a seventh embodiment of the present disclosure;
fig. 8 is a schematic diagram of an electronic device for implementing any one of the data processing methods of the embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the related art, when the off-line estimated value is inconsistent with the on-line estimated value, the difference between the off-line recommendation model and the on-line recommendation model is manually checked, and the automatic processing of the difference between the off-line recommendation model and the on-line recommendation model cannot be realized.
In order to achieve automated processing of the above differences, the present disclosure provides the following embodiments.
Fig. 1 is a schematic diagram according to a first embodiment of the present disclosure, which provides a data processing method, including:
101. and responding to the difference between the online recommendation model and the offline recommendation model, and acquiring the data to be compared based on the online recommendation model and the offline recommendation model.
102. And acquiring difference data of the difference based on the data to be compared.
103. Performing processing operations based on the difference data, the processing operations comprising: displaying a visual image-text corresponding to the difference data, and/or positioning the reason of the difference based on the difference data.
An application environment of the data processing method provided by this embodiment may be as shown in fig. 2, where the application environment may include: an online system 201, an offline system 202, and a visualization system 203.
The execution subject of this embodiment may be referred to as a data processing device, and the device may be located in the visualization system 203, and the visualization system may be software, hardware, or a combination of software and hardware. The visualization system can be located in an electronic device, the electronic device can be a server or a user terminal, the server can be a cloud server or a local server, and the user terminal can include a mobile device (such as a mobile phone or a tablet computer), a wearable device (such as a smart watch or a smart bracelet), a smart home device (such as a smart television or a smart sound box), and the like.
The model is generally divided into an offline stage and an online stage, where the offline stage refers to training the model offline to generate the model to be used online. The online phase is to input online input (such as recommended resources) into an online model to generate corresponding model output (such as recommendation probability corresponding to the recommended resources), where the recommended resources are articles to be recommended to the user, short videos, and the like.
The input of the recommendation model is recommendation resources, and the output is recommendation probability of the recommendation resources, and the recommendation model can be a Deep Neural Networks (DNN) model.
The model of the offline phase may be referred to as an offline recommendation model and the model of the online phase may be referred to as an online recommendation model.
The system corresponding to the online phase may be referred to as an online system 201 with an online recommendation model therein, and the system corresponding to the offline phase may be referred to as an offline system 202 with an offline recommendation model therein.
To improve the accuracy of the recommended resources, testing may be performed prior to formal application.
During testing, the same batch of estimation samples can be adopted to obtain an offline estimation value and an online estimation value through an offline recommendation model and an online recommendation model respectively.
The prediction samples may include recommended resource samples, and the prediction value is a recommended probability value of the recommended resource samples.
If the off-line estimate is inconsistent with the on-line estimate, a source of discrepancy may be located.
In the related art, the source of the positioning difference is generally manually determined, however, the manual method is less intuitive and cannot be automatically attributed, i.e., an automated processing scheme for the difference is lacking.
In this embodiment, the estimated samples may be respectively input into the online recommendation model and the offline recommendation model to respectively obtain an online predicted value and an offline predicted value, and if the online predicted value and the offline predicted value are different, or a difference value between the online predicted value and the offline predicted value is outside a preset range threshold, it is determined that there is a difference between the online recommendation model and the offline recommendation model.
The data to be compared may include data describing the network topology and/or the forward output data of the nodes in the model.
Difference data can be obtained based on the data to be compared, and the difference data can be used for describing the network topology difference between the online recommendation model and the offline recommendation model and/or the forward output data difference of the nodes.
Visualization (Visualization) is the use of computer graphics and image processing techniques to convert data into graphics or text for display on a screen.
Visualization means that the visualized content can be graphics and/or text.
For example, referring to fig. 3, dots represent nodes in the model, black dots correspond to online recommendation models, and white dots correspond to offline recommendation models. The numerical value of the dot edge is the forward output data of the corresponding node. The difference between the two can comprise network difference and data difference, and the difference can be displayed and can be attributed.
In this embodiment, by acquiring the data to be compared, acquiring the difference data based on the data to be compared, and executing the processing operation based on the difference data, the automatic processing of the difference between the offline recommendation model and the online recommendation model can be realized.
In some embodiments, the data to be aligned comprises: the obtaining of the data to be compared based on the online recommendation model and the offline recommendation model comprises the following steps:
acquiring online node data from an online log of an online system of the online recommendation model, wherein the online node data is acquired by the online system and recorded in the online log, the online node data is forward output data of a node in the online recommendation model, and the forward output data is acquired when a sample to be adopted is used as input of the online recommendation model and is subjected to forward output processing by the online recommendation model;
the offline node data are obtained from offline logs of an offline system of the offline recommendation model, the offline node data are obtained by the offline system and recorded in the offline logs, the offline node data are forward output data of nodes in the offline recommendation model, and the forward output data are obtained when the offline system takes a sample to be adopted as input of the offline recommendation model and the offline recommendation model is adopted to perform forward output processing on the sample to be adopted.
In some embodiments, the data to be aligned comprises: the obtaining of the data to be compared based on the online recommendation model and the offline recommendation model comprises the following steps: acquiring a prestored online network topology file from an online system of the online recommendation model, and carrying out network analysis on the online network topology file to obtain online network description data; and acquiring a prestored offline network topology file from an offline system of the offline recommendation model, and carrying out network analysis on the offline network topology file to acquire the offline network description data.
As shown in fig. 4, an online system (not shown in fig. 4) may store an online log, online node data may be recorded in the online log, an offline system (not shown in fig. 4) may store an offline log, and offline node data may be recorded in the offline log.
As shown in fig. 4, an online system (not shown in fig. 4) may store an online network topology file, online network description data may be recorded in the online network topology file, an offline system (not shown in fig. 4) may store an offline network topology file, and offline network description data may be recorded in the offline network topology file.
The online network description data is used for describing the network topology relationship of the online recommendation model, and the offline network description data is used for describing the network topology relationship of the offline recommendation model.
Taking the online network description data as an example, the online network description data may include node information in the online recommendation model, where the node information includes, for example: node name, the number of neurons included in the node, the input relationship and the output relationship of the node, and the like. The offline network description data is similar.
The visualization system may obtain online logs from an online system and offline logs from an offline system.
After the online log and the offline log are acquired by the visualization system, a data analysis service may be started, and the online log and the offline log are subjected to log analysis by the data analysis service to acquire online node data and offline node data, where the node data in fig. 4 includes the online node data and the offline node data.
After the online network topology file and the offline network topology file are obtained by the visualization system, a network parsing service may be started, and the network parsing service performs network parsing on the online network topology file and the offline network topology file to obtain online network description data and offline network description data, where the network description data in fig. 4 includes online network description data and offline network description data.
When the network is analyzed, the input and output relations of each node of the network can be analyzed, so that a directed network topological graph is formed, the topological relations are put in a storage, and the affiliated relations of the nodes are labeled, for example, the node belongs to both online and offline, is labeled as common, is only online, is labeled as on, is only offline, and is labeled as off.
Further, as shown in fig. 4, the network description data and node data may be stored in a database, such as Mysql, for subsequent data.
The above-mentioned step before the data is stored in the database is described as an example of the visualization system, or the visualization system may perform the above-mentioned processing by another system, and the data is stored in the database, and then the visualization system may obtain the data from the database to perform the subsequent processing such as calculation, display, attribution of difference, and the like.
By parsing based on the log, online node data and offline node data may be obtained.
By network-based parsing, online network description data and offline network description data may be obtained.
The online network topology file and the offline network topology file may be pre-stored in the corresponding systems.
The online node data in the online log may be obtained by the online system using an online recommendation model, and the offline node data in the offline log may be obtained by the offline system using an offline recommendation model.
The recommendation model is generally a deep neural network model, and the model may include multiple layers, each layer may include multiple nodes, and each node may include one or more neurons.
Further, the multi-layer recommendation model can be divided into an input layer, a hidden layer and an output layer, and the processing sequence of pre-estimating a sample from the input layer, passing through the input layer to the hidden layer, and passing through the hidden layer to the output layer can be called forward output processing. During the forward output processing, the forward output data of each node can be used as the node data of the corresponding model.
The node data of the recommended model may be a value between [0,1], e.g., 0.2, 0.6, etc.
The description of the recommendation model applies to both the online recommendation model and the offline recommendation model, if not specifically stated. Similarly, the description of node data applies to both online node data and offline node data. The description of the network description data applies to online network description data, offline network description data, and the like.
In some embodiments, the method may further comprise: obtaining a pre-estimated value difference value of each pre-estimated sample in a plurality of pre-estimated samples, wherein the pre-estimated value difference value is a difference value between an online pre-estimated value and an offline pre-estimated value, the online pre-estimated value is obtained after the online recommendation model performs pre-estimation processing on each pre-estimated sample, and the offline pre-estimated value is obtained after the offline recommendation model performs pre-estimation processing on each pre-estimated sample; and taking the estimated sample with the maximum estimated value difference as the sample to be adopted.
The same batch of estimated samples can be used as the input of an online recommendation model and an offline recommendation model respectively, the online recommendation model performs estimation processing on the batch of estimated samples to obtain online estimated values of all the estimated samples, and similarly, the offline recommendation model outputs the offline estimated values of all the estimated samples. The pre-estimated sample may be a recommended resource, such as an article, and the pre-estimated value (online or offline) is a recommendation probability corresponding to the article.
The online estimated value and the offline estimated value can be values between 0 and 1, and the absolute value of the difference between the online estimated value and the offline estimated value is used as the estimated value difference.
It is understood that if there are multiple estimated samples of the largest estimated difference value, one of the samples may be randomly selected as the sample to be taken.
It is understood that the selection of the predicted sample of the largest estimated difference value as the sample to be used is an example, and other manners may also be adopted, and the sample to be used is selected based on the estimated difference value, for example, the manner of selecting the second highest difference value, etc.
And selecting the more representative sample to analyze the difference between the online recommendation model and the offline recommendation model by taking the estimated sample with the largest estimated value difference as a sample to be adopted.
In some embodiments, the online recommendation model performs forward output processing of the online recommendation model after the first predictor switch is turned on, and the offline recommendation model performs forward output processing of the offline recommendation model after the second predictor switch is turned on, and the method further includes: generating a request based on the sample to be adopted; and starting the first pre-estimation switch of the online recommendation model and the second pre-estimation switch of the offline recommendation model based on the request.
As shown in fig. 5, it is assumed that the estimated value is represented by q, and q is the largest in difference between the online estimated value and the offline estimated value. Different estimated samples can be identified by nid, so that the estimated sample of nid with the largest q difference value can be obtained.
After obtaining the estimated sample of nid with the largest q difference value, the estimated sample can be carried in a request to be sent to an online system and an offline system, the request is used for triggering an estimated switch for starting an online recommendation model and an estimated switch for starting an offline recommendation model, the estimated switch for the online recommendation model can be called a first estimated switch for distinguishing, the estimated switch for the offline recommendation model can be called a second estimated switch for distinguishing, and therefore corresponding online node data and offline node data are obtained through the online recommendation model and the offline recommendation model and are respectively recorded in an online log and an offline log.
Specifically, for an online system, the dump may be turned on, and the system turning on the dump may print out a forward output of each node from an input layer to each layer of an output layer of a network in a model estimation process, and turn on a debug, where the debug may record a sample (sample or payload) after a request (request) is read in, an online prediction value (q), and the like in an online log (log).
For an offline system, format conversion can be performed on samples in an online log, that is, the contents of samples of an estimated sample input as an online recommendation model and an estimated sample input as an offline recommendation model are consistent, but the formats may not be consistent because of respective requirements, so that during offline processing, format conversion can be performed first, the samples in the online log are converted into an input sample format recognized by an offline training environment, offline training is started, dump is started, offline estimation is performed on the samples, and the output of each layer in the estimation process is recorded into a specified file to be used as an offline log.
After the online log and the offline log are obtained, subsequent processing can be performed. The subsequent processing includes, for example, obtaining online node data and offline node data based on the online log and the offline log, obtaining node difference data based on the online node data and the offline node data, and the like.
The specific process can comprise the following steps: requesting a platform interface, processing the back end of the platform, returning a task link, and inserting the task link into a pipeline report. The platform may refer to a visualization system, and the backend processing, such as log analysis, obtains node data, and the like. Wherein different processes can be implemented based on different tasks, and thus the task related processes described above can be performed.
By turning on the pre-estimation switch, the processing flows of the online recommendation model and the offline recommendation model can be triggered.
In some embodiments, the data to be aligned comprises: online network description data and offline network description data, and online node data and offline node data, the difference data comprising: the obtaining of the differential data based on the data to be compared includes: if the online network description data and the offline network description data corresponding to the same node identifier are different, taking the different online network description data and the different offline network description data as the topology difference data; and/or if the online node data and the offline node data corresponding to the same node identifier are different, taking the different online node data and the different offline node data as the node difference data.
As shown in fig. 4, the online network description data and the offline network description data, as well as the online node data and the offline node data, may be stored in Mysql, and the visualization system may obtain the data from the Mysql, and perform data processing on the data, where the data processing includes the above process of obtaining the difference data, and further, may generate a visualization graph based on the data.
Further, as shown in fig. 4, after data processing, a full amount of networks, a difference network, data details, difference data, difference attribution, and the like may be presented.
As shown in fig. 3, network differences and data differences are shown, and for network differences, it may be demonstrated that a node marked on, a node marked off, a node marked common but different nodes exist for input and output. For data differences, nodes with different node data may be exposed.
Specifically, referring to fig. 3, the model corresponding to the black filled node is an online recommendation model, the model corresponding to the white filled node is an offline recommendation model, and for the node C, there is a difference in the topological relationship, that is, the online network description data and the offline network data are different, so that there is topology difference data between the two. For node D, E, F, the online node data and the offline node data are different, and therefore, there is node difference data between the two.
It can be understood that for the purpose of visual differentiation, taking node C as an example, node C of the online recommendation model and node C of the offline recommendation model1Are not symbolically shown, but both correspond to the same node identifier in the data record for differential comparison.
By obtaining the topology difference data and the node difference data, the comparison of the difference of the network topology and the difference of the node data can be carried out, and the difference sources are enriched.
In some embodiments, the visualization comprises: the difference topological graph and/or the node difference data show the visual graph-text corresponding to the difference data, and the method comprises the following steps:
if the difference data comprises topology difference data, generating a difference topological graph corresponding to the difference based on the topology difference data, and displaying the difference topological graph; and/or if the difference data comprises node difference data, displaying the node difference data.
Visualization (Visualization) is the use of computer graphics and image processing techniques to convert data into graphics or text for display on a screen.
Visualization means that the visualized content can be graphics and/or text.
As shown in fig. 3, the difference may include a network difference and a data difference, and for the network difference, because the output relationship of the node C is different, the corresponding network topology may be generated and displayed based on the node C and the related nodes. In addition, the network difference may show node data of each node, such as node data of C is 0.2, in addition to showing the network topology (node and relationship between nodes). For data difference, the nodes with difference and the corresponding node data can be shown.
Further, difference attribution may also be performed based on the difference data. Differential attribution refers to locating the reasons for the differences between the online recommendation model and the offline recommendation model, for reasons such as: network topology reasons, forward output data reasons, etc.
By displaying the topological graph and/or the node data, the visualization is strong. In addition, automatic attribution can be achieved by performing difference attribution based on difference data, and attribution efficiency is high.
In some embodiments, at least one of the following is also included: generating an online panoramic topological graph based on the online network description data, and displaying the online panoramic topological graph; generating an offline panoramic topological graph based on the offline network description data, and displaying the offline panoramic topological graph; displaying data details of the nodes in the online recommendation model based on the online node data; and displaying the data details of the nodes in the offline recommendation model based on the offline node data.
Besides the difference display, the full display can be displayed, that is, the network topology corresponding to the online recommendation model, the network topology corresponding to the offline recommendation model, the data details of the nodes in the online recommendation model and the offline recommendation model, and the like are displayed.
In addition, the graph and the data may be combined during the presentation, for example, as shown in fig. 3, taking the online network model as an example, not only the online panoramic topology (upper left corner) of the online network model is presented, but also node data of each node (for example, node data of node a is 0.5) is presented.
In addition, for the data details, besides the node data (e.g. 0.5), the affiliation of the node, such as on, off, common, or the like, may be included.
Through multiple display modes, richer information can be displayed.
The above describes the relevant steps of the method separately, and examples combining various steps are given below. It is understood that all steps in the following embodiments are not necessarily selected, and may be reasonably selected and replaced according to actual requirements, and in addition, if the timing relationship of each step does not have a strong dependence relationship, and is not particularly described, there is no timing limitation relationship.
Fig. 6 is a schematic diagram according to a sixth embodiment of the present disclosure, which provides a data processing method, including:
601. the visualization system obtains the estimated value difference of each estimated sample in the plurality of estimated samples.
602. If the estimated value difference value larger than the preset threshold value exists, the visualization system determines that the online recommendation model and the offline recommendation model are different, and the estimated sample with the maximum estimated value difference value is used as the sample to be adopted.
603. And the visualization system responds to the difference, generates a request based on the sample to be adopted, and starts the first pre-estimation switch of the online recommendation model and the second pre-estimation switch of the offline recommendation model based on the request.
604. And after the first pre-estimation switch of an online recommendation model of the online system is started, carrying out forward output processing on the input sample to be adopted by adopting the online recommendation model so as to obtain online node data and record the online node data in an online log.
605. And after the second pre-estimation switch of the offline recommendation model of the offline system is started, carrying out forward output processing on the input sample to be adopted by adopting the offline recommendation model so as to obtain offline node data and recording the offline node data in an offline log.
606. The visualization system obtains the online logs and the offline logs, and performs data analysis on the online logs and the offline logs to obtain online node data and offline node data.
607. The visualization system acquires an online network topology file from the online system, acquires an offline network topology file from the offline system, and performs network analysis on the online network topology file and the offline network topology file to acquire online network description data and offline network description data.
608. The visualization system obtains topology difference data based on the online network description data and the offline network description data, and obtains node difference data based on the online node data and the offline node data.
609. And the visualization system generates a difference topological graph based on the topological difference data, displays the difference topological graph and the node difference data.
In the embodiment, the method can replace manual work, automatically position whether the network topology has difference, position the reason of the difference and the like, has high positioning efficiency and strong visualization, can directly trace to the problem source node according to the topology, and gives the approximate possibility of the occurrence of the problem.
Fig. 7 is a schematic diagram of a seventh embodiment according to the present disclosure, which provides a data processing apparatus 700 comprising: a first obtaining module 701, a second obtaining module 702 and a processing module 703.
The first obtaining module 701 is configured to, in response to a difference between an online recommendation model and an offline recommendation model, obtain data to be compared based on the online recommendation model and the offline recommendation model; a second obtaining module 702 is configured to obtain difference data of the difference based on the data to be compared; the processing module 703 is configured to perform processing operations based on the difference data, the processing operations including: displaying a visual image-text corresponding to the difference data, and/or positioning the reason of the difference based on the difference data.
In some embodiments, the data to be aligned comprises: the first obtaining module is specifically configured to:
acquiring online node data from an online log of an online system of the online recommendation model, wherein the online node data is acquired by the online system and recorded in the online log, the online node data is forward output data of a node in the online recommendation model, and the forward output data is acquired when a sample to be adopted is used as input of the online recommendation model and is subjected to forward output processing by the online recommendation model;
the offline node data are obtained from offline logs of an offline system of the offline recommendation model, the offline node data are obtained by the offline system and recorded in the offline logs, the offline node data are forward output data of nodes in the offline recommendation model, and the forward output data are obtained when the offline system takes a sample to be adopted as input of the offline recommendation model and the offline recommendation model is adopted to perform forward output processing on the sample to be adopted.
In some embodiments, further comprising:
the determining module is used for obtaining a pre-estimation difference value of each pre-estimation sample in a plurality of pre-estimation samples, wherein the pre-estimation difference value is a difference value between an online pre-estimation value and an offline pre-estimation value, the online pre-estimation value is obtained after the online recommendation model performs pre-estimation processing on each pre-estimation sample, and the offline pre-estimation value is obtained after the offline recommendation model performs pre-estimation processing on each pre-estimation sample; and taking the estimated sample with the maximum estimated value difference as the sample to be adopted.
In some embodiments, the online recommendation model performs forward output processing of the online recommendation model after the first predictor switch is turned on, and the offline recommendation model performs forward output processing of the offline recommendation model after the second predictor switch is turned on, and the apparatus further includes:
the generating module is used for generating a request based on the sample to be adopted;
and the starting module is used for starting the first pre-estimation switch of the online recommendation model and the second pre-estimation switch of the offline recommendation model based on the request.
In some embodiments, the data to be aligned comprises: the first obtaining module is specifically configured to:
acquiring a prestored online network topology file from an online system of the online recommendation model, and carrying out network analysis on the online network topology file to obtain online network description data;
and acquiring a prestored offline network topology file from an offline system of the offline recommendation model, and carrying out network analysis on the offline network topology file to acquire the offline network description data.
In some embodiments, the data to be aligned comprises: online network description data and offline network description data, and online node data and offline node data, the difference data comprising: the second obtaining module is specifically configured to:
if the online network description data and the offline network description data corresponding to the same node identifier are different, taking the different online network description data and the different offline network description data as the topology difference data; and/or the presence of a gas in the gas,
and if the online node data and the offline node data corresponding to the same node identification are different, taking the different online node data and the different offline node data as the node difference data.
In some embodiments, the visualization comprises: the processing module is specifically configured to:
if the difference data comprises topology difference data, generating a difference topological graph corresponding to the difference based on the topology difference data, and displaying the difference topological graph;
and if the difference data comprises node difference data, displaying the node difference data.
In some embodiments, the processing module is further configured to perform at least one of:
generating an online panoramic topological graph based on the online network description data, and displaying the online panoramic topological graph;
generating an offline panoramic topological graph based on the offline network description data, and displaying the offline panoramic topological graph;
displaying data details of the nodes in the online recommendation model based on the online node data;
and displaying the data details of the nodes in the offline recommendation model based on the offline node data.
In this embodiment, by acquiring the data to be compared, acquiring the difference data based on the data to be compared, and executing the processing operation based on the difference data, the automatic processing of the difference between the offline recommendation model and the online recommendation model can be realized.
In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.
It is to be understood that in the disclosed embodiments, the same or similar elements in different embodiments may be referenced.
It is to be understood that "first", "second", and the like in the embodiments of the present disclosure are used for distinction only, and do not indicate the degree of importance, the order of timing, and the like.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the electronic device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic apparatus 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the electronic device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (19)
1. A method of data processing, comprising:
responding to the difference between an online recommendation model and an offline recommendation model, and acquiring data to be compared based on the online recommendation model and the offline recommendation model;
acquiring difference data of the difference based on the data to be compared;
performing processing operations based on the difference data, the processing operations comprising: displaying a visual image-text corresponding to the difference data, and/or positioning the reason of the difference based on the difference data.
2. The method of claim 1, wherein the data to be aligned comprises: the obtaining of the data to be compared based on the online recommendation model and the offline recommendation model comprises the following steps:
acquiring online node data from an online log of an online system of the online recommendation model, wherein the online node data is acquired by the online system and recorded in the online log, the online node data is forward output data of a node in the online recommendation model, and the forward output data is acquired when a sample to be adopted is used as input of the online recommendation model and is subjected to forward output processing by the online recommendation model;
the offline node data are obtained from offline logs of an offline system of the offline recommendation model, the offline node data are obtained by the offline system and recorded in the offline logs, the offline node data are forward output data of nodes in the offline recommendation model, and the forward output data are obtained when the offline system takes a sample to be adopted as input of the offline recommendation model and the offline recommendation model is adopted to perform forward output processing on the sample to be adopted.
3. The method of claim 2, further comprising:
obtaining a pre-estimated value difference value of each pre-estimated sample in a plurality of pre-estimated samples, wherein the pre-estimated value difference value is a difference value between an online pre-estimated value and an offline pre-estimated value, the online pre-estimated value is obtained after the online recommendation model performs pre-estimation processing on each pre-estimated sample, and the offline pre-estimated value is obtained after the offline recommendation model performs pre-estimation processing on each pre-estimated sample;
and taking the estimated sample with the maximum estimated value difference as the sample to be adopted.
4. The method of claim 2, wherein the online recommendation model performs the forward output processing of the online recommendation model after a first predictor switch is turned on, and the offline recommendation model performs the forward output processing of the offline recommendation model after a second predictor switch is turned on, the method further comprising:
generating a request based on the sample to be adopted;
and starting the first pre-estimation switch of the online recommendation model and the second pre-estimation switch of the offline recommendation model based on the request.
5. The method of claim 1, wherein the data to be aligned comprises: the obtaining of the data to be compared based on the online recommendation model and the offline recommendation model comprises the following steps:
acquiring a prestored online network topology file from an online system of the online recommendation model, and carrying out network analysis on the online network topology file to obtain online network description data;
and acquiring a prestored offline network topology file from an offline system of the offline recommendation model, and carrying out network analysis on the offline network topology file to acquire the offline network description data.
6. The method of any one of claims 1-5, wherein the data to be aligned comprises: online network description data and offline network description data, and online node data and offline node data, the difference data comprising: the obtaining of the differential data based on the data to be compared includes:
if the online network description data and the offline network description data corresponding to the same node identifier are different, taking the different online network description data and the different offline network description data as the topology difference data; and/or the presence of a gas in the gas,
and if the online node data and the offline node data corresponding to the same node identification are different, taking the different online node data and the different offline node data as the node difference data.
7. The method of claim 6, wherein the visualization context comprises: the difference topological graph and/or the node difference data show the visual graph-text corresponding to the difference data, and the method comprises the following steps:
if the difference data comprises topology difference data, generating a difference topological graph corresponding to the difference based on the topology difference data, and displaying the difference topological graph; and/or the presence of a gas in the gas,
and if the difference data comprises node difference data, displaying the node difference data.
8. The method of claim 6, wherein the processing operations further comprise at least one of:
generating an online panoramic topological graph based on the online network description data, and displaying the online panoramic topological graph;
generating an offline panoramic topological graph based on the offline network description data, and displaying the offline panoramic topological graph;
displaying data details of the nodes in the online recommendation model based on the online node data;
and displaying the data details of the nodes in the offline recommendation model based on the offline node data.
9. A data processing apparatus comprising:
the first obtaining module is used for responding to the difference between an online recommendation model and an offline recommendation model and obtaining data to be compared based on the online recommendation model and the offline recommendation model;
a second obtaining module, configured to obtain difference data of the difference based on the data to be compared;
a processing module to perform processing operations based on the difference data, the processing operations comprising: displaying a visual image-text corresponding to the difference data, and/or positioning the reason of the difference based on the difference data.
10. The apparatus of claim 9, wherein the data to be aligned comprises: the first obtaining module is specifically configured to:
acquiring online node data from an online log of an online system of the online recommendation model, wherein the online node data is acquired by the online system and recorded in the online log, the online node data is forward output data of a node in the online recommendation model, and the forward output data is acquired when a sample to be adopted is used as input of the online recommendation model and is subjected to forward output processing by the online recommendation model;
the offline node data are obtained from offline logs of an offline system of the offline recommendation model, the offline node data are obtained by the offline system and recorded in the offline logs, the offline node data are forward output data of nodes in the offline recommendation model, and the forward output data are obtained when the offline system takes a sample to be adopted as input of the offline recommendation model and the offline recommendation model is adopted to perform forward output processing on the sample to be adopted.
11. The apparatus of claim 10, further comprising:
the determining module is used for obtaining a pre-estimation difference value of each pre-estimation sample in a plurality of pre-estimation samples, wherein the pre-estimation difference value is a difference value between an online pre-estimation value and an offline pre-estimation value, the online pre-estimation value is obtained after the online recommendation model performs pre-estimation processing on each pre-estimation sample, and the offline pre-estimation value is obtained after the offline recommendation model performs pre-estimation processing on each pre-estimation sample; and taking the estimated sample with the maximum estimated value difference as the sample to be adopted.
12. The apparatus of claim 10, wherein the online recommendation model performs the forward output processing of the online recommendation model after a first predictor switch is turned on, and the offline recommendation model performs the forward output processing of the offline recommendation model after a second predictor switch is turned on, the apparatus further comprising:
the generating module is used for generating a request based on the sample to be adopted;
and the starting module is used for starting the first pre-estimation switch of the online recommendation model and the second pre-estimation switch of the offline recommendation model based on the request.
13. The apparatus of claim 10, wherein the data to be aligned comprises: the first obtaining module is specifically configured to:
acquiring a prestored online network topology file from an online system of the online recommendation model, and carrying out network analysis on the online network topology file to obtain online network description data;
and acquiring a prestored offline network topology file from an offline system of the offline recommendation model, and carrying out network analysis on the offline network topology file to acquire the offline network description data.
14. The apparatus of any one of claims 9-13, wherein the data to be aligned comprises: online network description data and offline network description data, and online node data and offline node data, the difference data comprising: the second obtaining module is specifically configured to:
if the online network description data and the offline network description data corresponding to the same node identifier are different, taking the different online network description data and the different offline network description data as the topology difference data; and/or the presence of a gas in the gas,
and if the online node data and the offline node data corresponding to the same node identification are different, taking the different online node data and the different offline node data as the node difference data.
15. The apparatus of claim 14, wherein the visual context comprises: the processing module is specifically configured to:
if the difference data comprises topology difference data, generating a difference topological graph corresponding to the difference based on the topology difference data, and displaying the difference topological graph;
and if the difference data comprises node difference data, displaying the node difference data.
16. The apparatus of claim 14, wherein the processing module is further configured to perform at least one of:
generating an online panoramic topological graph based on the online network description data, and displaying the online panoramic topological graph;
generating an offline panoramic topological graph based on the offline network description data, and displaying the offline panoramic topological graph;
displaying data details of the nodes in the online recommendation model based on the online node data;
and displaying the data details of the nodes in the offline recommendation model based on the offline node data.
17. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.
19. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111154874.9A CN113836291B (en) | 2021-09-29 | 2021-09-29 | Data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111154874.9A CN113836291B (en) | 2021-09-29 | 2021-09-29 | Data processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113836291A true CN113836291A (en) | 2021-12-24 |
| CN113836291B CN113836291B (en) | 2023-08-15 |
Family
ID=78967607
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111154874.9A Active CN113836291B (en) | 2021-09-29 | 2021-09-29 | Data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113836291B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104216950A (en) * | 2014-08-18 | 2014-12-17 | 苏州同元软控信息技术有限公司 | Model comparison method based on physical topology |
| CN111105014A (en) * | 2019-11-27 | 2020-05-05 | 联想(北京)有限公司 | Data processing method, equipment and computer readable storage medium |
| CN112580823A (en) * | 2020-12-17 | 2021-03-30 | 北京嘀嘀无限科技发展有限公司 | Data processing method and device, readable storage medium and electronic equipment |
| CN112994940A (en) * | 2019-05-29 | 2021-06-18 | 华为技术有限公司 | Network anomaly detection method and device |
| CN113051479A (en) * | 2021-04-19 | 2021-06-29 | 北京百度网讯科技有限公司 | File processing and recommendation information generation method, device, equipment and storage medium |
| WO2021135588A1 (en) * | 2019-12-31 | 2021-07-08 | 百果园技术(新加坡)有限公司 | Recommendation method, model generation method and apparatus, medium and device |
| CN113190746A (en) * | 2021-04-27 | 2021-07-30 | 北京百度网讯科技有限公司 | Recommendation model evaluation method and device and electronic equipment |
-
2021
- 2021-09-29 CN CN202111154874.9A patent/CN113836291B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104216950A (en) * | 2014-08-18 | 2014-12-17 | 苏州同元软控信息技术有限公司 | Model comparison method based on physical topology |
| CN112994940A (en) * | 2019-05-29 | 2021-06-18 | 华为技术有限公司 | Network anomaly detection method and device |
| CN111105014A (en) * | 2019-11-27 | 2020-05-05 | 联想(北京)有限公司 | Data processing method, equipment and computer readable storage medium |
| WO2021135588A1 (en) * | 2019-12-31 | 2021-07-08 | 百果园技术(新加坡)有限公司 | Recommendation method, model generation method and apparatus, medium and device |
| CN112580823A (en) * | 2020-12-17 | 2021-03-30 | 北京嘀嘀无限科技发展有限公司 | Data processing method and device, readable storage medium and electronic equipment |
| CN113051479A (en) * | 2021-04-19 | 2021-06-29 | 北京百度网讯科技有限公司 | File processing and recommendation information generation method, device, equipment and storage medium |
| CN113190746A (en) * | 2021-04-27 | 2021-07-30 | 北京百度网讯科技有限公司 | Recommendation model evaluation method and device and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| JUN MA 等: "A review for dynamics in neuron and neuronal network", NONLINEAR DYNAMICS * |
| 郭玉荣 等: "基于离线模型参数修正的结构混合试验方法", 自然灾害学报, vol. 30, no. 2 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113836291B (en) | 2023-08-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113342345A (en) | Operator fusion method and device of deep learning framework | |
| CN113657100B (en) | Entity identification method, entity identification device, electronic equipment and storage medium | |
| CN114445047A (en) | Workflow generation method and device, electronic equipment and storage medium | |
| CN112835579A (en) | Method and device for determining interface code, electronic equipment and storage medium | |
| CN113190746A (en) | Recommendation model evaluation method and device and electronic equipment | |
| CN116737928B (en) | Text mining method and system for platform user requirements and cloud platform | |
| CN115186738B (en) | Model training method, device and storage medium | |
| CN114880498B (en) | Event information display method and device, equipment and medium | |
| CN114141236B (en) | Language model updating method and device, electronic equipment and storage medium | |
| CN113032251B (en) | Method, device and storage medium for determining service quality of application program | |
| CN113836291B (en) | Data processing method, device, equipment and storage medium | |
| CN113691403A (en) | Topological node configuration method, related device and computer program product | |
| CN113052325A (en) | Method, device, equipment, storage medium and program product for optimizing online model | |
| CN112989066A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN113835893B (en) | Data processing method, device, equipment, storage medium and program product | |
| CN114281981B (en) | News brief report generation method and device and electronic equipment | |
| CN115687141A (en) | Application program testing method and device, electronic equipment and storage medium | |
| CN114416678A (en) | Resource processing method, device, equipment and storage medium | |
| CN114997329A (en) | Method, apparatus, device, medium and product for generating a model | |
| CN113327133A (en) | Data recommendation method, data recommendation device, electronic equipment and readable storage medium | |
| CN113849170A (en) | Code sharing method and device, electronic equipment and storage medium | |
| CN114501112A (en) | Method, apparatus, device, medium and product for generating video notes | |
| CN117785165A (en) | Chart generation method and device based on large model and electronic equipment | |
| CN116578555A (en) | Data verification method, system, electronic equipment and storage medium | |
| CN115292179A (en) | Test data generation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |