CN116842238A

CN116842238A - Method and system for realizing enterprise data visualization based on big data analysis

Info

Publication number: CN116842238A
Application number: CN202310913881.5A
Authority: CN
Inventors: 高辉杰; 庄志远; 孙岚
Original assignee: Wuhan Saisiyun Technology Co ltd
Current assignee: Youlai Beijing Technology Co ltd
Priority date: 2023-07-24
Filing date: 2023-07-24
Publication date: 2023-10-03
Anticipated expiration: 2043-07-24
Also published as: CN116842238B

Abstract

The invention provides a method and a system for realizing enterprise data visualization based on big data analysis, which are characterized in that an enterprise service data set is obtained, and a service behavior representation vector of a target service type in a service data cluster is extracted; based on the service behavior characterization vector and the service behavior detection variable, determining probability parameters of the target service type corresponding to each service behavior result classification, and then determining service behavior result classification of target service type matching in the service data cluster based on the probability parameters; counting at least one business behavior result classification corresponding to each business data cluster in the enterprise business data set to obtain a classification counting result; and finally, generating visual display information according to the classified statistical result, and displaying the visual display information on a data visual system. Therefore, the business behavior detection model is debugged through the simulated data cluster, and a large number of actually generated enterprise data templates are not required to be collected for debugging the business behavior detection model, so that training cost is saved, and efficiency is improved.

Description

Method and system for realizing enterprise data visualization based on big data analysis

Technical Field

The application relates to the field of data processing and artificial intelligence, in particular to an enterprise data visualization realization method and system based on big data analysis.

Background

As more and more businesses change to digital, data has become a non-negligible production profile for the business. Products for managing enterprise data, such as cloud computing platforms providing one-stop data analysis management, are beginning to appear on the market. The data are subjected to full value chain management from data access integration, data processing, analysis and mining, and terminal visualization, and the full chain provides the basis based on technologies such as big data, artificial intelligence and the Internet of things, so that full industries and full scenes such as electronic commerce, automobiles, properties, finance, retail, energy sources and the Internet can be covered. Based on different analysis requirements and scenes, the data processing process also has a corresponding mode method, generally speaking, in the process of summarizing the data visualization realization process, classification statistics is carried out on the data, the order of magnitude visual presentation is completed, for example, in the e-commerce product investigation statistics, aiming at the target type e-commerce product, when the acceptance of the user audience to the target type e-commerce product is counted, the acceptance type can be divided according to the requirement of analysis granularity, such as like, interested, noninductive, annoyance and the like, the acceptance of different users is classified, and the polarity visual display is carried out after the statistics. Then, this involves a process of analyzing massive data (business response behavior of the user), and it is a big problem how to efficiently and accurately implement classification of data due to the large amount of data involved in the e-commerce platform.

The artificial intelligence provides a solution for high-efficiency analysis of data, and classification output can be automatically, quickly and accurately carried out on the data through training a neural network model, but the process depends on collection of early-stage large-range training samples, debugging training is carried out on the model, and acquisition of the early-stage training samples is a difficult task for a service provider, so that the process of realizing enterprise data analysis visualization based on the large data generates new technical problems to be solved.

Disclosure of Invention

The application aims to provide an enterprise data visualization realization method and system based on big data analysis, which solve the technical problems.

Other features and advantages of the application will be apparent from the following detailed description, or may be learned by the practice of the application.

According to an aspect of an embodiment of the present application, there is provided an enterprise data visualization implementation method based on big data analysis, which is characterized in that the method is applied to a data visualization system, and the method includes:

acquiring an enterprise business data set, wherein the enterprise business data set comprises a plurality of business data clusters to be analyzed;

acquiring each business data cluster to be analyzed;

Extracting a service behavior representation vector of a target service type in the service data cluster;

determining probability parameters of the target service type corresponding to each service behavior result classification based on the service behavior characterization vector and the service behavior detection variable; the service behavior detection variable is a neural network parameter obtained after debugging based on a simulated service data cluster and an actual service environment data cluster, wherein the simulated service data cluster comprises data cluster portrait environment service behaviors for simulating service behaviors generated by service types;

determining service behavior result classification matched with the target service type in the service data cluster based on the probability parameter;

counting at least one business behavior result classification corresponding to each business data cluster in the enterprise business data set to obtain a classification counting result; the classification statistical result comprises the quantity corresponding to each business behavior result classification;

and generating visual display information according to the classification statistical result, and displaying the visual display information on the data visual system.

In an alternative embodiment, before acquiring the service data cluster to be analyzed, the method further includes:

Acquiring the simulated service data cluster and the actual service environment data cluster; the simulation service data clusters are obtained through the output of a plurality of simulation service data sets, each simulation service data set carries classification indication information, and the classification indication information comprises a first training sample classification for indicating that the simulation service data set is a target classification service behavior and a second training sample classification for indicating that the simulation service data set is not the target classification service behavior;

based on the simulated service data cluster and the actual service environment data cluster, debugging the pre-trained service behavior detection model, and stopping debugging when the debugging stop requirement is met; the pre-trained business behavior detection model comprises a first construction model used for transforming an actual business environment data cluster into a simulated business environment data cluster, a second construction model used for transforming the simulated business environment data cluster into the actual business environment data cluster and a plurality of debugging analysis models, and when the debugging cut-off requirement is met, the model internal configuration variable of the first construction model is determined to be the business behavior detection variable.

In an optional implementation manner, the debugging the pre-trained service behavior detection model based on the simulated service data cluster and the actual service environment data cluster, and stopping the debugging when the debugging cut-off requirement is met includes:

loading the simulated service data cluster to the second construction model to construct a first comparison actual service environment data cluster; when the analysis result representation of the first comparison actual business environment data cluster meets the first comparison debugging cut-off requirement, loading the first comparison actual business environment data cluster into the first construction model to construct a first comparison simulation business environment data cluster; acquiring a first error result between the simulated service data cluster and the first contrast simulated service environment data cluster;

loading the actual service environment data cluster to the first construction model to construct a second comparison simulation service environment data cluster; when the analysis result representation of the second comparison simulation service environment data cluster meets the second comparison debugging cut-off requirement, loading the second comparison simulation service environment data cluster into the second construction model to construct a second comparison actual service environment data cluster; acquiring a second error result between the actual service environment data cluster and the second comparison actual service environment data cluster;

And when the first error result and the second error result meet the first error value requirement, determining that the business behavior detection model meets the debugging cut-off requirement.

In an optional implementation manner, the loading the simulated service data cluster into the second construction model, and constructing the first comparison actual service environment data cluster includes:

loading the plurality of simulated service data sets in the simulated service data cluster to the second construction model to obtain a comparison portrait actual service environment data cluster;

when the analysis result characterization of the first comparison actual business environment data cluster meets the first comparison debugging cut-off requirement, loading the first comparison actual business environment data cluster into the first construction model, and constructing a first comparison simulation business environment data cluster comprises the following steps: loading the comparison portrait actual service environment data cluster to a first debugging analysis model to obtain a first analysis result; optimizing model internal configuration variables of the second building model when the first analysis result characterization does not meet the first debugging cut-off requirement; when the first analysis result representation meets the first debugging cut-off requirement, loading the comparison portrait actual service environment data cluster into the first construction model to obtain a comparison simulation service environment data cluster;

The step of obtaining a first error result between the simulated service data cluster and the first contrast simulated service environment data cluster comprises the following steps: obtaining portrait differences between the simulated service behavior data clusters in the simulated service data set and the contrast simulated service environment data clusters; optimizing model internal configuration variables in the first construction model according to the portrait difference when the portrait difference characterization does not meet a second debugging cut-off requirement; and when the representation of the portrait difference meets the second debugging cut-off requirement, acquiring the first error result according to the portrait difference.

In an alternative embodiment, loading the simulated service data cluster into the second construction model, and constructing the first comparison actual service environment data cluster includes: loading business behavior context evolution information corresponding to the plurality of simulated business data sets in the simulated business data cluster to the second construction model to obtain an actual business environment data cluster for comparison with the business behavior context;

when the analysis result characterization of the first comparison actual business environment data cluster meets the first comparison debugging cut-off requirement, loading the first comparison actual business environment data cluster into the first construction model, and constructing a first comparison simulation business environment data cluster comprises the following steps: loading the actual business environment data cluster of the contrast business behavior context to a second debugging analysis model to obtain a second analysis result; optimizing model internal configuration variables of the second construction model when the second analysis result characterization does not meet a third debugging cut-off requirement; when the second analysis result representation meets the third debugging cut-off requirement, loading the comparison business behavior context actual business environment data cluster into the first construction model to obtain a comparison business behavior context simulation business environment data cluster;

The step of obtaining a first error result between the simulated service data cluster and the first contrast simulated service environment data cluster comprises the following steps: acquiring service behavior differences between the simulated service behavior data clusters in the simulated service data set and the simulated service environment data clusters of the contrast service behavior context; when the service behavior difference characterization does not meet a fourth debugging cut-off requirement, optimizing a model internal configuration variable in the first construction model according to the service behavior difference; and when the service behavior difference characterization meets the fourth debugging cut-off requirement, acquiring the first error result according to the service behavior difference.

In an optional implementation manner, the debugging the pre-trained service behavior detection model based on the simulated service data cluster and the actual service environment data cluster, and stopping the debugging when the debugging stop requirement is met further includes:

dividing the simulated service data cluster into a first simulated data cluster and a second simulated data cluster, and dividing the actual service environment data cluster into a first actual data cluster and a second actual data cluster, wherein the construction moment of the first simulated data cluster is positioned before the construction moment of the second simulated data cluster, and the construction moment of the first actual data cluster is positioned before the construction moment of the second actual data cluster;

Loading the first simulation data cluster to the second construction model to obtain a comparison time sequence simulation service environment data cluster; acquiring a first time difference between the second analog data cluster and the contrast time sequence analog service environment data cluster;

loading the first actual data cluster to the first construction model to obtain a comparison time sequence actual service environment data cluster; acquiring a second time difference between the second actual data cluster and the contrast time sequence actual service environment data cluster;

and when the first time difference and the second time difference meet the second error value requirement, determining that the business behavior detection model meets the debugging cut-off requirement.

In an alternative embodiment, determining that the business behavior detection model meets the debug cutoff requirement includes:

acquiring a fusion result of the first error result, the second error result, the first time difference and the second time difference;

and when the fusion result characterization meets a target numerical result, determining that the business behavior detection model meets the debugging cut-off requirement.

In an optional implementation manner, after the pre-trained service behavior detection model is debugged based on the simulated service data cluster and the actual service environment data cluster and the debugging stop requirement is met, the method further includes:

Obtaining output information of a classification mapping network connected with the first construction model in the business behavior detection model meeting the debugging cut-off requirement;

when the output information representation meets detection requirements, determining a model internal configuration variable of the first construction model as the business behavior detection variable; the detection requirement characterizes that the classification information corresponding to the output information is the same as the classification indication information of the analog service data set matched with the output information.

According to another aspect of the embodiment of the present application, there is provided a training method for a business behavior detection model, including:

obtaining a simulated service data cluster and an actual service environment data cluster, wherein the simulated service data cluster comprises a data cluster for generating service behaviors by simulating service types in a simulated service data set, the actual service environment data cluster comprises an actual service environment portrait data cluster and an actual service environment service behavior data cluster, the simulated service data cluster is obtained by generating a plurality of simulated service data sets, each simulated service data set carries classification indication information, and the classification indication information comprises a first training sample classification for indicating that the simulated service data set is a target classified service behavior and a second training sample classification for indicating that the simulated service data set is not the target classified service behavior;

And debugging the pre-trained business behavior detection model based on the simulated business data cluster and the actual business environment data cluster, and stopping debugging when the debugging cut-off requirement is met, wherein the pre-trained business behavior detection model comprises a first construction model for converting the actual business environment data cluster into the simulated business environment data cluster, a second construction model for converting the simulated business environment data cluster into the actual business environment data cluster and a plurality of debugging analysis models, and when the debugging cut-off requirement is met, the model internal configuration variable of the first construction model is determined to be the business behavior detection variable.

According to still another aspect of an embodiment of the present application, there is provided a data visualization system including:

a processor;

and a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method described above via execution of the executable instructions.

The beneficial effects of the application at least comprise:

the method comprises the steps of obtaining an enterprise business data set, wherein the enterprise business data set comprises a plurality of business data clusters to be analyzed; acquiring each service data cluster to be analyzed and extracting a service behavior representation vector of a target service type in the service data cluster; and determining probability parameters of the target service type corresponding to each service behavior result classification based on the service behavior characterization vector and the service behavior detection variable. The service behavior detection variable is a neural network parameter obtained after debugging based on a simulated service data cluster and an actual service environment data cluster, wherein the simulated service data cluster comprises a data cluster for simulating service types to generate service behaviors; determining service behavior result classification matched with the target service type in the service data cluster based on the probability parameter; counting at least one business behavior result classification corresponding to each business data cluster in the enterprise business data set to obtain a classification statistical result, wherein the classification statistical result comprises the number corresponding to each business behavior result classification; and finally, generating visual display information according to the classified statistical result, and displaying the visual display information on a data visual system. Therefore, the application debugs the business behavior detection model through the simulated data cluster, and debugs the business behavior detection model without collecting mass actually generated enterprise data templates, thereby saving the model training cost of the business behavior detection and improving the efficiency.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is evident that the drawings in the following description are only some embodiments of the present application and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.

Fig. 1 is a flowchart of an enterprise data visualization implementation method based on big data analysis according to an embodiment of the present application.

Fig. 2 is a schematic diagram of a functional module architecture of an enterprise data visualization implementation apparatus according to an embodiment of the present application.

Fig. 3 is a schematic diagram of a data visualization system according to an embodiment of the present application.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.

The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.

The enterprise data visualization implementation method based on big data analysis provided by the embodiment of the application is applied to a data visualization system in communication connection with a user terminal, referring to fig. 1, the method comprises the following steps:

step S110, an enterprise business data set is acquired.

The enterprise service data set comprises a plurality of service data clusters to be analyzed, for example, the enterprise service data set is a data set formed by service behavior data of a plurality of users collected by an internet platform, for example, an e-commerce platform enterprise A collects behavior data generated by users 1 to 1000 in a collection period 1 according to a preset buried point, the behavior data comprises response behaviors aiming at target service types, wherein the target service types can be pushed commodity information or live broadcast marketing information and the like, the service behaviors responded by the users can be clicking, browsing, sharing, collecting, purchasing, blackening, reporting, negatively feeding back and the like, and different service behaviors can be assigned by pre-assigned codes to obtain service behavior data (discrete information is converted into numerical information).

Step S120, each business data cluster to be analyzed is obtained.

Step S130, extracting service behavior characterization vector of target service type in service data cluster.

Step S140, determining probability parameters of the target service type corresponding to each service behavior result classification based on the service behavior characterization vector and the service behavior detection variable.

In the embodiment of the application, the service behavior detection variable is a neural network parameter obtained after debugging based on a simulated service data cluster and an actual service environment data cluster (the actual service environment can be the data environment of a user such as real commodity information, webpage information and the like), and the simulated service data cluster comprises a data cluster for simulating service types to generate service behaviors. The probability parameter of each business behavior result classification is the probability evaluation result of the corresponding business behavior result classification of the target business type (i.e. the business type to be analyzed, such as the above-mentioned various commodity pushing information), which can be represented by the probability parameter or can also be represented by the confidence level.

Step S150, determining the business behavior result classification matched with the target business type in the business data cluster based on the probability parameter.

Step S160, at least one business behavior result classification corresponding to each business data cluster in the enterprise business data set is counted, and a classification counting result is obtained.

The classification statistics result includes the number corresponding to each business action result classification, for example, the number corresponding to the business action result classification a is 119, the number corresponding to the business action result classification B is 310, and the number corresponding to the business action result classification C is 124 … ….

Step S170, generating visual display information according to the classified statistical result, and displaying the visual display information on a data visual system.

For example, the visual display information can be visually displayed through preset visual elements such as a histogram, a pie chart, a graph, a bubble chart and the like, and the visual display information is selected according to actual needs, and is not limited in particular.

In the above steps, the device for acquiring the service data cluster may be various user terminals, such as various PCs (desktop, tablet, notebook, etc.), mobile phones, smart TVs, wearable devices, etc. And acquiring the enterprise business data set to be analyzed through the user terminal, extracting the enterprise business data set to be analyzed according to the user account by adopting preprocessing to acquire a business data cluster, and processing the business data cluster. The user terminal is used for acquiring the user business behavior data which actually occurs on the enterprise platform. It can be known that the embodiment of the application combines the business behavior detection variables to detect the business behavior type of the business data set acquired in actual conditions. Because the learning samples of the enterprise business data set including the actual situation are not easy to acquire, if the business behavior detection variable is debugged based on the learning samples of the enterprise business data set including the business behavior of the actual situation, the number of the learning samples (i.e. templates or samples for training) may not be enough, and the obtained model may not be debugged, so that the effect may not reach the requirement, and therefore, the sample of the enterprise business data set generated in the massive actual situation needs to be acquired, thus increasing the cost of model training and having low efficiency.

Based on the problems, the application overcomes the problems by combining the simulated service data cluster and the actual service environment data cluster to debug the service behavior detection variable, and the simulated service data cluster learning sample is similar to the actual service data cluster learning sample, and can be automatically constructed to obtain the service data cluster learning sample containing the service behavior, so that the collection process of the service behavior data cluster learning sample can be greatly reduced.

When a service data cluster to be analyzed, which is acquired by a user terminal, is obtained, a service behavior characterization vector which is generated correspondingly to a target service type in the data cluster to be analyzed is extracted, wherein the service behavior characterization vector is vector characterization information for characterizing corresponding target service behavior feature information, and is a feature vector. And then, loading the service behavior characterization vector into a conversion model, and obtaining probability parameters through service behavior detection variables obtained based on debugging in the model. And then, completing business behavior classification statistics of all business data clusters in the whole enterprise business data set, generating corresponding visual display information for display, completing visual presentation of data, and improving enterprise data management and control capability in a multi-application scene. The method comprises the steps of obtaining a business data cluster to be analyzed, extracting a business behavior representation vector of a target business type in the business data cluster, determining probability parameters of the target business type corresponding to each business behavior result classification based on the business behavior representation vector and a business behavior detection variable, wherein the business behavior detection variable is a neural network parameter obtained after debugging based on a simulated business data cluster and an actual business environment data cluster, the simulated business data cluster comprises a data cluster for simulating business behaviors generated by the business types, and determining the business behavior result classification matched with the target business type in the business data cluster based on the probability parameters. The business behavior detection model is debugged based on the simulated data cluster, and the business behavior detection model is debugged without collecting mass enterprise data template data actually generated, so that the model training cost of the business behavior detection is saved, and the efficiency is improved.

In a specific embodiment, before acquiring the service data cluster to be analyzed, the method provided by the application further comprises:

step S101, obtaining a simulated service data cluster and an actual service environment data cluster.

The simulation service data clusters are obtained through the output of a plurality of simulation service data sets, each simulation service data set carries classification indication information, and the classification indication information comprises first training sample classification for indicating the simulation service data set as a target classification service behavior and second training sample classification for indicating the simulation service data set as a non-target classification service behavior.

Step S102, based on the simulated service data cluster and the actual service environment data cluster, the pre-trained service behavior detection model is debugged, and the debugging is stopped when the debugging stop requirement is met.

The pre-trained business behavior detection model comprises a first construction model used for transforming an actual business environment data cluster into a simulated business environment data cluster, a second construction model used for transforming the simulated business environment data cluster into the actual business environment data cluster and a plurality of debugging analysis models, and when the debugging cut-off requirement is met, the internal configuration variables of the models of the first construction model are determined to be business behavior detection variables.

In the embodiment of the application, the data clusters for performing model debugging are respectively derived from three data sets, specifically, the simulated business data clusters are derived from a simulated material database comprising business behaviors, and the actual business environment data clusters can be derived from an actual situation business behavior database based on actual business behavior portraits and a database for providing the actual situation portraits, wherein the portraits are user portraits and are user tag sequences obtained through pre-analysis. The acquired simulated material database then contains enterprise business data sets and data set business behavior indication information indicating whether the corresponding characterization data sets include business behaviors, in other words, one enterprise business data set corresponds to one classification indication information.

The detection of the service behavior is realized through a detection network, when the detection network is debugged, each time a simulated service data cluster is loaded, a data cluster formed by the service behavior data cluster of the actual situation and the portrait data cluster of the actual situation is arbitrarily selected to be loaded into a basic network for debugging until the debugging cut-off requirement is met.

Based on the above processes of the embodiment of the application, the simulated service data cluster and the actual service environment data cluster are obtained, the pre-trained service behavior detection model is debugged based on the simulated service data cluster and the actual service environment data cluster, and the debugging is stopped when the debugging stop requirement is met, in other words, the service behavior detection model is debugged based on the simulated data cluster, and a large amount of actually generated enterprise data templates are not required to be collected for debugging the service behavior detection model, so that the model training cost of service behavior detection is saved, and the efficiency is improved.

In a specific embodiment, based on the simulated service data cluster and the actual service environment data cluster, the pre-trained service behavior detection model is debugged, and the debugging is stopped when the debugging stop requirement is met, which specifically comprises:

step S201, loading the simulated service data cluster to a second construction model to construct a first comparison actual service environment data cluster; when the analysis result representation of the first comparison actual business environment data cluster meets the first comparison debugging cut-off requirement, loading the first comparison actual business environment data cluster into a first construction model to construct a first comparison simulation business environment data cluster; and obtaining a first error result between the simulated service data cluster and the first contrast simulated service environment data cluster.

Step S202, loading an actual business environment data cluster to a first construction model to construct a second contrast simulation business environment data cluster; when the analysis result representation of the second comparison simulation service environment data cluster meets the second comparison debugging cut-off requirement, loading the second comparison simulation service environment data cluster into a second construction model to construct a second comparison actual service environment data cluster; and obtaining a second error result between the actual service environment data cluster and a second comparison actual service environment data cluster.

Step S203, when the first error result and the second error result meet the first error value requirement, determining that the business behavior detection model meets the debugging cut-off requirement.

It should be understood that, based on the above embodiment, since the model for performing actual business behavior classification detection is obtained by using the simulated enterprise business data set debugging, when the debugging of the configuration variables inside the model is completed based on the above steps, it is necessary to ensure the portrait consistency of the simulated data and the data of the actual situation (including the user portrait of the user behavior dynamic style, the user basic situation static style, etc.). Then, in addition to focusing on the portrait changes of the data clusters, focusing on the differences between the simulated business behavior and the actual business behavior, and ensuring the sequence of the business behavior after construction. Based on the above consideration, the data environment consistency branch line of the embodiment of the application comprises three branches of portrait data environment consistency portrait, business behavior data environment consistency behavior and context time sequence limit context. The embodiment of the application completes parameter debugging of the portrait branch line based on the method, and the portrait branch line is used for realizing data environment consistency of enterprise business data set portraits.

The embodiment of the application can complete the conversion of data based on the loop generation countermeasure network, load a simulated enterprise service data set R1' to the network, extract the characterization vector by adopting the Net1, and interpolate the characterization vector to the original size based on Linear Interpolation (LI, linear interpolation) to obtain the enterprise service data set R1 of the simulated actual situation portrait constructed by the simulated service data cluster generation. And then loading R1 into an analysis network, wherein the analysis network can be a classification classifier which is used for identifying a true actual condition image service data set and a simulated actual condition image service data set. After debugging is finished, the probability of service behavior classification is close to 50%, in other words, the analysis network is difficult to judge reality and simulation, and then the data cluster constructed by the constructed model meets the requirement, and at the moment, after the Net1 is trained based on learning, the simulation service data cluster can be converted into the actual situation portrait service data cluster. And loading R1 into a second construction model Net2 to obtain a reconstructed simulation service data cluster R2, carrying out MSE (mean square error) processing on R2 and R1', gradually approaching R2 and R1', wherein the Net1 and the Net2 can be respectively transformed from the simulation service data cluster to an actual situation portrait service data cluster and from the actual situation portrait to the simulation service data cluster after training.

Based on the above embodiment, the simulated service data cluster is loaded to the second construction model, and the first comparison actual service environment data cluster is constructed; when the analysis result representation of the first comparison actual business environment data cluster meets the first comparison debugging cut-off requirement, loading the first comparison actual business environment data cluster into a first construction model to construct a first comparison simulation business environment data cluster; acquiring a first error result between the simulated service data cluster and a first contrast simulated service environment data cluster; loading the actual service environment data cluster into a first construction model to construct a second comparison simulation service environment data cluster; when the analysis result representation of the second comparison simulation service environment data cluster meets the second comparison debugging cut-off requirement, loading the second comparison simulation service environment data cluster into a second construction model to construct a second comparison actual service environment data cluster; acquiring a second error result between the actual service environment data cluster and a second comparison actual service environment data cluster; when the first error result and the second error result meet the first error value requirement, determining that the business behavior detection model meets the debugging cut-off requirement, so that a model adapting to simulating the data environment to the actual data environment representation is obtained through debugging, and the model representation conversion speed is improved. In the embodiment of the application, the debugging cut-off requirement means that the corresponding network reaches a convergence state, and the corresponding conditions can be that the preset debugging times are reached, the error reaches a preset value, and the like.

In one implementation manner, the method provided by the embodiment of the application further comprises the following steps:

step S301, loading the simulated service data cluster into a second construction model, and constructing a first comparison actual service environment data cluster includes: and loading a plurality of simulated service data sets in the simulated service data cluster to a second construction model to obtain the actual service environment data cluster of the comparison portrait.

Step S302, when the analysis result of the first comparison actual service environment data cluster indicates that the first comparison debugging cut-off requirement has been met, loading the first comparison actual service environment data cluster into a first construction model, and constructing a first comparison simulation service environment data cluster includes: loading the comparison portrait actual service environment data cluster into a first debugging analysis model to obtain a first analysis result; optimizing model internal configuration variables of the second construction model when the first analysis result representation does not meet the first debugging cut-off requirement; and when the first analysis result representation meets the first debugging cut-off requirement, loading the comparison portrait actual business environment data cluster into a first construction model to obtain a comparison simulation business environment data cluster.

Step S303, obtaining a first error result between the analog service data cluster and the first comparison analog service environment data cluster includes: obtaining portrait differences between a simulated business behavior data cluster in a simulated business data set and a contrast simulated business environment data cluster; when the representation difference characterization does not meet the second debugging cut-off requirement, optimizing a model internal configuration variable in the first construction model according to the representation difference; and when the representation of the portrait difference meets the second debugging cut-off requirement, acquiring a first error result according to the portrait difference.

In the foregoing process, because the constraint information (supervision information) is convoluted, net2 has poor effect, and in order to improve the binary nature of the model, the above process needs to be performed once in reverse, specifically including: and loading the actual condition image enterprise service data set R3 into a network, extracting the characterization vector based on Net2, and interpolating the characterization vector to the original size by LI to obtain a simulated service data cluster R1 constructed by R3. Based on the Net2, the constructed R2 is obtained and loaded to a Classifier to analyze whether the Classifier is a real simulated service data cluster. After debugging is completed, the probability of classifier recognition is close to 50%, and at this time, net2 can possess partial performance of converting actual situation portrait service data clusters into simulated service data clusters after training. And loading R2 into the construction model Net1 to obtain a reconstructed simulated actual condition image service data cluster R1, performing MSE processing on the R1 and the R3 to gradually make the R1 and the R3 similar, wherein at the moment, the Net1 and the Net2 respectively improve the performance of converting the simulated service data cluster into the actual condition image service data cluster from the actual condition image into the simulated service data cluster after retraining.

Based on the method, the reverse execution is completed by adopting the steps, so that the binary dual of the model is realized, and the accuracy of the detection model on the service behavior detection is improved. Alternatively, the above model may be debugged in combination with, and in other embodiments summarized, only the following steps may be used to debug the above model.

Specifically, the debugging process includes:

step S401, loading the simulated service data cluster to a second construction model, and constructing a first comparison actual service environment data cluster comprises: and loading the business behavior context evolution information corresponding to the plurality of simulated business data sets in the simulated business data cluster to a second construction model to obtain the actual business environment data cluster in comparison with the business behavior context.

Step S402, when the analysis result of the first comparison actual service environment data cluster indicates that the first comparison debugging cut-off requirement has been met, loading the first comparison actual service environment data cluster into a first construction model, and constructing a first comparison simulation service environment data cluster includes: loading the actual business environment data cluster of the contrast business behavior context to a second debugging analysis model to obtain a second analysis result; when the second analysis result characterization does not meet the third debugging cut-off requirement, optimizing the model internal configuration variables of the second construction model; and when the second analysis result representation meets the third debugging cut-off requirement, loading the actual business environment data cluster of the comparison business behavior context into the first construction model to obtain the simulated business environment data cluster of the comparison business behavior context.

Step S403, acquiring a first error result between the analog service data cluster and the first comparison analog service environment data cluster includes: acquiring business behavior differences between a business behavior simulating data cluster in a business behavior simulating data set and a business behavior context simulating data cluster; when the service behavior difference characterization does not meet the fourth debugging cut-off requirement, optimizing the model internal configuration variables in the first construction model according to the service behavior difference; and when the service behavior difference characterization meets the fourth debugging cut-off requirement, acquiring a first error result according to the service behavior difference.

The method is to carry out consistency debugging on the business behavior data environments of the simulated business environment data clusters and the actual business environment data clusters of the detection model. The data cluster combination of the service behavior data environment consistency is respectively the context evolution information of the simulated service data clusters (namely the change information of the service behavior data corresponding to the service type in different data clusters) and the context evolution information of the actual condition service data set, net1 and Net2 are still the above networks, and the analysis networks are respectively Net A for judging whether the input data is the actual condition service behavior evolution data set or not; netB to determine if the incoming data is a true analog service data cluster evolution dataset.

In actual execution, a context evolution information set R4' of a simulated service data cluster set is loaded to a network, after a characterization vector is extracted through Net1, the characterization vector is interpolated to the original size by LI to obtain a simulated actual situation context evolution enterprise service data set R4 constructed by the context evolution information of the simulated service data cluster, and then the R4 is loaded to an analysis network, namely a classifier NetA, to analyze the actual situation evolution data set and the simulated actual situation evolution data set. After the debugging is finished, the recognition probability is close to 50%, and the analysis network is difficult to distinguish between reality and simulation. At this time, net1 has a part of performance of transforming the analog service data cluster into the context evolution information of the actual situation after training. Loading R4 into a second construction model Net2 to obtain reconstructed simulated context evolution information R5, performing MSE processing on R5 and R4', and gradually enabling R5 and R4' to be similar, wherein Net1 and Net2 respectively have the performance of transforming the simulated service data cluster context evolution information into actual situation context evolution information and transforming the actual situation context evolution information into the simulated service data cluster context evolution information after training.

In one embodiment, because of the convolution of constraint information, net2 has insufficient capability to transform actual context evolution information into simulated context evolution information, based on which the above procedure is performed in reverse to ensure the binary nature of the model, which refers to the process of debugging the environmental consistency of the representation data.

Based on the above embodiment, the model is debugged based on the context evolution information of the simulated enterprise service data set and the context evolution information of the actual condition service data set, so that the transformation performance of the model on the consistency of the service behavior data environments of the simulated service data set and the actual condition service data set is increased, and the detection speed of the detection model is improved.

In a specific embodiment, based on the simulated service data cluster and the actual service environment data cluster, the pre-trained service behavior detection model is debugged, and when the debugging stop requirement is met, the method further comprises the steps of:

in step S501, the analog service data cluster is divided into a first analog data cluster and a second analog data cluster, and the actual service environment data cluster is divided into a first actual data cluster and a second actual data cluster.

The construction time of the first simulation data cluster is located before the construction time of the second simulation data cluster, and the construction time of the first actual data cluster is located before the construction time of the second actual data cluster.

Step S502, loading the first simulation data cluster to a second construction model to obtain a comparison time sequence simulation service environment data cluster; and acquiring a first time difference between the second analog data cluster and the contrast time sequence analog service environment data cluster.

Step S503, loading a first actual data cluster to a first construction model to obtain a data cluster of the actual business environment in comparison with the time sequence; and obtaining a second time difference between the second actual data cluster and the actual business environment data cluster of the comparison time sequence.

Step S504, when the first time difference and the second time difference meet the second error value requirement, determining that the business behavior detection model meets the debugging cut-off requirement.

In the embodiment of the present application, by loading a front business behavior data cluster T1 and a rear business behavior data cluster T2 of a simulated business data cluster set, actually loading T1 into Net1 to obtain T1', T2 and T1', performing MSE processing, so that the constructed T1 'and T2 are more similar, and loading T2 and T1' into a Classifier2 to determine whether the actual rear business behavior data cluster T2 is true or not.

Based on the above, the performance of the detection model on the context time sequence detection can be increased by adopting the above embodiment, so that the detection speed of the detection model on the service behavior is improved.

In an alternative embodiment, determining that the business behavior detection model meets the debug cutoff requirement may specifically include:

step S601, obtaining a first error result, a second error result, a fusion result of the first time difference and the second time difference.

For example, a corresponding weighting coefficient is given to each error result, and each error result is weighted and summed to obtain a fusion result. The numerical value of the weighting coefficient is not limited.

Step S602, when the fusion result characterization meets the target numerical result, determining that the business behavior detection model meets the debugging cut-off requirement.

And by deploying a target numerical result, the debugging performance is maximized, and the detection effect of the detection model obtained by debugging is improved.

In a specific embodiment, after the pre-trained business behavior detection model is debugged based on the simulated business data cluster and the actual business environment data cluster and the debugging is stopped when the debugging stop requirement is met, the method may further include:

Step S701, obtaining output information of a classification mapping network connected to the first building model and the second building model in the business behavior detection model meeting the debugging cut-off requirement.

In step S702, when the output information representation meets the detection requirement, determining that the model internal configuration variable of the first construction model is determined to be the service behavior detection variable, where the classification information corresponding to the detection requirement representation output information is the same as the classification indication information of the analog service data set matched with the output information.

It should be understood that, in the embodiment of the present application, one simulated service data cluster is loaded at a time, then one actual condition service data set and an actual condition image service data cluster are arbitrarily selected to form a data cluster, and the data cluster is loaded to a model for debugging, so as to obtain a model suitable for image, service behavior rationality and context time sequence limitation of the simulated service data cluster and the actual condition service data cluster, where the model includes a second construction model Net2, and a model internal configuration variable in the model is used as a service behavior detection variable, and service behavior detection is performed based on the model including the second construction model Net 2.

By adopting the process, the business data cluster to be analyzed is obtained, the business behavior representation vector of the target business type in the business data cluster is extracted, the probability parameter of the target business type corresponding to each business behavior result classification is determined based on the business behavior representation vector and the business behavior detection variable, wherein the business behavior detection variable is a neural network parameter obtained after the debugging is carried out based on the simulated business data cluster and the actual business environment data cluster, the simulated business data cluster comprises a data cluster for simulating business behaviors generated by the business types, and the business behavior result classification matched with the target business type in the business data cluster is determined based on the probability parameter. Based on the method, the business behavior detection model is debugged through the simulated data cluster, and the business behavior detection model is debugged without collecting mass enterprise data templates which are actually generated, so that the model training cost of the business behavior detection is saved, and the efficiency is improved.

The method for debugging the business behavior detection model comprises the following steps:

and step T100, acquiring a simulated service data cluster and an actual service environment data cluster.

The simulated service data clusters comprise data clusters for generating service behaviors by simulated service types in the simulated service data sets, the actual service environment data clusters comprise actual service environment portrait data clusters and actual service environment service behavior data clusters, the simulated service data clusters are obtained by generating a plurality of simulated service data sets, each simulated service data set carries classification indication information, and the classification indication information comprises first training sample classification for indicating that the simulated service data set is the target classified service behavior and second training sample classification for indicating that the simulated service data set is not the target classified service behavior.

And step T200, debugging the pre-trained business behavior detection model based on the simulated business data cluster and the actual business environment data cluster, and stopping debugging when the debugging cut-off requirement is met, wherein the pre-trained business behavior detection model comprises a first construction model for converting the actual business environment data cluster into the simulated business environment data cluster, a second construction model for converting the simulated business environment data cluster into the actual business environment data cluster and a plurality of debugging analysis models, and when the debugging cut-off requirement is met, the model internal configuration variable of the first construction model is determined to be the business behavior detection variable.

The model debugging method comprises two links of data preprocessing and image data environment consistency debugging. The acquired data clusters comprise a simulated business data cluster set covering business behaviors, an actual situation business behavior data cluster set based on an actual business behavior portrait, and a data cluster set providing an actual situation portrait, and the specific process is as described above.

According to the embodiment of the application, the simulated service data cluster and the actual service environment data cluster are obtained, the pre-trained service behavior detection model is debugged based on the simulated service data cluster and the actual service environment data cluster, and the debugging is stopped when the debugging stop requirement is met, so that a network structure capable of being used for service behavior recognition is obtained by debugging, a large number of actually generated enterprise data templates are not required to be collected for debugging the service behavior detection model, the model training cost of service behavior detection is saved, and the efficiency is improved.

In a specific embodiment, the method comprises the following steps:

step T101, acquiring a service data cluster;

and step T102, extracting a service behavior characterization vector.

And step T103, analyzing the extracted service behavior characterization vector based on the detection model.

And step T104, when the detection requirement is met, confirming that the service data cluster is a service data cluster of a preset type.

For steps T101 to T104, when the service data cluster to be analyzed of the user terminal is obtained, extracting a service behavior representation vector corresponding to the target service type in the data cluster to be analyzed, loading the service behavior representation vector into a model comprising a model Net2, and obtaining probability parameters through a service behavior detection variable obtained based on debugging in the model. It should be understood that the detection model in the above step T103 is obtained by debugging in the step T105 and the step T106.

Step T105, obtaining the debug data cluster.

The debug data cluster may refer to the data cluster mentioned in step S102.

And step T106, obtaining a detection model based on the same thought debugging in the embodiment.

When the embodiment of the application is used for training the debugging model, the business data cluster to be analyzed is obtained; extracting service behavior characterization vectors of target service types in service data clusters; determining probability parameters of target service types corresponding to classification of service behavior results based on service behavior characterization vectors and service behavior detection variables, wherein the service behavior detection variables are neural network parameters obtained after debugging based on simulated service data clusters and actual service environment data clusters, and the simulated service data clusters comprise data clusters for generating service behaviors by simulating service types; and determining the business behavior result classification matched with the target business type in the business data cluster based on the probability parameter. The business behavior detection model is debugged through the simulated data cluster, and a large number of actually generated enterprise data templates are not required to be collected to debug the business behavior detection model, so that the model training cost of the business behavior detection is saved, and the efficiency is improved.

It should be noted that although the steps of the methods of the present application are depicted in the accompanying drawings in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.

The following describes an embodiment of the apparatus of the present application, which may be used to perform the enterprise data visualization implementation method based on big data analysis in the above embodiment of the present application. Fig. 2 schematically illustrates a block diagram of an enterprise data visualization implementation apparatus provided by an embodiment of the present application. As shown in fig. 2, the enterprise data visualization implementation apparatus 200 includes:

a data acquisition module 210, configured to acquire an enterprise service data set, where the enterprise service data set includes a plurality of service data clusters to be analyzed; acquiring each business data cluster to be analyzed;

the feature extraction module 220 is configured to extract a service behavior characterization vector of a target service type in the service data cluster;

the probability prediction module 230 is configured to determine probability parameters of the target service type corresponding to each service behavior result classification based on the service behavior characterization vector and the service behavior detection variable; the service behavior detection variable is a neural network parameter obtained after debugging based on a simulated service data cluster and an actual service environment data cluster, wherein the simulated service data cluster comprises data cluster portrait environment service behaviors for simulating service behaviors generated by service types;

A behavior classification module 240, configured to determine a classification of service behavior results matched with a target service type in the service data cluster based on the probability parameter;

the classification statistics module 250 is configured to count at least one classification of business behavior results corresponding to each business data cluster in the enterprise business data set, so as to obtain a classification statistics result; the classification statistical result comprises the quantity corresponding to each business behavior result classification;

and the visual display module 260 is configured to generate visual display information according to the classification statistics result, and display the visual display information on the data visual system.

Specific details of the enterprise data visualization implementation apparatus provided in each embodiment of the present application have been described in the corresponding method embodiments, and are not described herein.

FIG. 3 schematically illustrates a block diagram of a computer system architecture for implementing a data visualization system in accordance with an embodiment of the present application.

It should be noted that, the computer system 300 of the data visualization system shown in fig. 3 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.

As shown in fig. 3, the computer system 300 includes a central processing unit 301 (Central Processing Unit, CPU) that can perform various appropriate actions and processes according to a program stored in a Read-Only Memory 302 (ROM) or a program loaded from a storage section 308 into a random access Memory 303 (Random Access Memory, RAM). In the random access memory 303, various programs and data required for the system operation are also stored. The central processing unit 301, the read only memory 302, and the random access memory 303 are connected to each other via a bus 304. An Input/Output interface 305 (i.e., an I/O interface) is also connected to bus 304.

The following components are connected to the input/output interface 305: an input section 306 including a keyboard, a mouse, and the like; an output portion 307 including a Cathode Ray Tube (CRT), a liquid crystal display (Liquid Crystal Display, LCD), and the like, a speaker, and the like; a storage section 308 including a hard disk or the like; and a communication section 309 including a network interface card such as a local area network card, a modem, or the like. The communication section 309 performs communication processing via a network such as the internet. The driver 310 is also connected to the input/output interface 305 as needed. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.

In particular, the processes described in the various method flowcharts may be implemented as computer software programs according to embodiments of the application. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 309, and/or installed from the removable medium 311. The computer program, when executed by the central processor 301, performs the various functions defined in the system of the present application.

It should be noted that, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains.

It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method for implementing enterprise data visualization based on big data analysis, the method being applied to a data visualization system, the method comprising:

acquiring each business data cluster to be analyzed;

2. The method of claim 1, wherein prior to acquiring the traffic data cluster to be analyzed, the method further comprises:

3. The method of claim 2, wherein the debugging the pre-trained business behavior detection model based on the simulated business data cluster and the actual business environment data cluster, stopping the debugging when a debugging cutoff requirement is met comprises:

4. The method of claim 2, wherein loading the simulated service data cluster into the second build model to build a first comparative actual service environment data cluster comprises:

5. The method of claim 3 or 4, wherein loading the simulated service data cluster into the second build model to build a first comparative actual service environment data cluster comprises: loading business behavior context evolution information corresponding to the plurality of simulated business data sets in the simulated business data cluster to the second construction model to obtain an actual business environment data cluster for comparison with the business behavior context;

6. The method of claim 3, wherein the debugging the pre-trained business behavior detection model based on the simulated business data cluster and the actual business environment data cluster, stopping the debugging when a debugging cutoff requirement is met further comprises:

7. The method of claim 6, wherein determining that the business behavior detection model meets the debug cutoff requirement comprises:

8. The method of claim 7, wherein after the pre-trained business behavior detection model is debugged based on the simulated business data cluster and the actual business environment data cluster, stopping debugging when a debugging cutoff requirement is met, further comprising:

9. The training method of the business behavior detection model is characterized by comprising the following steps of:

10. A data visualization system, comprising:

a processor;

and a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of any one of claims 1 to 8 via execution of the executable instructions.