CN113934769A - Intelligent data analysis method and device - Google Patents

Intelligent data analysis method and device Download PDF

Info

Publication number
CN113934769A
CN113934769A CN202111064062.5A CN202111064062A CN113934769A CN 113934769 A CN113934769 A CN 113934769A CN 202111064062 A CN202111064062 A CN 202111064062A CN 113934769 A CN113934769 A CN 113934769A
Authority
CN
China
Prior art keywords
data
analysis
data analysis
result
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111064062.5A
Other languages
Chinese (zh)
Inventor
赵铭
易文峰
杨育
李小芬
杨正刚
吴兰兰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Digital Power Grid Research Institute of China Southern Power Grid Co Ltd
Original Assignee
Shenzhen Digital Power Grid Research Institute of China Southern Power Grid Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Digital Power Grid Research Institute of China Southern Power Grid Co Ltd filed Critical Shenzhen Digital Power Grid Research Institute of China Southern Power Grid Co Ltd
Priority to CN202111064062.5A priority Critical patent/CN113934769A/en
Publication of CN113934769A publication Critical patent/CN113934769A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2465—Query processing support for facilitating data mining operations in structured databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2457—Query processing with adaptation to user needs

Abstract

The invention discloses an intelligent analysis method and device of data, wherein the method comprises the following steps: the data analysis system receives information of a data set sent by a data end which establishes communication connection with the data analysis system; the data analysis system carries out data preprocessing operation on the information of the data set to obtain a preprocessing result of the information of the data set; and the data analysis system inputs the preprocessing result into the data analysis model to perform data analysis operation, and obtains the analysis result of the data analysis model as the analysis result of the data set. Therefore, the data analysis method and the data analysis system can perform data analysis on a large amount of data in static modes and dynamic modes in a cross-system mode, visually display data analysis results, evaluate the data analysis results more intuitively, and are favorable for improving the analysis speed and accuracy of the data analysis.

Description

Intelligent data analysis method and device
Technical Field
The invention relates to the technical field of data analysis, in particular to an intelligent data analysis method and device.
Background
With the rapid development of the mobile internet and the internet of things, the total amount of data is increased explosively, and big data gradually becomes a strong driving force for enterprise development. The quality improvement of enterprise products and services depends on the development and application of big data, and the effective data analysis is helpful for innovating the business model of the enterprise, so that the core competitiveness of the enterprise is improved.
At present, a traditional data analysis system generally acquires data according to fixed requirements of users, and then performs data statistics by using classification and summarization. The traditional data analysis system can only process and analyze enterprise structured and relational data, but is difficult to rapidly analyze the characteristics, distribution conditions and other relations of user data in the process of developing enterprises by using massive dynamic data in various forms, so that the future dynamic requirements of users cannot be directly estimated. Therefore, it is important to improve the dynamic analysis capability and to realize a faster data analysis method.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide an intelligent data analysis method and apparatus, which can perform data analysis on a large amount of dynamic data across systems, visually display the data analysis result, more intuitively evaluate the data analysis result, and improve the analysis speed during data analysis.
In order to solve the above technical problem, a first aspect of the present invention discloses a method for intelligently analyzing data, including:
the data analysis system receives information of a data set sent by a data terminal which establishes communication connection with the data analysis system; the information of the data set is sent by the data terminal after screening basic data, the data set comprises a preset amount of data, the information of the data set comprises the information of all the data, and the mode of the information of the data set is a static mode and/or a dynamic mode;
the data analysis system carries out data preprocessing operation on the information of the data set to obtain a preprocessing result of the information of the data set;
and the data analysis system inputs the preprocessing result into a data analysis model to perform data analysis operation, and obtains an analysis result of the data analysis model as an analysis result of the data set.
As an optional implementation manner, in the first aspect of the present invention, the information of the data set includes at least one of source information of all the data, a programming language type of all the data, name information of all the data, and a data type of all the data, and the name information of each of the data includes a chinese name and/or an english name.
As an optional implementation manner, in the first aspect of the present invention, the inputting, by the data analysis system, the preprocessing result into a data analysis model to perform a data analysis operation, and obtaining an analysis result of the data analysis model as an analysis result of the data set, includes:
and the data analysis system calls algorithm configuration matched with the preprocessing result in a data analysis model according to the preprocessing result, and performs data rewriting operation on the preprocessing result according to the algorithm configuration to obtain an analysis result of the data analysis model as an analysis result of the data set.
As an optional implementation manner, in the first aspect of the present invention, after the data analysis system inputs the preprocessing result into a data analysis model to perform a data analysis operation, and obtains an analysis result of the data analysis model as an analysis result of the data set, the method further includes:
the data analysis system sends the analysis result to the data end to trigger the data end to execute matched operation on the analysis result;
and, the method further comprises:
and the data analysis system carries out evaluation operation on the data analysis model to obtain evaluation information of the data analysis model.
As an optional implementation manner, in the first aspect of the present invention, the evaluation information includes at least one of an accuracy of the data analysis model, a recall rate of the data analysis model, a variance ratio check value of the data analysis model, a significance check value of the data analysis model, and a correlation coefficient of the data analysis model.
As an optional implementation manner, in the first aspect of the present invention, the sending, by the data analysis system, the analysis result to the data end to trigger the data end to perform a matching operation on the analysis result, includes:
storing the analysis result to a matched storage position by the data terminal; the matched storage position comprises at least one of a memory, a file and a database;
and/or the presence of a gas in the gas,
configuring a designer for the analysis result by the data terminal according to the analysis result to obtain a display chart of the analysis result; the display chart of the analysis result is a static result display chart and/or a dynamic result display chart.
As an optional implementation manner, in the first aspect of the present invention, after the data analysis system performs an evaluation operation on the data analysis model to obtain evaluation information of the data analysis model, the method further includes:
the data analysis system detects whether a viewing instruction aiming at the evaluation information sent by the data terminal is received;
and when the receiving of the viewing instruction aiming at the evaluation information is detected, the data analysis system sends the evaluation information to the data terminal.
The second aspect of the present invention discloses an intelligent data analysis device, which comprises:
the data receiving module is used for receiving information of a data set sent by a data end which establishes communication connection with the data analysis system; the information of the data set is sent by the data terminal after screening basic data, the data set comprises a preset amount of data, the information of the data set comprises the information of all the data, and the mode of the information of the data set is a static mode and/or a dynamic mode;
the data preprocessing module is used for carrying out data preprocessing operation on the information of the data set to obtain a preprocessing result of the information of the data set;
and the data analysis module is used for inputting the preprocessing result into a data analysis model to perform data analysis operation, and obtaining an analysis result of the data analysis model as an analysis result of the data set.
As an optional implementation manner, in the second aspect of the present invention, the information of the data set includes at least one of source information of all the data, a programming language type of all the data, name information of all the data, and a data type of all the data, and the name information of each of the data includes a chinese name and/or an english name.
As an optional implementation manner, in the second aspect of the present invention, the data analysis module inputs the preprocessing result into a data analysis model to perform a data analysis operation, and a manner of obtaining an analysis result of the data analysis model as an analysis result of the data set is specifically:
and calling an algorithm configuration matched with the preprocessing result in a data analysis model according to the preprocessing result, and performing data rewriting operation on the preprocessing result according to the algorithm configuration to obtain an analysis result of the data analysis model as an analysis result of the data set.
As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:
the data sending module is used for sending the analysis result to the data terminal after the data analysis module inputs the preprocessing result into a data analysis model to execute data analysis operation and obtains the analysis result of the data analysis model as the analysis result of the data set, so as to trigger the data terminal to execute matched operation on the analysis result;
and, the apparatus further comprises:
and the model evaluation module is used for carrying out evaluation operation on the data analysis model to obtain evaluation information of the data analysis model.
As an optional implementation manner, in the second aspect of the present invention, the evaluation information includes at least one of an accuracy of the data analysis model, a recall rate of the data analysis model, a variance ratio check value of the data analysis model, a significance check value of the data analysis model, and a correlation coefficient of the data analysis model.
As an optional implementation manner, in the second aspect of the present invention, the manner in which the data sending module sends the analysis result to the data end to trigger the data end to execute the matched operation on the analysis result specifically is:
storing the analysis result to a matched storage position by the data terminal; the matched storage position comprises at least one of a memory, a file and a database;
and/or the presence of a gas in the gas,
configuring a designer for the analysis result by the data terminal according to the analysis result to obtain a display chart of the analysis result; the display chart of the analysis result is a static result display chart and/or a dynamic result display chart.
As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:
the detection module is used for detecting whether a viewing instruction aiming at the evaluation information sent by the data terminal is received or not after the model evaluation module carries out evaluation operation on the data analysis model to obtain the evaluation information of the data analysis model;
and the evaluation information sending module is used for sending the evaluation information to the data terminal when the detection module detects that a viewing instruction aiming at the evaluation information is received.
The third aspect of the present invention discloses another intelligent data analysis device, which includes:
a memory storing executable program code;
a processor coupled with the memory;
the processor calls the executable program codes stored in the memory to execute the intelligent analysis method of the data disclosed by the first aspect of the invention.
In a fourth aspect, the present invention discloses a computer-readable storage medium storing computer instructions for executing the intelligent data analysis method disclosed in the first aspect of the present invention when the computer instructions are called.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, a large amount of static and dynamic data are subjected to data analysis in a cross-system manner, and the data analysis result is displayed visually, so that the data analysis result is evaluated more intuitively. Therefore, the dynamic demand change of the massive dynamic data in various forms in enterprise development can be adapted, and the future dynamic demand of the enterprise user can be directly and accurately presumed. Therefore, the dynamic analysis capability during data analysis can be improved by implementing the method, so that the analysis speed and the accuracy of the data analysis are improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a method for intelligent analysis of data according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart diagram of another method for intelligent analysis of data according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of an intelligent data analysis device disclosed in an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of another intelligent data analysis device disclosed in the embodiment of the present invention;
fig. 5 is a schematic structural diagram of another intelligent data analysis device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," and the like in the description and claims of the present invention and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, article, or article that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or article.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The invention discloses an intelligent data analysis method and device, which can perform data analysis on a large amount of dynamic data in a cross-system manner, visually display data analysis results, evaluate the data analysis results more intuitively, improve the dynamic data analysis capability and further improve the analysis speed of data analysis. The following are detailed below.
Example one
Referring to fig. 1, fig. 1 is a schematic flow chart of an intelligent data analysis method according to an embodiment of the present invention. The intelligent analysis method of data described in fig. 1 may be applied to different business fields in an enterprise, and may be at least one of internet, electronic commerce, financial insurance, biomedical science, manufacturing, food and beverage takeout, sports and entertainment, transportation and logistics, and the like. Specifically, the method can be applied to strategic planning in strategic planning departments of enterprises, and can also be applied to marketing departments of the enterprises to investigate the enthusiasm of users on enterprise products (for example, collecting customer information and corresponding commodity purchasing information, and analyzing the preference value of the current customers for the commodities to be estimated). Optionally, the data analysis may be at least one of enterprise operation management process data analysis, task management and plan management data analysis, organization efficiency data analysis, process optimization and value output data analysis, industry research data analysis, product test data analysis, channel data analysis, price and sales forecast data analysis, project management data analysis, engineering management data analysis, establishment of a whole organization operation index system, marketing expense control data analysis, advertisement effect evaluation data analysis, public praise and public opinion monitoring data analysis, and media expense optimization analysis, which is not limited in the embodiments of the present invention. Specifically, the method may be implemented by a data analysis device, and further optionally, the data analysis device may be integrated in a data analysis device, or may be a local server or a cloud server for managing a data analysis process, and the embodiment of the present invention is not limited. As shown in fig. 1, the intelligent analysis method of data may include the following operations:
101. and the data analysis system receives the information of the data set sent by the data terminal which establishes communication connection with the data analysis system.
In the embodiment of the invention, the data analysis system establishes communication connection with the data end and receives the information of the data set acquired by the database in the data end, namely, the data analysis system can configure and manage the data end, so that the information of the data set is received from the database of the data end. Optionally, the data analysis system may directly store information of the data set in the database sent by the data terminal, so as to facilitate the invocation of data during subsequent analysis operations. Further optionally, the database in the data side may be a conventional relational database, that is, a table manner defined for the field is adopted for storage, the data therein is stored in the table in a row and column manner, and the data mode therein is static; the database in the data end may also be a non-relational database, that is, the stored data structure is irregular, incomplete or not predefined, such as storing documents, various reports, image information, etc., wherein the data mode is dynamic, and the information of the data set is updated in real time. Therefore, when the data analysis system is used for data acquisition, a large amount of data in a static mode can be imported, and a large amount of dynamic data can be captured to be processed by data mining and various data analysis in the subsequent steps. Based on the storage mode of the database data in the data terminal, the data analysis system can receive the information of the data set in the static mode and can also receive the information of the data set in the dynamic mode.
Further, the information of the data set is sent by the data side after the data screening operation is performed on the basic data. Optionally, the information of the data set may include at least one of source information of the data, a programming language type of the data, name information of the data, and a data type of the data. Specifically, the source information of the data may be connection information of each database in the data terminal, the data name information may be a chinese name or an english name, the programming language type of the data may be structured query language information of the data and unstructured query language information of the data, where the structured query language information of the data may be at least one of data update, data insertion, data deletion, and data selection, the unstructured query language information of the data may be at least one of data update, data insertion, data deletion, and data selection, and the data type of the data may be at least one of an integer type, a floating point type, a character type, and a boolean type. Therefore, aiming at the information of the structured and unstructured data sets in the enterprise, the enterprise data sets can be constructed according to business classification, each business data set can be established, the main data standard is unified, and the data source and the data standard are unified for the application of the information of the data sets.
Still further, the data screening operation may be implemented by data extraction or data cleaning. Specifically, data extraction can be realized by transferring data in different data sources into a database according to the data format of the database, or converting data files in the database into text files in a specified format, importing the converted database files into the specified database, extracting required data, and performing data cleaning to process missing data, repeated data and abnormal data, remove redundant data and data with high distinctiveness, and finally loading the data into the database. It should be noted that, loading data into the database may be incremental loading, that is, the target table only updates data changed by the source table, or may be full loading, that is, data loading is performed after the full table is deleted, according to the data size, system resources, and the real-time requirement of the data. Therefore, the screened data can improve the usability of the database for collecting and storing the historical data, and the accuracy of later-period data analysis is improved.
Still further, the basic data includes, but is not limited to, financial data, asset data, sales data, enterprise-owned data, market information data where the enterprise is a market subject, public policy information data, industry criteria data, expert experience data, online operation log data, internal control flow data, warehousing data, and the like of the enterprise. Further optionally, the data format of the basic data may be at least one of a table format, a key-value pair format, a document format, a picture format, an audio format, and a video format, which is not limited in the embodiment of the present invention. Therefore, the intelligent analysis method of the data can penetrate into each enterprise and each process, and can adapt to the change of the requirements in the production service process of the actual enterprise.
102. And the data analysis system carries out data preprocessing operation on the information of the data set to obtain a preprocessing result of the information of the data set.
In the embodiment of the invention, after the information of the data set is screened and sent to the data analysis system through the data end, the data analysis system can carry out data preprocessing operation on the information of the data set, and directly obtain the preprocessing result of the information of the data set, so that the preprocessing result of the information of the data set can be smoothly operated in a computer. Optionally, the data pre-processing operation may be, but is not limited to, data cleansing, data integration, data transformation, and data reduction. Specifically, data cleaning can be realized by filling in missing data values, smoothing noise data, identifying or deleting outlier cleaning data, standardizing data formats, and removing abnormal and repeated data; the data integration can combine and uniformly store the data of the data sources; the data transformation can convert the data form by means of smooth aggregation, generalization, normalization and the like; the data specification can be realized by data screening and data dimension reduction. It should be noted that after the information of the data set is preprocessed, the preprocessing result obtained by the data analysis system can maintain accuracy, integrity, consistency and timeliness with the information of the data set before processing. Therefore, after the information of the data set is preprocessed, the data analysis system can obtain a standard and continuous preprocessing result, the accuracy of a subsequent analysis result is improved, and the calculation process is shortened.
103. And the data analysis system inputs the preprocessing result into the data analysis model to perform data analysis operation, and obtains the analysis result of the data analysis model as the analysis result of the data set.
In the embodiment of the invention, after the information of the data set is preprocessed by the data analysis system, the data analysis system can directly input the preprocessing result into the data analysis model to perform data analysis operation, and the analysis result of the data analysis model is obtained and used as the analysis result of the data set. Thus, the data analysis system can efficiently and pertinently continue data analysis work according to the preprocessing result, thereby saving unnecessary steps.
Optionally, the data analysis model may be sent to the data analysis system by the data terminal, or may be pre-stored by the data analysis system. Further optionally, the data analysis model may adopt an unsupervised learning model or a semi-supervised learning model. It should be noted that the unsupervised learning model is used when the intrinsic structure of the data needs to be known without explicitly providing a data tag, and the semi-supervised learning model is used when there is a large amount of unlabeled data and a small amount of labeled data. Therefore, different data analysis models can be adopted according to different collected data types during data analysis, and the reliability of data analysis results is improved.
Furthermore, the analysis result of the data analysis model may be directly regarded as the analysis result of the data set, and the form of the analysis result of the data analysis model may be converted into at least one of forms such as a table, a file, a graph, and an image through computer graphics, an image processing technology, and other office software (such as excel), which is not limited in the embodiment of the present invention. Specifically, the data analysis model and the analysis result of the data analysis model may be directly stored in the data analysis system, or may be sent to the data terminal by the data analysis system. Therefore, the two systems can realize information interaction, and the time for acquiring information can be saved when the data analysis model or the analysis result of the data analysis model needs to be called.
Therefore, the embodiment of the invention can receive a large amount of information of the data sets in static and dynamic modes in a cross-system manner, flexibly perform preprocessing operation on the received information of the data sets, and improve the processing capacity of the information of the data sets, thereby improving the accuracy and the analysis speed of the subsequent analysis on the information of the data sets.
In an alternative embodiment, the step 103 of inputting the preprocessing result into the data analysis model by the data analysis system to perform a data analysis operation, and obtaining an analysis result of the data analysis model as an analysis result of the data set may include:
and the data analysis system calls algorithm configuration matched with the preprocessing result in the data analysis model according to the preprocessing result, performs data rewriting operation on the data set, and obtains an analysis result of the data analysis model as an analysis result of the data set.
In this optional embodiment, when performing data analysis, the data analysis system may call the algorithm configuration in the model according to the requirement, perform data rewriting on the information of the preprocessed data set according to the algorithm configuration, and directly generate an analysis result of the data analysis model as an analysis result of the data set. Optionally, the algorithm configuration may be an algorithm type, may also be a parameter in the algorithm, and may also be a dependent variable and an independent variable used for the algorithm operation. Further, the algorithm type may be at least one of a Kmeans clustering algorithm, a PCA principal component analysis algorithm, a unary linear regression algorithm, and a multiple linear regression algorithm. It should be noted that Kmeans clustering is generally applied to grouping the same things from a randomly distributed collection of things; PCA principal component analysis is generally applied to synthesize a plurality of variables with a plurality of correlations into several representative variables, wherein the variables can represent vast amount of information of original variables and are not correlated with each other; unary and multiple linear regression are commonly used to predict a continuous numerical variable (dependent variable) from known variables (independent variables). For example, when an enterprise needs to improve customer group information and further details the customer category according to the purchase history, interest or activity monitoring of the customer, the information can be realized by adopting a Kmeans clustering algorithm, which is beneficial for a company to make a specific advertisement aiming at a specific customer group; when the catering enterprises need to predict the dining scale or the turnover according to daily business data (including menu prices, the number of people having meals, the number of reserved people, special dish discounts and the like), the method can be realized by adopting a unitary linear regression algorithm and a multiple linear regression algorithm.
Therefore, the optional embodiment can enable an enterprise to call different algorithm configurations in the model according to different requirements when performing data analysis, and can modify the algorithm configurations according to the data analysis requirements, so that a reliable data analysis result can be quickly obtained.
Example two
Referring to fig. 2, fig. 2 is a schematic flow chart of an intelligent data analysis method according to an embodiment of the present invention. The intelligent analysis method of data described in fig. 2 may be applied to different business fields in an enterprise, and may be at least one of internet, electronic commerce, financial insurance, biomedical science, manufacturing, food and beverage takeout, sports and entertainment, transportation and logistics, and the like. Specifically, the method can be applied to strategic planning in strategic planning departments of enterprises, and can also be applied to marketing departments of the enterprises to investigate the enthusiasm of users on enterprise products. Optionally, the data analysis may be at least one of enterprise operation management process data analysis, task management and plan management data analysis, organization efficiency data analysis, process optimization and value output data analysis, industry research data analysis, product test data analysis, channel data analysis, price and sales forecast data analysis, project management data analysis, engineering management data analysis, establishment of a whole organization operation index system, marketing expense control data analysis, advertisement effect evaluation data analysis, public praise and public opinion monitoring data analysis, and media expense optimization analysis, which is not limited in the embodiments of the present invention. Specifically, the method may be implemented by a data analysis device, and further optionally, the data analysis device may be integrated in a data analysis device, or may be a local server or a cloud server for managing a data analysis process, and the embodiment of the present invention is not limited. As shown in fig. 2, the intelligent analysis method of data may include the following operations:
201. and the data analysis system receives the information of the data set sent by the data terminal which establishes communication connection with the data analysis system.
202. And the data analysis system carries out data preprocessing operation on the information of the data set to obtain a preprocessing result of the information of the data set.
203. And the data analysis system inputs the preprocessing result into the data analysis model to perform data analysis operation, and obtains the analysis result of the data analysis model as the analysis result of the data set.
In the embodiment of the present invention, for other descriptions of steps 201 to 203, please refer to the detailed description of steps 101 to 103 in the first embodiment, which is not repeated herein.
204. And the data analysis system sends the analysis result to the data end to trigger the data end to execute matched operation on the analysis result.
In the embodiment of the invention, after the data analysis system sends the analysis result to the data end, the data end executes the matched operation on the data analysis result. Optionally, the data end may perform a storage operation on the analysis result, and the data end may perform a result display operation on the analysis result. Further, the analysis result may be in the form of at least one of a table, a file, a graph, an image, and the like. Still further, the data analysis system may also directly store the analysis results to the matching storage location. Specifically, the data analysis system may store the analysis result in at least one of a file, a memory, and a database (e.g., a cloud database, a local database) at a matching storage location.
Therefore, the embodiment of the invention can not only quickly obtain reliable data analysis results, but also improve the storage capacity of different types of analysis results, so that different types of analysis results can be flexibly called from a data analysis system or a data terminal when needed.
In an optional embodiment, the method may further comprise the operations of:
205. and the data analysis system carries out evaluation operation on the data analysis model to obtain evaluation information of the data analysis model.
In this alternative embodiment, the data analysis system can verify the utility of the data analysis model in all dimensions through evaluation operations on the data analysis model. Optionally, the data analysis system may perform an evaluation operation on the unsupervised learning model, may also perform an evaluation operation on the semi-supervised learning model, and may obtain evaluation information of the data analysis model in a targeted manner. Further, the evaluation information of the data analysis model may be at least one of an accuracy of the data analysis model, a recall of the data analysis model, a variance ratio check value of the data analysis model, a significance check value of the data analysis model, and a correlation coefficient of the data analysis model. For example, when an enterprise needs to perform proportional calculation between correctly classified sample numbers and total sample numbers in a data analysis model on a given test data set, the accuracy of the data analysis model can be used for evaluation; when an enterprise needs an index to comprehensively consider the accuracy and the recall rate according to the requirement of a data analysis result, evaluation can be carried out according to the variance ratio test value of the data analysis model, and the weights of the accuracy and the recall rate are calculated.
Therefore, according to different requirements of an enterprise, the optional embodiment can flexibly adopt the evaluation information of different data analysis models to give evaluation indexes in different directions according to the practicability of the data analysis models, so that the data analysis requirements of different businesses in the enterprise are met.
In yet another alternative embodiment, the sending, by the data analysis system in step 204, the analysis result to the data end to trigger the data end to perform a matching operation on the analysis result may include:
and storing the analysis result to the matched storage position by the data terminal.
In this alternative embodiment, the analysis result obtained by the data analysis system may be sent to the data end, and the data end performs other matching operations on the received analysis result. Optionally, the data end may perform a storage operation on the analysis result, and store the analysis result in at least one matching storage location of the memory, the file, and the database. Furthermore, the analysis result can be stored by the mobile terminal, the local server and the cloud server. Still further, the data storage type of the analysis result may be at least one of an integer type, a floating point type, a character type, and a boolean type.
Therefore, the optional embodiment reflects the diversification of the data analysis result storage mode, can flexibly store various types of data analysis results even in unlimited occasions, and is beneficial to accelerating the speed of acquiring the data analysis results when an enterprise needs the data analysis results, so that the working efficiency of the enterprise is accelerated.
In yet another alternative embodiment, the sending, by the data analysis system in step 204, the analysis result to the data end to trigger the data end to perform a matching operation on the analysis result may include:
and the data end configures a designer for the analysis result according to the analysis result to obtain a display chart of the analysis result.
In this alternative embodiment, the analysis result obtained by the data analysis system may be sent to the data end, and the data end performs other matching operations on the received analysis result. Optionally, the data end may perform designer configuration on the analysis result, and perform result display operation. Specifically, the data terminal can configure the designer for the analysis result through Excel, a BI tool, Python and the like, and when the designer is configured, the analysis result can be selectively displayed according to the programming language type of the data set or according to different visual collocation components. Further, when the data end performs the result display operation, the display chart type of the analysis result may be a static display chart type, a dynamic display chart type, or a static and dynamic combined display chart type. The static presentation chart type can be at least one of charts such as a line chart, a bar chart, an area chart, a scatter chart, a radar chart, a biaxial chart and the like, and the dynamic presentation chart type can be at least one of a hotspot link, chart linkage, online analysis report editing, series dragging, video animation and the like. Therefore, various display methods of visual data analysis results can be provided, the system integration of data creation, analysis and display application is realized, and the seamless connection and interaction between a data analysis system and an application platform can be supported.
Therefore, the optional embodiment can selectively display the analysis result to be displayed by combining different visual components according to the data analysis requirements of the enterprise, so that the enterprise can more intuitively see the reliability of the analysis result, and the enterprise can make targeted judgment on the analysis result.
In yet another optional embodiment, after the data analysis system performs the evaluation operation on the data analysis model in step 205 to obtain the evaluation information of the data analysis model, the method further includes:
the data analysis system detects whether a checking instruction aiming at the evaluation information sent by the data terminal is received;
and when the receiving of the viewing instruction aiming at the evaluation information is detected, the data analysis system sends the evaluation information to the data terminal.
In this optional embodiment, after the data analysis system obtains the evaluation information of the data analysis model, optionally, the data analysis system may directly store the evaluation information in the file, the memory, and the database, or the data analysis system may first send the evaluation information to the data terminal and then store the evaluation information in the file, the memory, and the database. Further optionally, when the data analysis system detects that a viewing instruction for the evaluation information sent by the data end is received, the data analysis system may send at least one of an accuracy of a data analysis model, a recall rate of the data analysis model, a variance ratio check value of the data analysis model, a significance check value of the data analysis model, and a correlation coefficient of the data analysis model included in the evaluation information to the data end.
Therefore, the optional embodiment can provide the interaction method of the evaluation information of the cross-system data analysis model, which is not only beneficial for enterprises to flexibly call the evaluation information when evaluating the data analysis model, but also beneficial for the enterprises to make subsequent improvements on the data analysis model and the related algorithm in a targeted manner, thereby improving the reliability of the analysis result.
EXAMPLE III
Referring to fig. 3, fig. 3 is a schematic structural diagram of an intelligent data analysis device according to an embodiment of the present invention. The apparatus described in fig. 3 is used to implement data analysis requirements for different business processes in an enterprise, and optionally, the intelligent analysis apparatus for data may be integrated in a data analysis device, and may also be a local server or a cloud server for managing the data analysis processes, and the like. As shown in fig. 3, the intelligent analysis device for data may include:
the data receiving module 301 is configured to receive information of a data set sent by a data end that establishes a communication connection with a data analysis system; the information of the data set is sent after the data end screens basic data, the data set comprises a preset amount of data, the information of the data set comprises the information of all data, and the mode of the information of the data set is a static mode and/or a dynamic mode;
the data preprocessing module 302 is configured to perform data preprocessing operation on information of a data set to obtain a preprocessing result of the information of the data set;
and the data analysis module 303 is configured to input the preprocessing result into the data analysis model to perform a data analysis operation, and obtain an analysis result of the data analysis model as an analysis result of the data set.
Optionally, the information of the data set includes at least one of source information of all data, programming language type of all data, name information of all data, and data type of all data, and the name information of each data includes a chinese name and/or an english name.
It can be seen that the intelligent analysis device for implementing the data described in fig. 3 can receive a large amount of information of data sets in static and dynamic modes across systems, and can flexibly perform preprocessing operation on the received information of the data sets, thereby improving the processing capability of the information of the data sets, and further improving the accuracy and the analysis speed when the information of the data sets is analyzed subsequently.
In an optional embodiment, the manner in which the data analysis module 303 inputs the preprocessing result into the data analysis model to perform data analysis operation in the intelligent data analysis apparatus to obtain the analysis result of the data analysis model as the analysis result of the data set is specifically as follows:
and calling an algorithm configuration matched with the preprocessing result in the data analysis model according to the preprocessing result, and performing data rewriting operation on the preprocessing result according to the algorithm configuration to obtain an analysis result of the data analysis model as an analysis result of the data set.
It can be seen that the intelligent analysis device for implementing the data described in fig. 4 can call different algorithm configurations according to different requirements of an enterprise when analyzing data, and can modify the algorithm configurations according to the data analysis requirements, thereby obtaining a reliable data analysis result quickly.
In another optional embodiment, the intelligent analysis device for data may further include:
a data sending module 304, configured to send the analysis result to the data end after the data analysis module 303 performs the above-mentioned data analysis operation of inputting the preprocessing result into the data analysis model to obtain the analysis result of the data analysis model as the analysis result of the data set, so as to trigger the data end to perform a matching operation on the analysis result.
It can be seen that the intelligent analysis device implementing the data described in fig. 4 not only can quickly obtain reliable data analysis results, but also can improve the storage capacity of different types of analysis results, so that different types of analysis results can be flexibly called from the data analysis system or the data terminal when needed.
In yet another alternative embodiment, the intelligent analysis device for data may further include:
a model evaluation module 305, configured to perform an evaluation operation on the data analysis model after the data analysis module 303 performs the above-mentioned data analysis operation of inputting the preprocessing result into the data analysis model to obtain an analysis result of the data analysis model as an analysis result of the data set, so as to obtain evaluation information of the data analysis model.
Optionally, the evaluation information includes at least one of an accuracy of the data analysis model, a recall of the data analysis model, a variance ratio check value of the data analysis model, a significance check value of the data analysis model, and a correlation coefficient of the data analysis model.
It can be seen that, the intelligent analysis device for implementing the data described in fig. 4 can flexibly adopt the evaluation information of different data analysis models to give evaluation indexes of different orientations according to the practicability of the data analysis models according to different requirements of an enterprise, thereby meeting the data analysis requirements of different businesses in the enterprise.
In yet another optional embodiment, in the intelligent data analysis apparatus, the data sending module 304 sends the analysis result to the data end, and the manner of triggering the data end to perform the matching operation on the analysis result specifically is:
storing the analysis result to a matched storage position by the data terminal; the matched storage location comprises at least one of a memory, a file and a database.
Therefore, the intelligent analysis device for implementing the data described in fig. 4 embodies the diversification of the data analysis result storage mode, can flexibly store various types of data analysis results even in an unlimited place, and is beneficial to accelerating the speed of acquiring the data analysis results when an enterprise needs the data analysis results, thereby accelerating the working efficiency of the enterprise.
In yet another alternative embodiment, in the intelligent data analysis apparatus, the data sending module 304 sends the analysis result to the data end, so as to trigger the data end to perform a matching operation on the analysis result in a specific manner that:
the data end configures a designer for the analysis result according to the analysis result to obtain a display chart of the analysis result; the display chart of the analysis result is a static result display chart and/or a dynamic result display chart.
Therefore, the intelligent analysis device for implementing the data described in fig. 4 can selectively display the required analysis result according to the data analysis requirements of the enterprise by combining different visual components, which is not only beneficial for the enterprise to visually see the reliability of the analysis result, but also beneficial for the enterprise to make targeted judgment on the analysis result.
In yet another alternative embodiment, the intelligent analysis device for data may further include:
a detection module 306, configured to detect whether a check instruction for the evaluation information sent by the data end is received after the model evaluation module 305 performs the above evaluation operation on the data analysis model to obtain the evaluation information of the data analysis model;
an evaluation information sending module 307, configured to send the evaluation information to the data side when the detection module 306 detects that a viewing instruction for the evaluation information is received.
It can be seen that the intelligent analysis device implementing the data described in fig. 4 can transmit the evaluation information of the data analysis model across systems, which is not only beneficial for enterprises to flexibly retrieve the evaluation information when evaluating the data analysis model, but also beneficial for enterprises to make subsequent improvements on the data analysis model and the related algorithms thereof in a targeted manner, thereby improving the reliability of the analysis result.
Example four
Referring to fig. 5, fig. 5 is a schematic structural diagram of another apparatus for intelligently analyzing data according to an embodiment of the present invention. As shown in fig. 5, the apparatus for intelligent analysis of data may include:
a memory 401 storing executable program code;
a processor 402 coupled with the memory 401;
the processor 402 calls the executable program code stored in the memory 401 to execute the steps of the intelligent analysis method of data described in one or more embodiments of the present invention.
EXAMPLE five
The embodiment of the invention discloses a computer storage medium, which stores computer instructions, and the computer instructions are used for executing the steps in the intelligent data analysis method described in the first embodiment or the second embodiment of the invention when being called.
EXAMPLE six
An embodiment of the invention discloses a computer program product comprising a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the method for intelligent analysis of data described in the first or second embodiment.
The above-described embodiments of the apparatus are merely illustrative, and the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above detailed description of the embodiments, those skilled in the art will clearly understand that the embodiments may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. Based on such understanding, the above technical solutions may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, where the storage medium includes a Read-Only Memory (ROM), a Random Access Memory (RAM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), a One-time Programmable Read-Only Memory (OTPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc-Read-Only Memory (CD-ROM), or other disk memories, CD-ROMs, or other magnetic disks, A tape memory, or any other medium readable by a computer that can be used to carry or store data.
Finally, it should be noted that: the method and apparatus for intelligently analyzing data disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used for illustrating the technical solutions of the present invention, not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art; the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. An intelligent analysis method for data, which is applied to a data analysis system, and comprises the following steps:
the data analysis system receives information of a data set sent by a data terminal which establishes communication connection with the data analysis system; the information of the data set is sent by the data terminal after screening basic data, the data set comprises a preset amount of data, the information of the data set comprises the information of all the data, and the mode of the information of the data set is a static mode and/or a dynamic mode;
the data analysis system carries out data preprocessing operation on the information of the data set to obtain a preprocessing result of the information of the data set;
and the data analysis system inputs the preprocessing result into a data analysis model to perform data analysis operation, and obtains an analysis result of the data analysis model as an analysis result of the data set.
2. The method of claim 1, wherein the information of the data set comprises at least one of source information of all the data, programming language type of all the data, name information of all the data, and data type of all the data, and the name information of each data comprises a chinese name and/or an english name.
3. The intelligent data analysis method of claim 1, wherein the data analysis system inputs the preprocessing result into a data analysis model to perform a data analysis operation, and obtains an analysis result of the data analysis model as an analysis result of the data set, and the method comprises:
and the data analysis system calls algorithm configuration matched with the preprocessing result in a data analysis model according to the preprocessing result, and performs data rewriting operation on the preprocessing result according to the algorithm configuration to obtain an analysis result of the data analysis model as an analysis result of the data set.
4. The intelligent analysis method for data according to claim 1, wherein after the data analysis system inputs the preprocessing result into a data analysis model to perform a data analysis operation, and obtains an analysis result of the data analysis model as an analysis result of the data set, the method further comprises:
the data analysis system sends the analysis result to the data end to trigger the data end to execute matched operation on the analysis result;
and, the method further comprises:
and the data analysis system carries out evaluation operation on the data analysis model to obtain evaluation information of the data analysis model.
5. The intelligent analysis method for data according to claim 4, wherein the evaluation information comprises at least one of accuracy of the data analysis model, recall of the data analysis model, variance ratio test value of the data analysis model, significance test value of the data analysis model, and correlation coefficient of the data analysis model.
6. The intelligent data analysis method of claim 4, wherein the data analysis system sends the analysis result to the data end to trigger the data end to perform a matching operation on the analysis result, and the method comprises:
storing the analysis result to a matched storage position by the data terminal; the matched storage position comprises at least one of a memory, a file and a database;
and/or the presence of a gas in the gas,
configuring a designer for the analysis result by the data terminal according to the analysis result to obtain a display chart of the analysis result; the display chart of the analysis result is a static result display chart and/or a dynamic result display chart.
7. The intelligent analysis method for data according to claim 6, wherein after the data analysis system performs an evaluation operation on the data analysis model to obtain evaluation information of the data analysis model, the method further comprises:
the data analysis system detects whether a viewing instruction aiming at the evaluation information sent by the data terminal is received;
and when the receiving of the viewing instruction aiming at the evaluation information is detected, the data analysis system sends the evaluation information to the data terminal.
8. An intelligent data analysis device, which is applied to a data analysis system, and comprises:
the data receiving module is used for receiving information of a data set sent by a data end which establishes communication connection with the data analysis system; the information of the data set is sent by the data terminal after screening basic data, the data set comprises a preset amount of data, the information of the data set comprises the information of all the data, and the mode of the information of the data set is a static mode and/or a dynamic mode;
the data preprocessing module is used for carrying out data preprocessing operation on the information of the data set to obtain a preprocessing result of the information of the data set;
and the data analysis module is used for inputting the preprocessing result into a data analysis model to perform data analysis operation, and obtaining an analysis result of the data analysis model as an analysis result of the data set.
9. An apparatus for intelligent analysis of data, the apparatus comprising:
a memory storing executable program code;
a processor coupled with the memory;
the processor calls the executable program code stored in the memory to perform the intelligent analysis method of data according to any one of claims 1 to 7.
10. A computer-storable medium that stores computer instructions that, when invoked, perform a method for intelligent analysis of data according to any of claims 1-7.
CN202111064062.5A 2021-09-10 2021-09-10 Intelligent data analysis method and device Pending CN113934769A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111064062.5A CN113934769A (en) 2021-09-10 2021-09-10 Intelligent data analysis method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111064062.5A CN113934769A (en) 2021-09-10 2021-09-10 Intelligent data analysis method and device

Publications (1)

Publication Number Publication Date
CN113934769A true CN113934769A (en) 2022-01-14

Family

ID=79275411

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111064062.5A Pending CN113934769A (en) 2021-09-10 2021-09-10 Intelligent data analysis method and device

Country Status (1)

Country Link
CN (1) CN113934769A (en)

Similar Documents

Publication Publication Date Title
EP3121738A1 (en) Data storage extract, transform and load operations for entity and time-based record generation
US20190095507A1 (en) Systems and methods for autonomous data analysis
US7028036B2 (en) System and method for visualization of continuous attribute values
US20040177053A1 (en) Method and system for advanced scenario based alert generation and processing
US20150134401A1 (en) In-memory end-to-end process of predictive analytics
Al-Azmi Data, text and web mining for business intelligence: a survey
CN111401777B (en) Enterprise risk assessment method, enterprise risk assessment device, terminal equipment and storage medium
US11042525B2 (en) Extracting and labeling custom information from log messages
CN107491983A (en) A kind of wechat customer action feedback method, equipment and storage medium
CN112115363A (en) Recommendation method, computing device and storage medium
CN113157947A (en) Knowledge graph construction method, tool, device and server
CN111179051A (en) Financial target customer determination method and device and electronic equipment
US7580878B1 (en) Data fusion for automated business decisions
CN115545886A (en) Overdue risk identification method, overdue risk identification device, overdue risk identification equipment and storage medium
US20190026759A1 (en) System and method for universal data modeling
CN112631889A (en) Portrayal method, device and equipment for application system and readable storage medium
CN113934769A (en) Intelligent data analysis method and device
Das et al. A Review of Data Warehousing Using Feature Engineering
CN115563176A (en) Electronic commerce data processing system and method
CN115619245A (en) Portrait construction and classification method and system based on data dimension reduction method
CN115408236A (en) Log data auditing system, method, equipment and medium
CN114996579A (en) Information pushing method and device, electronic equipment and computer readable medium
Hanif Applications of data mining techniques for churn prediction and cross-selling in the telecommunications industry
CN113936183A (en) Data prediction method and device based on model training
CN111861537A (en) Operation analysis method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination