CN115374316A - Intelligent variable-scale data analysis method and system supported by real-time decision - Google Patents
Intelligent variable-scale data analysis method and system supported by real-time decision Download PDFInfo
- Publication number
- CN115374316A CN115374316A CN202211299589.0A CN202211299589A CN115374316A CN 115374316 A CN115374316 A CN 115374316A CN 202211299589 A CN202211299589 A CN 202211299589A CN 115374316 A CN115374316 A CN 115374316A
- Authority
- CN
- China
- Prior art keywords
- concept
- analysis
- data analysis
- data
- decision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
- G06N5/025—Extracting rules from data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0637—Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
Abstract
The invention provides an intelligent variable-scale data analysis method and system with real-time decision support, and relates to the technical field of intelligent data analysis. Obtaining a thought sequence through a concept pair in a thought concept diagram, calculating the similarity of the thought sequence, and aggregating the thought sequence by using a similarity result to form an analysis problem; and calculating and analyzing the similarity of the problems, aggregating the similarity analysis problems, and identifying the analysis subjects in the thought subject map. And determining the data analysis tasks contained in the analysis topics according to the structural morphological characteristics of different data analysis tasks. And establishing a multi-scale data model. And matching a metadata analysis algorithm according to the data analysis task judgment result, and performing variable-scale data analysis on the multi-scale data model. The method solves the technical problems of intelligent decision making such as analysis subject discovery, data analysis task judgment, data analysis hierarchical conversion and the like, improves the accuracy and efficiency of data analysis task judgment, and realizes real-time decision making support for cross-industry management decision making scenes.
Description
Technical Field
The invention relates to the technical field of intelligent data analysis, in particular to an intelligent variable-scale data analysis method and system with real-time decision support.
Background
Computer-aided decision-making is always taken as a key engine for promoting the efficient application of enterprise knowledge and intellectual capital, and real-time decision support is one of the core capabilities of artificial intelligence technology, and is widely applied to the construction of new-generation information infrastructures and the high-quality development of intelligent economy under the drive of assisted artificial intelligence. Although the existing data-driven decision analysis method can extract valuable information and knowledge from massive, multi-source and heterogeneous service data after a decision target of an actual service scene is definite, and assists a governance subject and a manager to make policy and tactics, due to the lack of an automatic and engineering identification method for initial decision requirements, the time for obtaining data mining results often lags behind the moment for proposing the decision requirements to a great extent, even an analyst must assign an analysis subject and a data analysis task, and the intelligence level of the decision process still needs to be improved urgently.
Document 1 (wangwai, decision support-oriented variable-scale clustering analysis technology [ D ], beijing university of science and technology, 2020) aims at promoting the engineering application process of data mining technology, targets common clustered data analysis tasks in decision analysis scenes, and establishes a variable-scale clustering analysis method system by simulating scale-transformation thinking characteristics of analysts in the decision process, so that an enterprise decision analysis system constructed by using the variable-scale clustering analysis method system has automatic execution capability. However, the existing variable-scale cluster analysis technology still has the defect that only a single data analysis task type (cluster analysis) can be supported, and particularly, an automatic and intelligent identification method which can match an analysis theme and an analysis task of a scale space (decision problem solving space) model of the technology is lacked aiming at a cross-industry real-time decision support scene.
Disclosure of Invention
The invention provides an intelligent variable-scale data analysis method and system with real-time decision support, aiming at the problems of intelligent decisions such as analysis subject discovery, data analysis task judgment, data analysis hierarchical conversion and the like in the prior art.
In order to solve the technical problems, the invention provides the following technical scheme:
in one aspect, an intelligent variable-scale data analysis method supported by real-time decision is provided, and the method is applied to electronic equipment and comprises the following steps:
s1: acquiring multi-source heterogeneous original data reflecting enterprise decision requirements;
s2: constructing a thinking concept graph model according to the obtained original data, and determining an analysis theme according to the thinking concept graph model;
the thinking conceptual diagram model is used for representing knowledge experience and business logic related to the decision-making problem; the thinking conceptual diagram model comprises: various aspects of the decision-making problem and the concept pair sets of various angles, and the structural connection between different concept pairs;
s3: identifying structural morphology features between different concept pairs, and judging data analysis tasks contained in each analysis topic according to the structural morphology features;
s4: obtaining a judgment result of the data analysis tasks, and respectively matching metadata analysis algorithms for the analysis tasks according to the judgment result; and performing variable-scale data analysis on the service data by adopting a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
Optionally, the concept pair is: the cognitive subject directly links the precursor concept to another subsequent concept through imagination or association to form a thinking basic information unit with a partial order relationship.
Optionally, in step S2, constructing a thinking concept graph model according to the obtained raw data, and determining an analysis subject according to the thinking concept graph model, including:
s21: generating a thought sequence by performing a forward expansion, a backward expansion, or a bidirectional expansion process on the concept pair according to the acquired data;
s22: connecting the thought sequences with the same concept starting point or terminal point to obtain a thought concept graph model;
s23: calculating the similarity between different thought sequences, and aggregating the analysis problems by using the similarity result of the thought sequences;
s24: and calculating the similarity among different analysis problems, and determining an analysis theme by using the result of the similarity of the analysis problems.
Optionally, step S2 further comprisesThe method comprises the following steps: calculating the concept level weight of each node in the thinking concept graph model
By weight of
Establishing a feature vector of each analysis topic, wherein the feature vector comprises the following steps: analysis problem feature decision threshold
Similarity threshold of thinking sequence
，
Analyzing a problem similarity threshold
，
。
Optionally, in step S3, identifying structural morphological features between different concept pairs, and determining a data analysis task included in each analysis topic according to the structural morphological features, includes:
s31: calculating the concept characteristics of each node according to the concept level weight of each concept node in the analysis subject;
s32: identifying nodes with larger concept feature values as concept centers, and judging data analysis tasks according to the out-degree and in-degree of each concept center node and the structural morphological characteristics of different data analysis tasks;
wherein, the out degree is the total number of nodes which can be directly reached by the concept center; the in-degree is the total number of nodes that can reach the concept center directly.
Optionally, step S3 further includes: incorporating excavation depth parameters
，
And the total number of concept nodes in the thought concept graph model
，
Determining the result of descending order of concept features, and arranging the result before
Determining each node as a concept center and determining an analysis task; wherein the data analysis task decision parameter is
，
。
Optionally, in step S32, identifying a node with a large value of the concept feature as a concept center, and performing data analysis task determination according to the degree of departure and degree of entry of each concept center node and structural morphological features of different data analysis tasks, including:
if the central divergence type sub-concept graph with the single concept center is identified, the concept center node is used as an object, all the associated nodes are used as attributes, a clustering type data analysis task is constructed, and concept type declarative knowledge hidden in the service data is obtained;
if two concepts with integral connection are identified, the two concept nodes are respectively used as objects, an association rule type data analysis task is constructed, and proposition type declarative knowledge hidden in service data is obtained;
if the central convergent type sub-concept graph with the single concept center is identified, the concept center node is used as a class label, each associated node is used as a decision attribute, a classification prediction type data analysis task is constructed, and the question network type declarative knowledge hidden in the service data is obtained.
Optionally, in step S4, the variable-scale data analysis includes:
and adopting a conservative scale transformation strategy to carry out variable scale data analysis, wherein the technical parameters are adapted to the algorithm parameters and threshold values of the selected metadata analysis algorithm.
In one aspect, an intelligent variable-scale data analysis system for real-time decision support is provided, and the system is applied to an electronic device, and includes:
the data acquisition module is used for acquiring multi-source heterogeneous original data reflecting enterprise decision requirements;
the data preprocessing module is used for constructing a thinking conceptual diagram model according to the acquired original data and determining an analysis theme according to the thinking conceptual diagram model;
the thinking conceptual diagram model is used for representing knowledge experience and business logic related to the decision-making problem; the thinking conceptual diagram model comprises: various aspects of the decision-making problem, and a set of concept pairs for various angles, and structural connections between different concept pairs;
the data analysis module is used for identifying structural morphological characteristics between different concept pairs and judging data analysis tasks contained in each analysis topic according to the structural morphological characteristics;
obtaining a judgment result of the data analysis tasks, and respectively matching metadata analysis algorithms for the analysis tasks according to the judgment result; and performing variable-scale data analysis on the service data by adopting a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
Optionally, the concept pair is: the cognitive subject directly links the precursor concept to another subsequent concept through imagination or association to form a thinking basic information unit with a partial order relationship.
In one aspect, an electronic device is provided, where the electronic device includes a processor and a memory, where the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned method for intelligent variable-scale data analysis with real-time decision support.
In one aspect, a computer-readable storage medium is provided, where at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned method for real-time decision-support intelligent variable-scale data analysis.
The technical scheme of the embodiment of the invention at least has the following beneficial effects:
in the scheme, (1) the analysis subject discovery method realizes the structural representation of decision requirements in the cross-industry field by providing a thinking concept graph model and a concept pair acquisition technology thereof, provides an automatic and engineering decision requirement identification method from a thinking sequence to an analysis problem to an analysis subject by simulating the thinking activity characteristics of an analyst in the decision process, and improves the cross-industry real-time decision support capability in the computer-aided decision field.
(2) The data analysis task judgment method realizes direct automatic judgment of types and data structure composition of various specific data analysis tasks contained in the analysis subject chart based on the characteristics of different data analysis tasks, expands the defect that the original variable-scale clustering analysis technology can only solve the single management decision scene suitable for the clustering analysis tasks, and improves the accuracy and efficiency of data analysis task judgment.
(3) The invention discloses an intelligent variable-scale data analysis method supported by real-time decision, which provides a uniform and complete characterization method for a solution space of a decision problem, and particularly comprises the steps of identifying all analysis topics in an initial thought concept graph based on the result of an analysis topic discovery method so as to obtain an analysis topic graph; determining all data analysis tasks contained in each analysis topic based on the result of the data analysis task judgment method, and further obtaining a structured concept network; a scale space modeling method based on variable scale data analysis obtains all observation scales of all attribute concept nodes in a structured concept network, which can participate in data analysis hierarchical transformation, and further obtains a concept space, thereby completing complete representation of a problem solution space of a decision problem.
(4) The technical parameters and the data processing system of the real-time decision-support intelligent variable-scale data analysis method solve the technical problems of intelligent decision-making of analysis subject discovery, data analysis task judgment and data analysis hierarchical conversion, so that the time for obtaining a data mining result does not need to lag behind the time for proposing a decision-making requirement, the data mining result can be executed in real time, an analyst does not need to subjectively determine an analysis subject and a data analysis task, and the overall intelligence level of a decision-making process is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flowchart of an intelligent variable-scale data analysis method for real-time decision support according to an embodiment of the present invention;
FIG. 2 is a flowchart of an intelligent variable-scale data analysis method for real-time decision support according to an embodiment of the present invention;
FIG. 3 is a block diagram of an intelligent scale-variable data analysis system with real-time decision support according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
To make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides an intelligent variable-scale data analysis method supported by real-time decision, which can be realized by electronic equipment, wherein the electronic equipment can be a terminal or a server. As shown in fig. 1, a flow chart of an intelligent variable-scale data analysis method with real-time decision support may include the following steps:
s101: acquiring multi-source heterogeneous original data reflecting enterprise decision requirements;
s102: constructing a thinking concept graph model according to the acquired original data, and determining an analysis theme according to the thinking concept graph model;
the thinking conceptual diagram model is used for representing knowledge experience and business logic related to decision problems; the thinking conceptual diagram model comprises: various aspects of the decision-making problem and the concept pair sets of various angles, and the structural connection between different concept pairs;
s103: identifying structural morphology features between different concept pairs, and judging data analysis tasks contained in each analysis topic according to the structural morphology features;
s104: obtaining a judgment result of the data analysis tasks, and respectively matching metadata analysis algorithms for the analysis tasks according to the judgment result; and performing variable-scale data analysis on the service data by adopting a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
Optionally, the concept pair is: the cognitive subject directly links the precursor concept to another subsequent concept through imagination or association to form a thinking basic information unit with a partial order relationship.
Optionally, in step S102, constructing a thinking concept graph model according to the acquired raw data, and determining an analysis subject according to the thinking concept graph model, including:
s121: generating a thought sequence by performing a forward extension, a backward extension or a bidirectional extension process on the concept pair according to the acquired data;
s122: connecting the thought sequences with the same concept starting point or terminal point to obtain a thought concept graph model;
s123: calculating the similarity between different thought sequences, and aggregating the analysis problems by using the similarity result of the thought sequences;
s124: and calculating the similarity among different analysis problems, and determining an analysis theme by using the result of the similarity of the analysis problems.
Optionally, step S102 further includes: computingConcept level weight of each node in thinking concept graph model
By weight of
Establishing a feature vector of each analysis topic, wherein the feature vector comprises the following steps: analysis problem feature decision threshold
Similarity threshold of thinking sequence
，
Analyzing a problem similarity threshold
，
。
Optionally, in step S103, identifying structural morphological features between different concept pairs, and determining a data analysis task included in each analysis topic according to the structural morphological features, includes:
s131: calculating the concept characteristics of each node according to the concept hierarchy weight of each concept node in the analysis theme;
s132: identifying nodes with larger concept feature values as concept centers, and judging data analysis tasks according to the out-degree and in-degree of each concept center node and the structural morphological characteristics of different data analysis tasks;
wherein, the out degree is the total number of nodes which can be directly reached by the concept center; the in degree is the total number of nodes that can directly reach the concept center.
Optionally, step S103 further includes: incorporating excavation depth parameters
，
And the total number of concept nodes in the thought concept graph model
，
Determining the result of descending order of concept features, and arranging the result before
Determining each node as a concept center and determining an analysis task; wherein the data analysis task decision parameter is
，
。
Optionally, in step S132, identifying a node with a large value of the concept feature as a concept center, and performing data analysis task determination according to the degree of departure and degree of entry of each concept center node and structural morphological features of different data analysis tasks, including:
if the central divergence type sub-concept graph with the single concept center is identified, the concept center node is used as an object, all the associated nodes are used as attributes, a clustering type data analysis task is constructed, and concept type declarative knowledge hidden in the service data is obtained;
if two concepts with integral connection are identified, the two concept nodes are respectively used as objects, an association rule type data analysis task is constructed, and proposition type declarative knowledge hidden in service data is obtained;
if the central convergent type sub-concept graph with the single concept center is identified, the concept center node is used as a class label, each associated node is used as a decision attribute, a classification prediction type data analysis task is constructed, and the question network type declarative knowledge hidden in the service data is obtained.
Optionally, in step S104, the variable-scale data analysis includes:
and performing variable-scale data analysis by adopting a conservative scale transformation strategy, wherein the technical parameters are adapted to the algorithm parameters and the threshold value of the selected metadata analysis algorithm.
In the embodiment of the invention, the technical problems of intelligent decision making such as analysis subject discovery, data analysis task judgment, data analysis hierarchical conversion and the like are solved, the accuracy and the efficiency of data analysis task judgment are improved, and the real-time decision support for cross-industry management decision scenes is realized.
The embodiment of the invention provides an intelligent variable-scale data analysis method supported by real-time decision, which can be realized by electronic equipment, wherein the electronic equipment can be a terminal or a server. As shown in fig. 2, a flow chart of an intelligent variable-scale data analysis method for real-time decision support, a processing flow of the method may include the following steps:
s201: and acquiring multi-source heterogeneous original data reflecting the decision requirements of the enterprise.
In a practical implementation manner, massive, multi-source and heterogeneous original data capable of reflecting or hiding decision requirements of an enterprise can be obtained in real time, and the massive, multi-source and heterogeneous original data includes but is not limited to unstructured data inside and outside the enterprise, such as project reports, written papers, news reports, conference records, satisfaction questionnaires and the like, and daily business data of the enterprise with obvious structural characteristics.
S202: generating a thought sequence by performing a forward extension, a backward extension or a bidirectional extension process on the concept pair according to the acquired data;
s203: connecting the thought sequences with the same concept starting point or terminal point to obtain a thought concept graph model;
in one possible implementation, the thought conceptual diagram model is used for representing knowledge experience and business logic related to the decision-making problem; the thought conceptual diagram is in a form of thought fragments; the thinking conceptual diagram model comprises: various aspects of the decision-making problem, as well as the concept pair sets at various angles, and the structural associations between different concept pairs.
In a feasible implementation mode, according to a word co-occurrence phenomenon, by combining with artificial intelligent open source technologies such as natural language processing, text mining, a conceptual knowledge base and the like, all concept pairs in original data obtained by a data acquisition module are automatically obtained, and the same concept nodes are combined to obtain a thought concept graph capable of representing decision-making requirements. Meanwhile, according to the basic scale of enterprise business data collection and the decision analysis level required by the enterprise business data collection, a scale space model capable of representing a decision problem solving space is established for each attribute node (such as an attribute node of a clustering type data analysis task, a decision attribute node of a classification prediction type data analysis task and the like) in the thought conceptual diagram, and a multi-scale data model is obtained by combining specific business data.
In one possible implementation, the concept pairs are: the cognitive main body directly relates the precursor concept to other subsequent concepts through imagination or association to form a thinking basic information unit with a partial order relation.
S204: calculating the similarity between different thought sequences, and aggregating the analysis problems by using the similarity result of the thought sequences;
s205: and calculating the similarity among different analysis problems, and determining an analysis theme by using the result of the similarity of the analysis problems.
In one possible embodiment, determining the analysis topic includes:
calculating concept level weight of each node in thought concept graph
By weight of
Establishing a feature vector of each analysis topic, wherein the feature vector comprises the following steps: analysis problem feature decision threshold
Similarity threshold of thinking sequence
Analysis of problem similarity threshold
。
In a possible implementation, as shown in fig. 2, first, any one of the concepts in the "concept pair" obtained from the data acquisition module of the data processing system is used as a node, and the nodes are connected according to the partial order relationship between the concepts in the "concept pair" to form a thought concept graph capable of characterizing the decision requirement. The thought sequence is generated by extending concept node chains formed by concept pairs in the thought concept graph in a forward, backward or bidirectional extension manner. Based on the cosine similarity calculation principle, the similarity of thought sequences is calculated, and the similarity threshold parameter of the thought sequences is utilized
And screening the similarity calculation result of the thought sequence to obtain an analysis problem. Recalculating the similarity of the analysis problems, and using the threshold parameter of the similarity of the analysis problems
And evaluating the similarity calculation result of the analysis problem, and identifying the analysis subject, thereby completing the analysis subject discovery implementation process.
In the embodiment of the invention, by providing a thinking concept graph model and a concept pair acquisition technology thereof, the structural representation of decision requirements in the cross-industry field is realized, and by simulating the thinking activity characteristics of an analyst in the decision process, an automatic and engineered decision requirement identification method from a thinking sequence to an analysis problem to an analysis subject is provided, so that the cross-industry real-time decision support capability in the computer-aided decision field is improved.
S206: calculating the concept characteristics of each node according to the concept hierarchy weight of each concept node in the analysis theme;
s207: identifying the nodes with larger concept characteristic numerical values as concept centers, and judging data analysis tasks according to the out-degree and in-degree of each concept center node and the structural morphological characteristics of different data analysis tasks;
wherein, the out degree is the total number of nodes which can be directly reached by the concept center; the in degree is the total number of nodes that can directly reach the concept center.
In one possible embodiment, the determination of the analysis task is performed by a data analysis module in the system.
In one possible embodiment, analyzing the task decisions includes: incorporating excavation depth parameters
And total number of concept nodes in the thought concept graph
Determining the result of descending order of concept features, and arranging the result before
Determining each node as a concept center and determining an analysis task; wherein the data analysis task decision parameter is
。
In a possible implementation manner, according to the structural morphological features of different data analysis tasks in the analysis topic map, all the data analysis tasks required to be executed by each analysis topic are determined, including a plurality of different data structures under the same analysis task type (such as classification prediction), and the data analysis task decision implementation process is completed.
In one possible embodiment, the structural morphological feature recognition between concept pairs includes the following three cases:
if the central divergent sub-concept graph with the single concept center is identified, constructing a cluster type data analysis task by taking the concept center node as an object and taking each associated node as an attribute, thereby acquiring the conceptual declarative knowledge hidden by the business data;
if two concepts with integral connection are identified, the two concept nodes are respectively used as objects, and an association rule type data analysis task is constructed, so that the propositional declarative knowledge hidden in the service data is obtained;
if the central convergent type sub-concept graph with the single concept center is identified, the concept center node is used as a class label, each associated node is used as a decision attribute, and a classification prediction type data analysis task is constructed, so that the question network type declarative knowledge hidden in the service data is obtained.
In the embodiment of the invention, the data analysis task judgment method realizes the direct automatic judgment of the types and the data structure composition of various specific data analysis tasks contained in the analysis subject chart based on the characteristics of different data analysis tasks, expands the defect that the original variable-scale cluster analysis technology can only solve the single management decision scene suitable for the cluster analysis tasks, and improves the accuracy and the efficiency of the data analysis task judgment.
S208: obtaining a judgment result of the data analysis tasks, and respectively matching a metadata analysis algorithm for each data analysis task according to the judgment result; and performing variable-scale data analysis on the service data by adopting a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
In a possible implementation, a scale space model for representing a decision problem solution space is constructed according to a scale transformation mechanism, and a multi-scale data model is established by combining business data. According to the judgment result of the data analysis tasks, matching a metadata analysis algorithm for each data analysis task by combining the data characteristics (such as high-dimensional sparsity, multiple complex values, mixed data types and the like) of each multi-scale data model, and carrying out variable-scale data analysis on the metadata analysis algorithm to automatically obtain a satisfactory solution.
In one possible embodiment, the analysis of the variable scale data comprises:
a conservative scaling strategy is employed, wherein the technical parameters are subject to the algorithm parameters of the selected metadata analysis algorithm and their thresholds. The conservative scale transformation strategy requires that in the process of scale transformation of the business data, an observation scale with the minimum scale transformation rate is selected to implement scale transformation. The scale conversion rate is used for the degree of change of data distribution characteristics of the service data when the accuracy quantity is converted into different observation scales.
In a possible embodiment, the method for analyzing intelligent variable-scale data with real-time decision support provided by the present invention further includes:
evaluating the effect of executing the scale transformation: and evaluating the execution scale transformation effect through a man-machine interaction submodule. By obtaining part or all of satisfactory results obtained by the analyst through the initial observation scale transformation, and taking the numerical value with the highest evaluation result or score as a satisfactory result evaluation threshold value of the subsequent scale iterative transformation, the analysis result of the variable scale data is ensured to have satisfactory and definite scale characteristics while obtaining a solution meeting the decision preference of the analyst.
And (3) storing system data: and storing a feasible solution set of the management object according to the technical parameter iterative learning result of the variable-scale data analysis method on the business field and the enterprise business characteristics.
In the embodiment of the invention, the intelligent variable-scale data analysis method supported by real-time decision provides a uniform and complete characterization method for the solution space of the decision problem, and specifically comprises the steps of identifying all analysis topics in an initial thought concept graph based on the result of an analysis topic discovery method, and further obtaining an analysis topic graph; determining all data analysis tasks contained in each analysis topic based on the result of the data analysis task judgment method, and further obtaining a structured concept network; a scale space modeling method based on variable scale data analysis obtains all observation scales of all attribute concept nodes in a structured concept network, which can participate in data analysis hierarchical transformation, and further obtains a concept space, thereby completing complete representation of a problem solution space of a decision problem.
FIG. 3 is a block diagram of an intelligent, scale-changing data analysis system with real-time decision support, according to an exemplary embodiment. Referring to fig. 3, the system 300 includes:
the data acquisition module 310 is used for acquiring multi-source heterogeneous original data reflecting enterprise decision requirements;
the data preprocessing module 320 is used for constructing a thinking conceptual diagram model according to the acquired original data and determining an analysis theme according to the thinking conceptual diagram model;
the thinking conceptual diagram model is used for representing knowledge experience and business logic related to decision problems; the thinking concept graph model comprises: various aspects of the decision-making problem and the concept pair sets of various angles, and the structural connection between different concept pairs;
the data analysis module 330 is configured to identify structural morphological features between different concept pairs, and determine a data analysis task included in each analysis topic according to the structural morphological features;
obtaining a judgment result of the data analysis tasks, and respectively matching metadata analysis algorithms for the analysis tasks according to the judgment result; and performing variable-scale data analysis on the service data by adopting a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
Optionally, the concept pair is: the cognitive main body directly relates the precursor concept to other subsequent concepts through imagination or association to form a thinking basic information unit with a partial order relation.
Optionally, a data preprocessing module 320, configured to generate a thought sequence by performing a forward expansion, backward expansion, or bidirectional expansion process on the concept pair according to the acquired data and the acquired data;
connecting the thought sequences with the same concept starting point or terminal point to obtain a thought concept graph model;
calculating the similarity between different thought sequences, and aggregating the analysis problems by using the similarity result of the thought sequences;
and calculating the similarity among different analysis problems, and determining an analysis theme by using the result of the similarity of the analysis problems.
Optionally, the data preprocessing module 320 is further configured to:
calculating concept level weight of each node in thought concept graph
By weight of
Establishing a feature vector of each analysis topic, wherein the feature vector comprises the following steps: analysis problem feature determination threshold
Similarity threshold of thinking sequence
Analysis of problem similarity threshold
。
Optionally, the data analysis module 330 is configured to calculate a concept feature of each node according to the concept hierarchy weight of each concept node in the analysis topic;
identifying the nodes with larger concept characteristic numerical values as concept centers, and judging data analysis tasks according to the out-degree and in-degree of each concept center node and the structural morphological characteristics of different data analysis tasks;
wherein, the out degree is the total number of nodes which can be directly reached by the concept center; the in-degree is the total number of nodes that can reach the concept center directly.
Optionally, the data analysis module 330 is further configured to: incorporating excavation depth parameters
And the total number of concept nodes in the thought concept graph model
，
Determining the result of descending order of concept features, and arranging the result before
Determining each node as a concept center and determining an analysis task; wherein the data analysis task decision parameter is
，
。
Optionally, the structural morphological feature recognition between the concept pairs includes the following three cases:
if the central divergent sub-concept graph with the single concept center is identified, constructing a cluster type data analysis task by taking the concept center node as an object and taking each associated node as an attribute, thereby acquiring the conceptual declarative knowledge hidden by the business data;
if two concepts with integral connection are identified, the two concept nodes are respectively used as objects, and an association rule type data analysis task is constructed, so that the propositional declarative knowledge hidden in the service data is obtained;
if the central convergent type sub-concept graph with the single concept center is identified, the concept center node is used as a class label, each associated node is used as a decision attribute, and a classification prediction type data analysis task is constructed, so that the proposition network type declarative knowledge hidden in the service data is obtained.
Optionally, the variable scale data analysis comprises:
and adopting a conservative scale transformation strategy to carry out variable scale data analysis, wherein the technical parameters are adapted to the algorithm parameters and threshold values of the selected metadata analysis algorithm.
In a possible embodiment, the system of the invention further comprises:
the man-machine interaction submodule 3321 is an important component of a data mining result acquisition logic unit with relatively independent functions in the data analysis module; for performing a scaling effect evaluation procedure. By obtaining part or all of satisfactory results obtained by the analyst through the initial observation scale transformation, and taking the numerical value with the highest evaluation result or score as a satisfactory result evaluation threshold value of the subsequent scale iterative transformation, the analysis result of the variable scale data is ensured to have satisfactory and definite scale characteristics while obtaining a solution meeting the decision preference of the analyst.
And the data storage module 340 is configured to store a feasible solution set of the management object according to the technical parameter iterative learning result of the variable-scale data analysis method on the business field and the enterprise business features.
In the embodiment of the invention, the structure of a data processing system of the intelligent variable-scale data analysis method supported by real-time decision is shown in fig. 3. The data acquisition module 310 provides massive, multi-source, heterogeneous raw data reflecting enterprise decision requirements, and completes concept pair acquisition and thinking concept graph construction through the data preprocessing module 320. The data analysis module 330 completes the variable-scale data analysis based on the data mining task discovery unit 331, the result obtaining unit 332 and the result application unit 333, obtains a satisfactory solution and stores the solution in the module 340. The man-machine interaction sub-module 3321 ensures that the result obtaining unit is implemented efficiently by obtaining the satisfactory evaluation result of the initial scale transformation.
Through the mode, the intelligent variable-scale data analysis method and system for real-time decision support have the advantages that (1) the analysis subject discovery method realizes decision demand characterization and automatic and intelligent recognition by simulating thinking activity characteristics of an analyst in a decision process, and improves cross-industry real-time decision support capability in the field of computer-aided decision; (2) The data analysis task judgment method realizes automatic determination of various data analysis tasks and data structures thereof, and improves the accuracy and efficiency of determination of the data mining task compared with the existing mode of assigning analysis subjects and data analysis tasks by analysts; (3) The intelligent variable-scale data analysis method supported by real-time decision realizes a uniform and complete characterization method for the problem solving space of the decision problem, and obtains a satisfactory solution by automatically completing data analysis hierarchical conversion in the data mining application process; (4) The data processing system of the intelligent variable-scale data analysis method supported by real-time decision provides intelligent deployment technologies such as case recommendation of satisfactory solutions, so that the real-time decision support technology can serve as one of artificial intelligence core capabilities to serve new-generation information infrastructure construction, and intelligent economic high-quality development under the drive of artificial intelligence is promoted.
Fig. 4 is a schematic structural diagram of an electronic device 400 according to an embodiment of the present invention, where the electronic device 400 may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 401 and one or more memories 402, where at least one instruction is stored in the memory 402, and the at least one instruction is loaded and executed by the processor 401 to implement the following steps of the intelligent variable-scale data analysis method with real-time decision support:
s1: acquiring massive, multi-source and heterogeneous original data capable of reflecting or hiding decision requirements of an enterprise in real time, wherein the massive, multi-source and heterogeneous original data comprises but is not limited to unstructured data inside and outside the enterprise;
s2: constructing a thought conceptual diagram model according to the acquired data, and determining an analysis theme according to the thought conceptual diagram model;
the thinking conceptual diagram model is used for representing knowledge experience and business logic related to decision problems; the thinking conceptual diagram model comprises: various aspects of the decision-making problem, and a set of concept pairs for various angles, and structural connections between different concept pairs;
s3: identifying structural morphology features between different concept pairs, and determining a data analysis task contained in each analysis topic according to the structural morphology features;
s4: respectively matching metadata analysis algorithms for the analysis tasks according to the judgment results of the data analysis tasks; and carrying out variable-scale data analysis on the service data according to a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
In an exemplary embodiment, a computer-readable storage medium, such as a memory, is also provided that includes instructions executable by a processor in a terminal to perform the above-described intelligent variable-scale data analysis method for real-time decision support. For example, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. An intelligent variable-scale data analysis method supported by real-time decision is characterized by comprising the following steps:
s1: acquiring multi-source heterogeneous original data reflecting enterprise decision requirements;
s2: constructing a thought conceptual diagram model according to the acquired original data, and determining an analysis subject according to the thought conceptual diagram model;
wherein, the thinking conceptual graph model is used for representing knowledge experience and business logic related to decision-making problems; the thinking concept graph model comprises: the decision problem is characterized by comprising a set of concept pairs of various aspects and angles of the decision problem and structural connections between different concept pairs;
s3: identifying structural morphology features between different concept pairs, and judging data analysis tasks contained in each analysis topic according to the structural morphology features;
s4: obtaining a judgment result of the data analysis tasks, and respectively matching metadata analysis algorithms for the data analysis tasks according to the judgment result; and performing variable-scale data analysis on the service data by adopting a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
2. The method of claim 1, wherein the concept pair is: the cognitive subject directly links the precursor concept to another subsequent concept through imagination or association to form a thinking basic information unit with a partial order relationship.
3. The method according to claim 2, wherein in step S2, constructing a thinking conceptual diagram model according to the acquired raw data, and determining an analysis subject according to the thinking conceptual diagram model comprises:
s21: generating a thought sequence by performing a forward extension, a backward extension or a bidirectional extension process on the concept pair according to the acquired data;
s22: connecting the thought sequences with the same concept starting point or terminal point to obtain a thought concept graph model;
s23: calculating the similarity between different thought sequences, and aggregating the analysis problems by using the similarity result of the thought sequences;
s24: and calculating the similarity among different analysis problems, and determining an analysis theme by using the result of the similarity of the analysis problems.
4. The method according to claim 3, wherein the step S2 further comprises: calculating the concept level weight of each node in the thinking concept graph model
By weight of
Establishing a feature vector of each analysis topic, wherein the feature vector comprises the following steps: analysis problem feature determination threshold
Similarity threshold of thinking sequence
，
Analyzing a problem similarity threshold
，
。
5. The method according to claim 4, wherein in step S3, structural morphology features between different concept pairs are identified, and the data analysis task included in each analysis topic is determined according to the structural morphology features, and the method comprises:
s31: calculating the concept characteristics of each node according to the concept hierarchy weight of each concept node in the analysis theme;
s32: identifying the nodes with larger concept characteristic numerical values as concept centers, and judging data analysis tasks according to the out-degree and in-degree of each concept center node and the structural morphological characteristics of different data analysis tasks;
wherein, the out degree is the total number of nodes directly reachable by the concept center; the in degree is the total number of nodes that can directly reach the concept center.
6. The method of claim 5, wherein the step S3 further comprises: incorporating excavation depth parameters
，
And the total number of concept nodes in the thought concept graph model
，
Determining the result of descending order of concept features, and arranging the result before
Determining each node as a concept center and determining an analysis task; wherein the data analysis task decision parameter is
，
。
7. The method according to claim 5, wherein the step S32 of identifying a node with a large value of the concept feature as a concept center, and performing the data analysis task determination according to the out-degree and in-degree of each concept center node and the structural morphological features of different data analysis tasks comprises:
if the central divergence type sub-concept graph with the single concept center is identified, the concept center node is used as an object, all the associated nodes are used as attributes, a clustering type data analysis task is constructed, and concept type declarative knowledge hidden in the service data is obtained;
if two concepts with integral connection are identified, the two concept nodes are respectively used as objects, an association rule type data analysis task is constructed, and proposition type declarative knowledge hidden in service data is obtained;
if the central convergent type sub-concept graph with the single concept center is identified, the concept center node is used as a class label, each associated node is used as a decision attribute, a classification prediction type data analysis task is constructed, and the question network type declarative knowledge hidden in the service data is obtained.
8. The method of claim 1, wherein in step S4, the variable scale data analysis comprises:
and adopting a conservative scale transformation strategy to carry out variable scale data analysis, wherein the technical parameters are adapted to the algorithm parameters and threshold values of the selected metadata analysis algorithm.
9. An intelligent variable-scale data analysis system with real-time decision support, wherein the system is adapted to the method of any one of the preceding claims 1 to 8, and the system comprises:
the data acquisition module is used for acquiring multi-source heterogeneous original data reflecting enterprise decision requirements;
the data preprocessing module is used for constructing a thinking conceptual diagram model according to the acquired original data and determining an analysis theme according to the thinking conceptual diagram model;
wherein, the thinking conceptual graph model is used for representing knowledge experience and business logic related to decision-making problems; the thinking conceptual diagram model comprises: the decision problem is characterized by comprising a set of concept pairs of various aspects and angles of the decision problem and structural connections between different concept pairs;
the data analysis module is used for identifying structural morphological characteristics among different concept pairs and judging data analysis tasks contained in each analysis topic according to the structural morphological characteristics;
obtaining a judgment result of the data analysis tasks, and respectively matching metadata analysis algorithms for the data analysis tasks according to the judgment result; and performing variable-scale data analysis on the service data by adopting a scale transformation mechanism to obtain a solution, and completing intelligent variable-scale data analysis supported by real-time decision.
10. The system of claim 9, wherein the concept pair is: the cognitive subject directly links the precursor concept to another subsequent concept through imagination or association to form a thinking basic information unit with a partial order relationship.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211299589.0A CN115374316B (en) | 2022-10-24 | 2022-10-24 | Intelligent variable-scale data analysis method and system supported by real-time decision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211299589.0A CN115374316B (en) | 2022-10-24 | 2022-10-24 | Intelligent variable-scale data analysis method and system supported by real-time decision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115374316A true CN115374316A (en) | 2022-11-22 |
| CN115374316B CN115374316B (en) | 2023-01-03 |
Family
ID=84073654
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211299589.0A Active CN115374316B (en) | 2022-10-24 | 2022-10-24 | Intelligent variable-scale data analysis method and system supported by real-time decision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115374316B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200234605A1 (en) * | 2019-01-17 | 2020-07-23 | Laird Harrison Shuart | Telecommunications - enabled semitransparent thought- directed cognitive and conceptual communication linkage method for a workplace brain/cognitive education, training, and augmentation program |
| CN113160563A (en) * | 2021-03-31 | 2021-07-23 | 西安交通大学 | Short-time road condition prediction method, system and equipment based on historical road conditions |
| CN113379344A (en) * | 2021-05-27 | 2021-09-10 | 芜湖锋珉信息科技有限公司 | Intelligent logistics processing method and system based on cloud computing processing |
| CN114580038A (en) * | 2022-03-08 | 2022-06-03 | 中国科学院空天信息创新研究院 | Process-oriented ocean space-time dynamic evolution structure extraction method and device |
| CN114722276A (en) * | 2022-03-29 | 2022-07-08 | 河南图真智能科技有限公司 | Data management and analysis method for smart city service |
-
2022
- 2022-10-24 CN CN202211299589.0A patent/CN115374316B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200234605A1 (en) * | 2019-01-17 | 2020-07-23 | Laird Harrison Shuart | Telecommunications - enabled semitransparent thought- directed cognitive and conceptual communication linkage method for a workplace brain/cognitive education, training, and augmentation program |
| CN113160563A (en) * | 2021-03-31 | 2021-07-23 | 西安交通大学 | Short-time road condition prediction method, system and equipment based on historical road conditions |
| CN113379344A (en) * | 2021-05-27 | 2021-09-10 | 芜湖锋珉信息科技有限公司 | Intelligent logistics processing method and system based on cloud computing processing |
| CN114580038A (en) * | 2022-03-08 | 2022-06-03 | 中国科学院空天信息创新研究院 | Process-oriented ocean space-time dynamic evolution structure extraction method and device |
| CN114722276A (en) * | 2022-03-29 | 2022-07-08 | 河南图真智能科技有限公司 | Data management and analysis method for smart city service |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115374316B (en) | 2023-01-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114721833B (en) | Intelligent cloud coordination method and device based on platform service type | |
| CN111369344B (en) | Method and device for dynamically generating early warning rules | |
| CN117271767B (en) | Operation and maintenance knowledge base establishing method based on multiple intelligent agents | |
| El Bakry et al. | Big data classification using fuzzy K-nearest neighbor | |
| Zhang | Optimization of the marketing management system based on cloud computing and big data | |
| Popchev et al. | DataScience: Experience and Trends | |
| Huang et al. | Intangible cultural heritage management using machine learning model: A case study of northwest folk song Huaer | |
| CN114328277A (en) | Software defect prediction and quality analysis method, device, equipment and medium | |
| CN111708919B (en) | Big data processing method and system | |
| CN111984874B (en) | Parallel recommendation method integrating emotion calculation and network crowdsourcing | |
| CN115374316B (en) | Intelligent variable-scale data analysis method and system supported by real-time decision | |
| Gerrits | Soul of a new machine: Self-learning algorithms in public administration | |
| Ali et al. | Distributed data mining systems: techniques, approaches and algorithms | |
| CN113742488B (en) | Embedded knowledge graph completion method and device based on multitask learning | |
| CN112817560B (en) | Computing task processing method, system and computer readable storage medium based on table function | |
| Lopes et al. | A parallel graph partitioning approach to enhance community detection in social networks | |
| Dai | Analysis of data interaction process based on data mining and neural Network topology visualization | |
| CN115600818A (en) | Multi-dimensional scoring method and device, electronic equipment and storage medium | |
| Piest et al. | Smarter interoperability based on automatic schema matching and intelligence amplification | |
| Park et al. | A new forecasting system using the latent dirichlet allocation (LDA) topic modeling technique | |
| Ding et al. | Constructing a Knowledge Graph for the Chinese Subject Based on Collective Intelligence | |
| Ryabinin et al. | Ontology-Driven Data Mining Platform for Fuzzy Classification of Mental Maps. | |
| Fugini et al. | A text analytics architecture for smart companies | |
| CN117217392B (en) | Method and device for determining general equipment guarantee requirement | |
| CN115208684B (en) | Hypergraph association-based APT attack clue expansion method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |