CN117033467A - Data mining method and device based on pre-calculation and storage medium - Google Patents

Data mining method and device based on pre-calculation and storage medium Download PDF

Info

Publication number
CN117033467A
CN117033467A CN202311057065.5A CN202311057065A CN117033467A CN 117033467 A CN117033467 A CN 117033467A CN 202311057065 A CN202311057065 A CN 202311057065A CN 117033467 A CN117033467 A CN 117033467A
Authority
CN
China
Prior art keywords
data
mining analysis
service request
mining
mined
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311057065.5A
Other languages
Chinese (zh)
Inventor
王永恒
王乐乐
李炳强
罗实
王智
魏明雅
陈昱宇
葛晓东
巫英才
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Lab
Original Assignee
Zhejiang Lab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Lab filed Critical Zhejiang Lab
Priority to CN202311057065.5A priority Critical patent/CN117033467A/en
Publication of CN117033467A publication Critical patent/CN117033467A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2465Query processing support for facilitating data mining operations in structured databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24553Query execution of query operations
    • G06F16/24561Intermediate data storage techniques for performance improvement

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Mathematical Physics (AREA)
  • Fuzzy Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本说明书公开了一种基于预计算的数据挖掘方法、装置及存储介质,获取待挖掘数据,响应于用户的业务请求,根据所述业务请求,对所述待挖掘数据进行预计算,获得中间结果,并存储;根据所述业务请求,在已存储的挖掘分析结果集中,判断是否存在所述业务请求对应的挖掘分析结果,若否,则当预计算完成时,根据所述中间结果,对所述待挖掘数据进行挖掘分析,获得所述业务请求对应的挖掘分析结果,并展示。本方法通过预计算的中间结果进行挖掘分析,减少了对待挖掘数据进行挖掘分析的时间,提高了数据挖掘分析的效率。

This specification discloses a data mining method, device and storage medium based on precomputation, which obtains data to be mined, responds to the user's business request, precomputes the data to be mined according to the business request, and obtains intermediate results. , and store it; according to the business request, determine whether there is a mining analysis result corresponding to the business request in the stored mining analysis result set. If not, when the pre-calculation is completed, based on the intermediate result, all Perform mining analysis on the data to be mined, obtain the mining analysis results corresponding to the business request, and display them. This method performs mining analysis through pre-calculated intermediate results, reduces the time for mining and analyzing the data to be mined, and improves the efficiency of data mining analysis.

Description

一种基于预计算的数据挖掘方法、装置及存储介质A data mining method, device and storage medium based on precomputation

技术领域Technical field

本说明书涉及计算机领域,尤其涉及一种基于预计算的数据挖掘方法、装置及存储介质。This specification relates to the field of computers, and in particular to a data mining method, device and storage medium based on precomputation.

背景技术Background technique

随着互联网技术的发展及用户的需求变化,计算机需要处理的数据也越来越多。计算机在处理数据时,可对数据进行分析,以挖掘数据中有效信息,并通过增加计算资源,如,增加内存,以在数据挖掘分析时,实时获取数据挖掘分析结果。但增加计算资源的同时,也增加了数据维护及硬件的成本。并且,当数据较多或数据挖掘分析的算法较为复杂时,仅通过增加计算资源,无法实时获取数据挖掘分析结果,导致数据挖掘效率较低。With the development of Internet technology and changes in user needs, computers need to process more and more data. When the computer processes data, it can analyze the data to mine effective information in the data, and by increasing computing resources, such as increasing memory, it can obtain data mining analysis results in real time during data mining analysis. However, while increasing computing resources, it also increases the cost of data maintenance and hardware. Moreover, when there is a large amount of data or the data mining analysis algorithm is more complex, the data mining analysis results cannot be obtained in real time simply by increasing computing resources, resulting in low data mining efficiency.

基于此,本说明书提供一种基于预计算的数据挖掘方法。Based on this, this specification provides a data mining method based on precomputation.

发明内容Contents of the invention

本说明书提供一种基于预计算的数据挖掘方法、装置、存储介质及电子设备,以部分的解决现有技术存在的上述问题。This specification provides a data mining method, device, storage medium and electronic device based on precomputation to partially solve the above problems existing in the existing technology.

本说明书采用下述技术方案:This manual adopts the following technical solutions:

本说明书提供了一种基于预计算的数据挖掘方法,包括:This manual provides a data mining method based on precomputation, including:

获取待挖掘数据;Obtain data to be mined;

响应于用户的业务请求,根据所述业务请求,对所述待挖掘数据进行预计算,获得中间结果,并存储;根据所述业务请求,在已存储的挖掘分析结果集中,判断是否存在所述业务请求对应的挖掘分析结果;In response to the user's business request, precompute the data to be mined according to the business request, obtain intermediate results, and store them; according to the business request, determine whether the data to be mined exists in the stored mining analysis result set according to the business request. Mining analysis results corresponding to business requests;

若否,则当预计算完成时,根据所述中间结果,对所述待挖掘数据进行挖掘分析,获得所述业务请求对应的挖掘分析结果,并展示。If not, when the pre-calculation is completed, perform mining analysis on the data to be mined based on the intermediate result, obtain the mining analysis result corresponding to the business request, and display it.

可选地,根据所述业务请求,对所述待挖掘数据进行预计算,获得中间结果,具体包括:Optionally, according to the business request, precompute the data to be mined to obtain intermediate results, which specifically includes:

根据所述业务请求,在预先构建的若干个预计算模型中,确定所述业务请求匹配的预计算模型;According to the business request, among several pre-built pre-computing models, determine the pre-computing model matching the business request;

将所述待挖掘数据输入所述预计算模型,获得所述预计算模型输出的中间结果,其中,所述预计算包括数据清洗、特征提取。The data to be mined is input into the pre-computation model to obtain an intermediate result output by the pre-computation model, where the pre-computation includes data cleaning and feature extraction.

可选地,预先构建预计算模型,具体包括:Optionally, pre-build a pre-computed model, including:

针对每个业务,获取该业务的数据;For each business, obtain the data of the business;

根据该业务的数据,配置预计算模型参数,以构建预计算模型。According to the data of the business, configure the pre-computed model parameters to build the pre-computed model.

可选地,获得中间结果之后,所述方法还包括:Optionally, after obtaining the intermediate result, the method further includes:

针对每个中间结果,建立所述业务请求与该中间结果的索引,并存储。For each intermediate result, an index of the service request and the intermediate result is established and stored.

可选地,根据所述中间结果,对所述待挖掘数据进行挖掘分析,具体包括:Optionally, perform mining analysis on the data to be mined based on the intermediate results, specifically including:

根据所述索引,确定所述业务请求的中间结果;Determine the intermediate result of the service request according to the index;

根据所述中间结果,对所述待挖掘数据进行挖掘分析。According to the intermediate results, the data to be mined is mined and analyzed.

可选地,所述方法还包括:Optionally, the method also includes:

存储所述挖掘分析结果,并存储所述业务请求与所述挖掘分析结果的对应关系。The mining analysis results are stored, and the corresponding relationship between the service request and the mining analysis results is stored.

可选地,根据所述业务请求,在已存储的挖掘分析结果集中,判断是否存在所述业务请求对应的挖掘分析结果,具体包括:Optionally, according to the business request, determine whether there is a mining analysis result corresponding to the business request in the stored mining analysis result set, specifically including:

根据所述业务请求及已存储的挖据分析结果集中的对应关系,判断所述已存储的挖掘分析结果集中是否存在所述业务请求对应的挖掘分析结果。According to the corresponding relationship between the business request and the stored mining analysis result set, it is determined whether there is a mining analysis result corresponding to the business request in the stored mining analysis result set.

本说明书提供了一种基于预计算的数据挖掘装置,包括:This manual provides a data mining device based on precomputation, including:

数据获取模块,用于获取待挖掘数据;Data acquisition module, used to obtain data to be mined;

响应模块,用于响应于用户的业务请求,根据所述业务请求,对所述待挖掘数据进行预计算,获得中间结果,并存储;根据所述业务请求,在已存储的挖掘分析结果集中,判断是否存在所述业务请求对应的挖掘分析结果;The response module is used to respond to the user's business request, perform pre-calculation on the data to be mined according to the business request, obtain intermediate results, and store them; according to the business request, in the stored mining analysis result set, Determine whether there is a mining analysis result corresponding to the business request;

结果获取模块,用于若否,则当预计算完成时,根据所述中间结果,对所述待挖掘数据进行挖掘分析,获得所述业务请求对应的挖掘分析结果,并展示。The result acquisition module is configured to, if not, perform mining analysis on the data to be mined according to the intermediate result when the pre-calculation is completed, obtain the mining analysis result corresponding to the business request, and display it.

可选地,所述响应模块具体用于,根据所述业务请求,在预先构建的若干个预计算模型中,确定所述业务请求匹配的预计算模型;将所述待挖掘数据输入所述预计算模型,获得所述预计算模型输出的中间结果,其中,所述预计算包括数据清洗、特征提取。Optionally, the response module is specifically configured to, according to the business request, determine a pre-computing model matching the business request among several pre-built pre-computing models; and input the data to be mined into the pre-computing model. Calculate the model to obtain the intermediate results output by the pre-computation model, where the pre-computation includes data cleaning and feature extraction.

可选地,所述装置还包括:Optionally, the device also includes:

模型构建模块,用于针对每个业务,获取该业务的数据;根据该业务的数据,配置预计算模型参数,以构建预计算模型。The model building module is used to obtain the data of each business and configure the pre-computed model parameters based on the data of the business to build a pre-computed model.

可选地,所述装置还包括:Optionally, the device also includes:

索引建立模块,用于针对每个中间结果,建立所述业务请求与该中间结果的索引,并存储。An index creation module is used to establish an index of the service request and the intermediate result for each intermediate result, and store the index.

可选地,所述结果获取模块具体用于,根据所述索引,确定所述业务请求的中间结果;根据所述中间结果,对所述待挖掘数据进行挖掘分析。Optionally, the result acquisition module is specifically configured to determine the intermediate result of the service request according to the index; and perform mining analysis on the data to be mined according to the intermediate result.

可选地,所述装置还包括:Optionally, the device also includes:

存储模块,用于存储所述挖掘分析结果,并存储所述业务请求与所述挖掘分析结果的对应关系。A storage module, configured to store the mining analysis results and store the corresponding relationship between the business request and the mining analysis results.

可选地,所述结果获取模块具体用于,根据所述业务请求及已存储的挖据分析结果集中的对应关系,判断所述已存储的挖掘分析结果集中是否存在所述业务请求对应的挖掘分析结果。Optionally, the result acquisition module is specifically configured to determine whether there is a mining corresponding to the business request in the stored mining analysis result set based on the corresponding relationship between the business request and the stored mining analysis result set. Analyze the results.

本说明书提供了一种计算机可读存储介质,所述存储介质存储有计算机程序,所述计算机程序被处理器执行时实现上述基于预计算的数据挖掘方法。This specification provides a computer-readable storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned precomputation-based data mining method is implemented.

本说明书提供了一种电子设备,包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序,所述处理器执行所述程序时实现上述基于预计算的数据挖掘方法。This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-mentioned precomputation-based data mining method.

本说明书采用的上述至少一个技术方案能够达到以下有益效果:At least one of the above technical solutions adopted in this manual can achieve the following beneficial effects:

从本说明书提供的基于预计算的数据挖掘方法可以看出,本方法对待挖掘数据进行预计算,同时,判断已存储的挖掘分析结果中是否存在当前业务请求对应的结果,若否,则利用预计算的中间结果对待挖掘数据进行挖掘分析,以获取当前业务请求对应的结果。通过预计算的中间结果进行挖掘分析,减少了对待挖掘数据进行挖掘分析的时间,提高了数据挖掘分析的效率。It can be seen from the data mining method based on precomputation provided in this manual that this method precomputes the data to be mined, and at the same time, determines whether there is a result corresponding to the current business request in the stored mining analysis results. If not, use the precalculated The intermediate results of the calculation are mined and analyzed on the data to be mined to obtain the results corresponding to the current business request. Mining and analyzing through pre-computed intermediate results reduces the time for mining and analyzing the data to be mined, and improves the efficiency of data mining and analysis.

附图说明Description of the drawings

此处所说明的附图用来提供对本说明书的进一步理解,构成本说明书的一部分,本说明书的示意性实施例及其说明用于解释本说明书,并不构成对本说明书的不当限定。在附图中:The drawings described here are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the attached picture:

图1为本说明书中提供的一种基于预计算的数据挖掘方法的流程示意图;Figure 1 is a schematic flow chart of a data mining method based on precomputation provided in this specification;

图2为本说明书提供的一种基于预计算的数据挖掘装置结构的示意图;Figure 2 is a schematic diagram of the structure of a data mining device based on precomputation provided in this specification;

图3为本说明书提供的一种对应于图1的电子设备的结构示意图。FIG. 3 is a schematic structural diagram of an electronic device corresponding to FIG. 1 provided in this specification.

具体实施方式Detailed ways

为使本说明书的目的、技术方案和优点更加清楚,下面将结合本说明书具体实施例及相应的附图对本说明书技术方案进行清楚、完整地描述。显然,所描述的实施例仅是本说明书一部分实施例,而不是全部的实施例。基于本说明书中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本说明书保护的范围。In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of this specification, but not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this specification.

以下结合附图,详细说明本说明书各实施例提供的技术方案。The technical solutions provided by each embodiment of this specification will be described in detail below with reference to the accompanying drawings.

图1为本说明书中提供的一种基于预计算的数据挖掘方法的流程示意图,包括以下步骤:Figure 1 is a schematic flowchart of a precomputation-based data mining method provided in this manual, including the following steps:

S100:获取待挖掘数据。S100: Obtain the data to be mined.

数据实时挖掘分析强调挖掘分析结果的时效性,通常采用计算升级思路,即增加计算机资源,再通过对CPU、内存等计算资源的合理调配,实现低响应时间的数据挖掘分析。然而,这造成数据维护及硬件等相关成本不断增加,此外仅仅靠计算升级,在数据规模较大或算法复杂时,仍存在难以支持分析的低响应问题,导致数据挖掘效率较低。因此,本说明书一种基于预计算的数据挖掘分析方法。Real-time data mining analysis emphasizes the timeliness of mining analysis results, and usually adopts the computing upgrade idea, that is, increasing computer resources, and then through reasonable allocation of computing resources such as CPU and memory to achieve low response time data mining analysis. However, this has resulted in increasing costs related to data maintenance and hardware. In addition, only relying on computing upgrades, when the data scale is large or the algorithm is complex, there is still a low response problem that is difficult to support analysis, resulting in low data mining efficiency. Therefore, this description is a data mining analysis method based on precomputation.

本说明书的执行主体可为用于数据挖掘分析的服务器,也可以是其他可用于数据挖掘分析的电子设备。为了便于说明,本说明书以服务器为执行主体,对本说明书提供的基于预计算的数据挖掘方法进行说明。The execution subject of this instruction can be a server used for data mining analysis, or other electronic equipment that can be used for data mining analysis. For the convenience of explanation, this specification uses the server as the execution subject to describe the precomputation-based data mining method provided in this specification.

为了对数据进行挖掘分析,服务器可先获取待挖掘数据,该待挖掘数据可为数据库中的数据。In order to perform data mining and analysis, the server can first obtain the data to be mined, and the data to be mined can be data in a database.

S102:响应于用户的业务请求,根据所述业务请求,对所述待挖掘数据进行预计算,获得中间结果,并存储。S102: In response to the user's service request, precompute the data to be mined according to the service request, obtain intermediate results, and store them.

在本说明书一个或多个实施例中,业务包括最短路径、关联规则、子图匹配等。针对不同业务,该业务的待挖掘数据不同,那么,相应的预计算存在差异。因此,服务器可根据所述业务请求,在预先构建的若干个预计算模型中,确定所述业务请求匹配的预计算模型。服务器可通过业务请求的标识,确定与自身业务请求匹配的预计算模型,例如,业务请求的标识为1,则若预计算模型标识也为1,则该预计算模型与该业务请求匹配。本说明书不限制确定与业务请求匹配的预计算模型的方式。In one or more embodiments of this specification, services include shortest paths, association rules, subgraph matching, etc. For different businesses, the data to be mined for the business is different, so the corresponding pre-calculation is different. Therefore, according to the service request, the server can determine a pre-computing model matching the service request among several pre-built pre-computing models. The server can determine the pre-computing model that matches its own business request through the identifier of the business request. For example, if the identifier of the business request is 1, then if the pre-computation model identifier is also 1, then the pre-computation model matches the business request. This specification does not limit the manner in which the precomputed model that matches the business request is determined.

再将所述待挖掘数据输入所述预计算模型,获得所述预计算模型输出的中间结果,并存储。当业务为确定最短路径时,中间结果包括网络图中任意两个节点之间的最短路径。当业务为子图匹配时,中间结果包括与业务匹配的数据图和子图。针对不同业务,中间结果存在差异。其中,所述预计算包括数据清洗、特征提取。数据清洗包括处理缺失值、删除重复数据、纠正格式错误、去除噪声等,以提高数据质量。当业务为子图匹配时,特征提取包括为节点和边提取顶点/边嵌入等特征。Then the data to be mined is input into the pre-computation model, and the intermediate result output by the pre-computation model is obtained and stored. When the business is to determine the shortest path, the intermediate results include the shortest path between any two nodes in the network graph. When the business is a subgraph match, the intermediate results include the data graph and subgraph that match the business. For different businesses, the intermediate results are different. Wherein, the pre-calculation includes data cleaning and feature extraction. Data cleaning includes processing missing values, deleting duplicate data, correcting format errors, removing noise, etc. to improve data quality. When the business is subgraph matching, feature extraction includes extracting features such as vertex/edge embeddings for nodes and edges.

需要说明的是,由于业务不同,待挖掘数据不同,预计算模型也不相同。对于不同类型的业务,需要构建不同的预计算模型。其中,不同类型的业务是指业务不同和/或待挖掘数据不同。例如,现有一路径图,图中包括A、B、C、D四个节点,图的边为节点之间的路径,业务1为确定A与B之间的最短路径,业务2为确定C与D之间的最短路径,由于业务1与业务2均为确定最短路径,且待挖掘数据均为该路径图,则业务1与业务2可使用同样的预计算模型。It should be noted that due to different businesses, different data to be mined, and different pre-computing models. For different types of businesses, different precomputing models need to be built. Among them, different types of businesses refer to different businesses and/or different data to be mined. For example, there is a path graph, which includes four nodes A, B, C, and D. The edges of the graph are the paths between the nodes. Business 1 is to determine the shortest path between A and B. Business 2 is to determine the shortest path between C and B. The shortest path between D, since both business 1 and business 2 are determined shortest paths, and the data to be mined are all path graphs, business 1 and business 2 can use the same pre-computation model.

此外,由于预计算模型存在差异,为了提高数据挖掘的效率,可预先构建预计算模型。In addition, due to differences in precomputed models, in order to improve the efficiency of data mining, precomputed models can be built in advance.

具体的,针对每个业务,获取该业务的数据,根据该业务的数据,配置预计算模型参数,以构建预计算模型。也就是说,针对每个业务,获取该业务的数据库中的数据,根据该业务及该数据,选择进行预计算的挖掘算法,并配置预计算模型参数,构建预计算模型。当该业务为确定最短路径时,在支持预计算的算法中,选任一算法,如广度优先搜索算法(Breadth-First Search,BFS),根据确定最短路径的图及该算法,配置预计算模型参数,构建预计算模型。Specifically, for each business, the data of the business is obtained, and the pre-computing model parameters are configured according to the data of the business to build the pre-computing model. That is to say, for each business, obtain the data in the database of the business, select the mining algorithm for precomputation based on the business and the data, configure the precomputation model parameters, and build the precomputation model. When the business is to determine the shortest path, select any algorithm among the algorithms that support precomputation, such as the Breadth-First Search algorithm (Breadth-First Search, BFS), and configure the precomputation model parameters based on the graph that determines the shortest path and the algorithm. , build a precomputed model.

S104:根据所述业务请求,在已存储的挖掘分析结果集中,判断是否存在所述业务请求对应的挖掘分析结果,若是,执行S106,若否,执行S108。S104: According to the business request, determine whether there is a mining analysis result corresponding to the business request in the stored mining analysis result set. If yes, execute S106. If not, execute S108.

服务器在业务请求对应的挖掘分析结果之后,可存储所述挖掘分析结果,并存储所述业务请求与所述挖掘分析结果的对应关系,以便后续执行同样的业务请求时,直接使用已存储的挖掘分析结果。因此,服务器在确定挖据分析结果时,可根据所述业务请求,在已存储的挖掘分析结果集中,判断是否存在所述业务请求对应的挖掘分析结果。After the server requests the mining analysis results corresponding to the business request, it can store the mining analysis results and store the corresponding relationship between the business requests and the mining analysis results, so that when the same business request is subsequently executed, the stored mining analysis results can be directly used. Analyze the results. Therefore, when determining the data mining analysis result, the server can determine whether there is a mining analysis result corresponding to the business request in the stored mining analysis result set according to the business request.

具体的,根据所述业务请求及已存储的挖据分析结果集中的对应关系,判断所述已存储的挖掘分析结果集中是否存在所述业务请求对应的挖掘分析结果。本说明书不限制确定业务请求与挖掘分析结果的对应关系的方式。Specifically, based on the corresponding relationship between the business request and the stored mining analysis result set, it is determined whether there is a mining analysis result corresponding to the business request in the stored mining analysis result set. This specification does not limit the method of determining the correspondence between business requests and mining analysis results.

S106:将所述业务请求对应的挖掘分析结果返回至所述用户。S106: Return the mining analysis results corresponding to the service request to the user.

S108:当预计算完成时,根据所述中间结果,对所述待挖掘数据进行挖掘分析,获得所述业务请求对应的挖掘分析结果,并展示。S108: When the pre-calculation is completed, perform mining analysis on the data to be mined based on the intermediate results, obtain the mining analysis results corresponding to the business request, and display them.

若在已存储的挖掘分析结果集中,不存在所述业务请求对应的挖掘分析结果,那么,还可判断预计算是否完成,以利用中间结果,获取挖掘分析结果,提高挖据分析效率。If there is no mining analysis result corresponding to the business request in the stored mining analysis result set, it can also be determined whether the pre-calculation is completed, so as to use the intermediate results to obtain the mining analysis results and improve the efficiency of data mining analysis.

具体的,当预计算完成时,服务器根据所述中间结果,对所述待挖掘数据进行挖掘分析,获得所述业务请求对应的挖掘分析结果,并展示。需要说明的是,在获取中间结果之后,针对每个中间结果,建立所述业务请求与该中间结果的索引,并存储。根据所述索引,确定所述业务请求的中间结果,根据所述中间结果,对所述待挖掘数据进行挖掘分析。当业务为确定最短路径时,根据所述索引从中间结果中,即从图中任意两个节点之间的最短路径中,选择该业务的最短路径,得到该业务请求对应的挖掘分析结果。Specifically, when the pre-calculation is completed, the server performs mining analysis on the data to be mined based on the intermediate result, obtains the mining analysis result corresponding to the business request, and displays it. It should be noted that after obtaining the intermediate results, for each intermediate result, an index of the business request and the intermediate results is established and stored. According to the index, an intermediate result of the service request is determined, and according to the intermediate result, the data to be mined is mined and analyzed. When the business is to determine the shortest path, the shortest path of the business is selected from the intermediate results according to the index, that is, from the shortest path between any two nodes in the graph, and the mining analysis result corresponding to the business request is obtained.

基于图1所示的基于预计算的数据挖掘方法,对待挖掘数据进行预计算,同时,判断已存储的挖掘分析结果中是否存在当前业务请求对应的结果,若否,则利用预计算的中间结果对待挖掘数据进行挖掘分析,以获取当前业务请求对应的结果。通过预计算的中间结果进行挖掘分析,减少了对待挖掘数据进行挖掘分析的时间,提高了数据挖掘分析的效率。Based on the precomputation-based data mining method shown in Figure 1, the data to be mined is precomputed. At the same time, it is judged whether there is a result corresponding to the current business request in the stored mining analysis results. If not, the precomputed intermediate result is used. Conduct mining and analysis on the data to be mined to obtain results corresponding to the current business request. Mining and analyzing through pre-computed intermediate results reduces the time for mining and analyzing the data to be mined, and improves the efficiency of data mining and analysis.

需要说明的是,最短路径、关联规则、子图匹配可应用在各个领域中,如商品推荐,最短路径可以找到用户的最近邻,关联规则发现商品关联,子图匹配检测用户行为模式。异常交易检测,最短路径识别可疑关联交易,关联规则找出高风险指标,子图匹配发现异常行为图模式。物流运输,最短路径规划物流路线,关联规则优化仓储配送,子图匹配分析交通网络。网络安全,最短路径识别攻击路径,关联规则发现漏洞关联,子图匹配检测攻击模式等等。相对的待挖掘数据为实际业务场景的数据,本说明书不再赘述。It should be noted that shortest path, association rules, and subgraph matching can be applied in various fields, such as product recommendation. The shortest path can find the user's nearest neighbor, association rules can discover product associations, and subgraph matching can detect user behavior patterns. Abnormal transaction detection, shortest path identification of suspicious related transactions, association rules to find high-risk indicators, and subgraph matching to discover abnormal behavior graph patterns. Logistics and transportation, shortest path planning of logistics routes, association rules optimization of warehousing and distribution, subgraph matching and analysis of transportation networks. Network security, shortest path identification of attack paths, association rules to discover vulnerability associations, subgraph matching to detect attack patterns, etc. The data to be mined are data from actual business scenarios, and will not be described in detail in this manual.

针对步骤S108,沿用S102中的例子,若服务器已经获取到业务1的挖掘分析结果,那么,在执行业务2时,可根据业务1的中间结果,获取业务2的挖掘分析结果。也就是说,针对同类业务,可根据已获取的业务请求对应的挖据分析结果的中间结果,获取当前业务请求的挖掘分析结果。Regarding step S108, following the example in S102, if the server has obtained the mining analysis results of business 1, then when executing business 2, it can obtain the mining analysis results of business 2 based on the intermediate results of business 1. That is to say, for similar businesses, the mining analysis results of the current business request can be obtained based on the intermediate results of the mining analysis results corresponding to the acquired business requests.

以上为本说明书的一个或多个实施的方法,基于图1所示的基于预计算的数据挖掘方法的流程示意图,本说明书还提供了相应的基于预计算的数据挖掘装置,如图2所示。The above is one or more implementation methods of this specification. Based on the flow diagram of the pre-computation-based data mining method shown in Figure 1, this specification also provides a corresponding pre-computation-based data mining device, as shown in Figure 2 .

图2为本说明书提供的一种的基于预计算的数据挖掘装置的示意图,包括:Figure 2 is a schematic diagram of a precomputation-based data mining device provided in this specification, including:

数据获取模块200,用于获取待挖掘数据;Data acquisition module 200, used to acquire data to be mined;

响应模块202,用于响应于用户的业务请求,根据所述业务请求,对所述待挖掘数据进行预计算,获得中间结果,并存储;根据所述业务请求,在已存储的挖掘分析结果集中,判断是否存在所述业务请求对应的挖掘分析结果;The response module 202 is configured to respond to the user's business request, precompute the data to be mined according to the business request, obtain intermediate results, and store them; according to the business request, in the stored mining analysis result set , determine whether there is a mining analysis result corresponding to the business request;

结果获取模块204,用于若否,则当预计算完成时,根据所述中间结果,对所述待挖掘数据进行挖掘分析,获得所述业务请求对应的挖掘分析结果,并展示。The result acquisition module 204 is configured to, if not, perform mining analysis on the data to be mined according to the intermediate result when the pre-calculation is completed, obtain the mining analysis result corresponding to the business request, and display it.

可选地,所述响应模块202具体用于,根据所述业务请求,在预先构建的若干个预计算模型中,确定所述业务请求匹配的预计算模型;将所述待挖掘数据输入所述预计算模型,获得所述预计算模型输出的中间结果,其中,所述预计算包括数据清洗、特征提取。Optionally, the response module 202 is specifically configured to, according to the business request, determine a pre-computing model matching the business request among several pre-built pre-computing models; and input the data to be mined into the Precompute the model to obtain the intermediate results output by the precomputation model, where the precomputation includes data cleaning and feature extraction.

可选地,所述装置还包括:Optionally, the device also includes:

模型构建模块206,用于针对每个业务,获取该业务的数据;根据该业务的数据,配置预计算模型参数,以构建预计算模型。The model building module 206 is used for obtaining the data of each business and configuring the pre-calculated model parameters according to the data of the business to build a pre-calculated model.

可选地,所述装置还包括:Optionally, the device also includes:

索引建立模块208,用于针对每个中间结果,建立所述业务请求与该中间结果的索引,并存储。The index creation module 208 is configured to create an index of the service request and the intermediate result for each intermediate result, and store the index.

可选地,所述结果获取模块204具体用于,根据所述索引,确定所述业务请求的中间结果;根据所述中间结果,对所述待挖掘数据进行挖掘分析。Optionally, the result acquisition module 204 is specifically configured to determine the intermediate result of the service request according to the index; and perform mining analysis on the data to be mined according to the intermediate result.

可选地,所述装置还包括:Optionally, the device also includes:

存储模块210,用于存储所述挖掘分析结果,并存储所述业务请求与所述挖掘分析结果的对应关系。The storage module 210 is used to store the mining analysis results, and store the correspondence between the business request and the mining analysis results.

可选地,所述结果获取模块204具体用于,根据所述业务请求及已存储的挖据分析结果集中的对应关系,判断所述已存储的挖掘分析结果集中是否存在所述业务请求对应的挖掘分析结果。Optionally, the result acquisition module 204 is specifically configured to determine, based on the corresponding relationship between the business request and the stored mining analysis result set, whether there is a data corresponding to the business request in the stored mining analysis result set. Mining analysis results.

本说明书还提供了一种计算机可读存储介质,该存储介质存储有计算机程序,计算机程序可用于执行上述图1提供的一种基于预计算的数据挖掘方法。This specification also provides a computer-readable storage medium that stores a computer program. The computer program can be used to execute a precomputation-based data mining method provided in Figure 1 above.

本说明书还提供了图3所示的一种对应于图1的电子设备的结构示意图。如图3所示,在硬件层面,该电子设备包括处理器、内部总线、网络接口、内存以及非易失性存储器,当然还可能包括其他业务所需要的硬件。处理器从非易失性存储器中读取对应的计算机程序到内存中然后运行,以实现上述图1所述的基于预计算的数据挖掘方法。This specification also provides a schematic structural diagram of the electronic device shown in FIG. 3 corresponding to FIG. 1 . As shown in Figure 3, at the hardware level, the electronic device includes a processor, internal bus, network interface, memory and non-volatile memory, and of course may also include other hardware required for business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the pre-computation-based data mining method described in Figure 1 above.

当然,除了软件实现方式之外,本说明书并不排除其他实现方式,比如逻辑器件抑或软硬件结合的方式等等,也就是说以下处理流程的执行主体并不限定于各个逻辑单元,也可以是硬件或逻辑器件。Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and may also be hardware or logic device.

在20世纪90年代,对于一个技术的改进可以很明显地区分是硬件上的改进(例如,对二极管、晶体管、开关等电路结构的改进)还是软件上的改进(对于方法流程的改进)。然而,随着技术的发展,当今的很多方法流程的改进已经可以视为硬件电路结构的直接改进。设计人员几乎都通过将改进的方法流程编程到硬件电路中来得到相应的硬件电路结构。因此,不能说一个方法流程的改进就不能用硬件实体模块来实现。例如,可编程逻辑器件(Programmable Logic Device,PLD)(例如现场可编程门阵列(Field Programmable GateArray,FPGA))就是这样一种集成电路,其逻辑功能由用户对器件编程来确定。由设计人员自行编程来把一个数字系统“集成”在一片PLD上,而不需要请芯片制造厂商来设计和制作专用的集成电路芯片。而且,如今,取代手工地制作集成电路芯片,这种编程也多半改用“逻辑编译器(logic compiler)”软件来实现,它与程序开发撰写时所用的软件编译器相类似,而要编译之前的原始代码也得用特定的编程语言来撰写,此称之为硬件描述语言(Hardware Description Language,HDL),而HDL也并非仅有一种,而是有许多种,如ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware DescriptionLanguage)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(RubyHardware Description Language)等,目前最普遍使用的是VHDL(Very-High-SpeedIntegrated Circuit Hardware Description Language)与Verilog。本领域技术人员也应该清楚,只需要将方法流程用上述几种硬件描述语言稍作逻辑编程并编程到集成电路中,就可以很容易得到实现该逻辑方法流程的硬件电路。In the 1990s, improvements in a technology could be clearly distinguished as hardware improvements (for example, improvements in circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements in method processes). However, with the development of technology, many improvements in today's method processes can be regarded as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement of a method flow cannot be implemented using hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit, and its logic function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system on a PLD, instead of asking chip manufacturers to design and produce dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. Before compiling, The original code must also be written in a specific programming language, which is called Hardware Description Language (HDL). There is not only one type of HDL, but many types, such as ABEL (Advanced Boolean Expression Language) , AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., the most commonly used ones currently are VHDL (Very-High-SpeedIntegrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that by simply logically programming the method flow using the above-mentioned hardware description languages and programming it into the integrated circuit, the hardware circuit that implements the logical method flow can be easily obtained.

控制器可以按任何适当的方式实现,例如,控制器可以采取例如微处理器或处理器以及存储可由该(微)处理器执行的计算机可读程序代码(例如软件或固件)的计算机可读介质、逻辑门、开关、专用集成电路(Application Specific Integrated Circuit,ASIC)、可编程逻辑控制器和嵌入微控制器的形式,控制器的例子包括但不限于以下微控制器:ARC 625D、Atmel AT91SAM、Microchip PIC18F26K20以及Silicone Labs C8051F320,存储器控制器还可以被实现为存储器的控制逻辑的一部分。本领域技术人员也知道,除了以纯计算机可读程序代码方式实现控制器以外,完全可以通过将方法步骤进行逻辑编程来使得控制器以逻辑门、开关、专用集成电路、可编程逻辑控制器和嵌入微控制器等的形式来实现相同功能。因此这种控制器可以被认为是一种硬件部件,而对其内包括的用于实现各种功能的装置也可以视为硬件部件内的结构。或者甚至,可以将用于实现各种功能的装置视为既可以是实现方法的软件模块又可以是硬件部件内的结构。The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (eg, software or firmware) executable by the (micro)processor. , logic gates, switches, Application Specific Integrated Circuit (ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the memory control logic. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the controller can be completely programmed with logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded logic by logically programming the method steps. Microcontroller, etc. to achieve the same function. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the means for implementing various functions can be considered as structures within hardware components as well as software modules implementing the methods.

上述实施例阐明的系统、装置、模块或单元,具体可以由计算机芯片或实体实现,或者由具有某种功能的产品来实现。一种典型的实现设备为计算机。具体的,计算机例如可以为个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任何设备的组合。The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or A combination of any of these devices.

为了描述的方便,描述以上装置时以功能分为各种单元分别描述。当然,在实施本说明书时可以把各单元的功能在同一个或多个软件和/或硬件中实现。For the convenience of description, when describing the above device, the functions are divided into various units and described separately. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and/or hardware.

本领域内的技术人员应明白,本说明书的实施例可提供为方法、系统、或计算机程序产品。因此,本说明书可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本说明书可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art will understand that embodiments of the present specification may be provided as methods, systems, or computer program products. Thus, the present description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

本说明书是参照根据本说明书实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The specification is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the specification. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.

这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

在一个典型的配置中,计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

内存可能包括计算机可读介质中的非永久性存储器,随机存取存储器(RAM)和/或非易失性内存等形式,如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。Memory may include non-permanent storage in computer-readable media, random access memory (RAM), and/or non-volatile memory in the form of read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括,但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带,磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质,可用于存储可以被计算设备访问的信息。按照本文中的界定,计算机可读介质不包括暂存电脑可读媒体(transitory media),如调制的数据信号和载波。Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory. (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

还需要说明的是,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。It should also be noted that the terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements not only includes those elements, but also includes Other elements are not expressly listed or are inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

本领域技术人员应明白,本说明书的实施例可提供为方法、系统或计算机程序产品。因此,本说明书可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且,本说明书可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art will appreciate that embodiments of the present specification may be provided as methods, systems, or computer program products. Thus, the present description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

本说明书可以在由计算机执行的计算机可执行指令的一般上下文中描述,例如程序模块。一般地,程序模块包括执行特定任务或实现特定抽象数据类型的例程、程序、对象、组件、数据结构等等。也可以在分布式计算环境中实践本说明书,在这些分布式计算环境中,由通过通信网络而被连接的远程处理设备来执行任务。在分布式计算环境中,程序模块可以位于包括存储设备在内的本地和远程计算机存储介质中。This specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present description may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through communications networks. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

本说明书中的各个实施例均采用递进的方式描述,各个实施例之间相同相似的部分互相参见即可,每个实施例重点说明的都是与其他实施例的不同之处。尤其,对于系统实施例而言,由于其基本相似于方法实施例,所以描述的比较简单,相关之处参见方法实施例的部分说明即可。Each embodiment in this specification is described in a progressive manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment.

以上所述仅为本说明书的实施例而已,并不用于限制本说明书。对于本领域技术人员来说,本说明书可以有各种更改和变化。凡在本说明书的精神和原理之内所作的任何修改、等同替换、改进等,均应包含在本说明书的权利要求范围之内。The above descriptions are only examples of this specification and are not intended to limit this specification. Various modifications and variations may occur to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification shall be included in the scope of the claims of this specification.

Claims (10)

1. A pre-calculation based data mining analysis method, the method comprising:
acquiring data to be mined;
responding to a service request of a user, pre-calculating the data to be mined according to the service request, obtaining an intermediate result, and storing the intermediate result; judging whether a mining analysis result corresponding to the service request exists in a stored mining analysis result set according to the service request;
if not, when pre-calculation is completed, mining and analyzing the data to be mined according to the intermediate result, obtaining mining and analyzing results corresponding to the service request, and displaying the mining and analyzing results.
2. The method of claim 1, wherein the pre-computing the data to be mined according to the service request to obtain an intermediate result specifically comprises:
according to the service request, determining a pre-calculation model matched with the service request from a plurality of pre-calculation models which are pre-constructed;
inputting the data to be mined into the pre-calculation model to obtain an intermediate result output by the pre-calculation model, wherein the pre-calculation comprises data cleaning and feature extraction.
3. The method according to claim 2, characterized in that the pre-calculation model is pre-built, in particular comprising:
for each service, acquiring data of the service;
according to the data of the service, pre-calculation model parameters are configured to construct a pre-calculation model.
4. The method of claim 1, wherein after obtaining the intermediate result, the method further comprises:
for each intermediate result, an index of the service request and the intermediate result is established and stored.
5. The method of claim 4, wherein the mining analysis is performed on the data to be mined according to the intermediate result, specifically comprising:
determining an intermediate result of the service request according to the index;
and mining and analyzing the data to be mined according to the intermediate result.
6. The method of claim 1, wherein the method further comprises:
and storing the mining analysis result, and storing the corresponding relation between the service request and the mining analysis result.
7. The method of claim 6, wherein determining whether there is a mining analysis result corresponding to the service request in the stored mining analysis result set according to the service request, specifically comprises:
judging whether the mining analysis result corresponding to the service request exists in the stored mining analysis result set according to the corresponding relation between the service request and the stored mining analysis result set.
8. A pre-calculation based data mining analysis apparatus, the apparatus comprising:
the data acquisition module is used for acquiring data to be mined;
the response module is used for responding to a service request of a user, pre-calculating the data to be mined according to the service request, obtaining an intermediate result and storing the intermediate result; judging whether a mining analysis result corresponding to the service request exists in a stored mining analysis result set according to the service request;
and the result acquisition module is used for carrying out mining analysis on the data to be mined according to the intermediate result when the pre-calculation is finished if not, so as to obtain mining analysis results corresponding to the service request and display the mining analysis results.
9. A computer-readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method of any of the preceding claims 1-7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of the preceding claims 1-7 when executing the program.
CN202311057065.5A 2023-08-18 2023-08-18 Data mining method and device based on pre-calculation and storage medium Pending CN117033467A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311057065.5A CN117033467A (en) 2023-08-18 2023-08-18 Data mining method and device based on pre-calculation and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311057065.5A CN117033467A (en) 2023-08-18 2023-08-18 Data mining method and device based on pre-calculation and storage medium

Publications (1)

Publication Number Publication Date
CN117033467A true CN117033467A (en) 2023-11-10

Family

ID=88644751

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311057065.5A Pending CN117033467A (en) 2023-08-18 2023-08-18 Data mining method and device based on pre-calculation and storage medium

Country Status (1)

Country Link
CN (1) CN117033467A (en)

Similar Documents

Publication Publication Date Title
CN113574517B (en) Rule compiler engine apparatus, method, and medium for generating a distributed system
US8751620B2 (en) Validating deployment patterns in a networked computing environment
US10042932B2 (en) Analytics based on pipes programming model
JP2019504380A (en) Order clustering method and apparatus and method and apparatus for countering malicious information
CN112583620B (en) Network topology map generation method, device, electronic equipment and storage medium
CN116757278B (en) Training method and device of prediction model, storage medium and electronic equipment
CN109981311B (en) Method and device for realizing graph layout
CN114491536B (en) Code analysis method and device based on knowledge graph
WO2020150001A1 (en) Generating a debugging network for a synchronous digital circuit during compilation of program source code
CN113961797A (en) Resource recommendation method, apparatus, electronic device and readable storage medium
WO2024255306A1 (en) Image reconstruction method and apparatus, device, and storage medium
WO2024212719A1 (en) Prediction method and apparatus, readable storage medium and electronic device
CN117033467A (en) Data mining method and device based on pre-calculation and storage medium
CN117057162A (en) Task execution method and device, storage medium and electronic equipment
JP2018073223A (en) Analysis program, analysis method, and analysis apparatus
TWI662486B (en) Method and device for checking completeness of distributed business processing
US11379468B1 (en) Control flow graph refining via execution data
JP7589260B2 (en) Method and apparatus for processing information - Patents.com
CN115409576A (en) Product recommendation method and device, computer equipment and storage medium thereof
US20230012202A1 (en) Graph computing over micro-level and macro-level views
CN111880776A (en) Hierarchical relationship obtaining method and device and electronic equipment
CN114201568A (en) Information processing method, generating method, device, electronic equipment and storage medium
CN117171401B (en) Query method and device for shortest path in graph data based on hierarchical pre-calculation
US10475111B1 (en) Selecting and configuring metrics for monitoring
US20250005333A1 (en) Machine learning to reduce resources for generating solutions to multi-node problems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination