CN111967961B - Data mining method and device - Google Patents
Data mining method and device Download PDFInfo
- Publication number
- CN111967961B CN111967961B CN202010825572.9A CN202010825572A CN111967961B CN 111967961 B CN111967961 B CN 111967961B CN 202010825572 A CN202010825572 A CN 202010825572A CN 111967961 B CN111967961 B CN 111967961B
- Authority
- CN
- China
- Prior art keywords
- data mining
- time
- mining
- instance
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2216/00—Indexing scheme relating to additional aspects of information retrieval not explicitly covered by G06F16/00 and subgroups
- G06F2216/03—Data mining
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Development Economics (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Economics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a data mining method and a device, wherein the method comprises the following steps: deploying the data mining program on all the instances; according to the data mining requirement, configuring a data mining parameter file and a data mining time file; wherein, the data mining time file includes: a data mining time point; monitoring system time, and acquiring real-time load information of all examples when the system time reaches a data mining time point; determining an actual data mining instance according to the real-time load information of all the instances; calling a data mining program in an actual data mining example, and performing data mining by using a data mining parameter file to determine mining information; and determining the report file according to the mining information. The embodiment of the invention realizes the automatic monitoring operation of data mining and the load balancing of the data mining process, and generates report files for business personnel; the accuracy, timeliness and high efficiency of data mining are guaranteed.
Description
Technical Field
The present invention relates to the field of computer data processing technologies, and in particular, to a data mining method and apparatus.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
Because overseas branches of banks are different in national supervision requirements, abnormal login and transaction conditions of clients need to be monitored regularly to meet supervision requirements, and meanwhile, clients are mastered to better serve clients, accurate marketing is achieved according to client preference, and data are required to be mined and analyzed to form a report.
The existing method is that when data mining is needed, a program is manually written in an instance to perform mining, a more idle instance cannot be calculated according to the load balancing condition, online transaction conflict with a user is easy to occur, and customer experience is affected; and the data mining program cannot be automatically operated to automatically mine.
Therefore, how to provide a new solution to the above technical problem is a technical problem to be solved in the art.
Disclosure of Invention
The embodiment of the invention provides a data mining method, which realizes the automatic monitoring operation of data mining and the load balancing of the data mining process, and comprises the following steps:
deploying the data mining program on all the instances;
according to the data mining requirement, configuring a data mining parameter file and a data mining time file; wherein, the data mining time file includes: a data mining time point;
monitoring system time, and acquiring real-time load information of all examples when the system time reaches a data mining time point;
determining an actual data mining instance according to the real-time load information of all the instances;
calling a data mining program in an actual data mining example, and performing data mining by using a data mining parameter file to determine mining information;
and determining the report file according to the mining information.
The embodiment of the invention also provides a data mining device, which comprises:
an instance deployment module for deploying the data mining program on all instances;
the parameter configuration module is used for configuring a data mining parameter file and a data mining time file according to the data mining requirement; wherein, the data mining time file includes: a data mining time point;
the load information acquisition module is used for monitoring the system time and acquiring real-time load information of all the examples when the system time reaches a data mining time point;
the actual data mining instance determining module is used for determining actual data mining instances according to the real-time load information of all the instances;
the mining information determining module is used for calling a data mining program in an actual data mining example, performing data mining by utilizing a data mining parameter file, and determining mining information;
and the report file determining module is used for determining the report file according to the mining information.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the data mining method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for executing the data mining method.
The embodiment of the invention provides a data mining method and a data mining device, wherein first, a data mining program is deployed on all examples; then, configuring a data mining parameter file and a data mining time file according to the data mining requirement; wherein, the data mining time file includes: a data mining time point; monitoring system time, and acquiring real-time load information of all the examples when the system time reaches a data mining time point; further determining an actual data mining instance according to the real-time load information of all the instances; further calling a data mining program in the actual data mining example, and performing data mining by using the data mining parameter file to determine mining information; and finally, determining the report file according to the mining information. According to the embodiment of the invention, the data mining is automatically carried out without manual intervention by configuring the data mining parameter file and the data mining time file and monitoring the system time; the invention also screens the examples needing data mining by acquiring the implementation load information of all the examples, avoids occupying the resources normally accessed by clients, realizes the load balancing of the data mining process, and generates report files for business personnel; the invention realizes the data mining of load balancing and ensures the accuracy, timeliness and high efficiency of the data mining.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
fig. 1 is a schematic diagram of a data mining method according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a big data frame of a data mining method according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of a computer device running a data mining method embodying the present invention.
Fig. 4 is a schematic diagram of a data mining apparatus according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of another embodiment of a data mining apparatus according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
Fig. 1 is a schematic diagram of a data mining method according to an embodiment of the present invention, where as shown in fig. 1, the embodiment of the present invention provides a data mining method, which implements automatic monitoring of data mining and load balancing in a data mining process, and the method includes:
step 101: deploying the data mining program on all the instances;
step 102: according to the data mining requirement, configuring a data mining parameter file and a data mining time file; wherein, the data mining time file includes: a data mining time point;
step 103: monitoring system time, and acquiring real-time load information of all examples when the system time reaches a data mining time point;
step 104: determining an actual data mining instance according to the real-time load information of all the instances;
step 105: calling a data mining program in an actual data mining example, and performing data mining by using a data mining parameter file to determine mining information;
step 106: and determining the report file according to the mining information.
The embodiment of the invention provides a data mining method and a data mining device, wherein first, a data mining program is deployed on all examples; then, configuring a data mining parameter file and a data mining time file according to the data mining requirement; wherein, the data mining time file includes: a data mining time point; monitoring system time, and acquiring real-time load information of all the examples when the system time reaches a data mining time point; further determining an actual data mining instance according to the real-time load information of all the instances; further calling a data mining program in the actual data mining example, and performing data mining by using the data mining parameter file to determine mining information; and finally, determining the report file according to the mining information. According to the embodiment of the invention, the data mining is automatically monitored without manual intervention by configuring the data mining parameter file and the data mining time file and monitoring the system time; the invention also screens the examples needing data mining by acquiring the implementation load information of all the examples, avoids occupying the resources normally accessed by clients, realizes the load balancing of the data mining process, and generates report files for business personnel; the invention realizes the data mining of load balancing and ensures the accuracy, timeliness and high efficiency of the data mining.
In order to realize automatic operation of data mining, the data mining method according to the embodiment of the invention can comprise the following steps:
deploying the data mining program on all the instances; according to the data mining requirement, configuring a data mining parameter file and a data mining time file; wherein, the data mining time file includes: a data mining time point; monitoring system time, and acquiring real-time load information of all examples when the system time reaches a data mining time point; determining an actual data mining instance according to the real-time load information of all the instances; calling a data mining program in an actual data mining example, and performing data mining by using a data mining parameter file to determine mining information; and determining the report file according to the mining information.
In an embodiment of the present invention, the deploying a data mining program on all the examples includes: because the bank system has larger access quantity and higher concurrency, the bank system is deployed by multiple instances and multiple backups, and in order not to influence the normal access of clients, resources are required to be fully utilized in the deployment environments of the multiple instances and the multiple backups, and the data mining program is deployed on all the instances. The mining analysis of overseas client data and the generation of report files can be realized through the report data mining system based on multiple instances.
In an embodiment of the present invention, the configuring a data mining time file according to the data mining requirement includes:
acquiring the time zone in which each instance is located;
according to the data mining requirement, converting the time in the time zone of each instance into Beijing time, determining the data mining time point corresponding to each instance, and writing the data mining time point into a data mining time file.
In an embodiment, configuring a data mining parameter file and a data mining time file according to a data mining requirement includes:
configuring a data mining parameter file according to the data mining requirement; the data mining parameter file at least comprises: data items for data mining, system numbers for mining, coding formats, environmental parameters, and the like.
Configuring a data mining time file according to data mining requirements, including:
acquiring the time zone in which each instance is located; because of different countries and regions established by overseas branches of banks, time zones in which the examples of the branches are located are also different, and in order to uniformly manage time, it is necessary to acquire the time zone in which each example is located.
According to the data mining requirement, converting the time in the time zone of each instance into Beijing time, determining the data mining time point corresponding to each instance, and writing the data mining time point into a data mining time file. The time of the time zone of the example of overseas branches is converted into Beijing time, so that consistency in data mining time can be realized, and unified management is facilitated.
In an embodiment of the present invention, the monitoring system time, when the system time reaches a data mining time point, acquires real-time load information of all instances, including: monitoring the bank system time in real time, and when the system time reaches a data mining time point, sending a detection signal to each instance to acquire real-time load information of all the instances; the real-time load information refers to the situation that the instance occupies resources when running the program, such as CPU occupancy rate, memory usage rate, disk access situation, etc. If the system time does not reach the number of the mining time points, continuing to monitor the system time.
In an embodiment of the present invention, the determining an actual data mining instance according to the real-time load information of all the instances includes:
according to the real-time load information of all the examples, determining the load idle ratio of each example; the load idle ratio is the ratio of the example idle load to the example rated load;
and screening the load idle ratio of each instance according to the set load idle ratio, and determining the instance higher than the set load idle ratio as an actual data mining instance.
In an embodiment, according to real-time load information of all the examples, determining a load idle ratio of each example; the load idle ratio is the ratio of the example idle load to the example rated load; the method comprises the steps of determining the load idle ratio of each instance, wherein the step of determining the load idle ratio of each instance comprises the step of comparing the current idle load of each instance with a rated load; the load idle ratio represents the current load condition of the instance, when the load idle ratio is small, the load idle ratio represents that the tasks running in the instance are more, more resources are occupied, and if the data mining is run on the instance at the moment, the normal access of a client is influenced, and the resources normally accessed by the client are occupied; when the load idle ratio is larger, the running tasks in the instance are fewer, the resource occupation is smaller, and the data mining program can be run in the instance, so that normal access of clients is basically not influenced, and the high efficiency of data mining and the load balancing of the instance are realized.
And screening the load idle ratio of each instance according to the set load idle ratio, and determining the instance higher than the set load idle ratio as an actual data mining instance. The load idle ratio can be set according to the performance of the system and the resource important program in actual operation, for example, the load idle ratio can be set to be 50%, an instance higher than 50% can be used as an actual data mining office instance to perform data mining, and an instance lower than 50% does not perform data mining.
In an embodiment of the present invention, the foregoing invoking a data mining program in an actual data mining instance, performing data mining using a data mining parameter file, and determining mining information includes:
and calling a data mining program in an actual data mining example, importing a data mining parameter file into the data mining program, operating the mining program by using a big data technology to perform data mining, and determining mining information.
Fig. 2 is a schematic diagram of a big data frame of a data mining method according to an embodiment of the present invention, as shown in fig. 2, in the embodiment, after an actual data mining instance is screened out, a data mining program is called in the actual data mining instance, a data mining parameter file is imported into the data mining program, and the mining program is operated to perform data mining by using a big data technology, so as to generate required mining information; the mining program may be run through a big data framework as shown in fig. 2.
In an embodiment of the present invention, the determining, according to the mining information, the report file may include: after the data mining is completed, processing mining information to generate report files; after the report file is generated, the report file needs to be compressed in order to prevent the file from being oversized; after compression is completed, encrypting the file to prevent sensitive content from being leaked; after encryption, the file is backed up; after backup, the files are sent to a reporting system for inspection and use by business personnel.
The embodiment of the invention has the following advantages: automatic monitoring, wherein the system automatically monitors in real time; load balancing calculation, namely calling a data analysis module in a more idle example according to the actual load condition of each example, so as to avoid affecting the use of customers; analyzing the data, namely analyzing and analyzing the relevant fields through a big data technology according to the configuration condition, and generating information required by service personnel; and automatically generating a file, and automatically generating a report file to be issued. The invention automatically searches for the idle examples by utilizing the big data analysis technology to perform data mining, and ensures the accuracy, timeliness and high efficiency of data analysis on the basis of not affecting the use of clients.
Fig. 3 is a schematic diagram of a computer device for running a data mining method according to an embodiment of the present invention, and as shown in fig. 3, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the data mining method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for implementing the data mining method.
The embodiment of the invention also provides a data mining device, which is described in the following embodiment. Because the principle of the device for solving the problem is similar to that of a data mining method, the implementation of the device can refer to the implementation of a data mining method, and the repetition is omitted.
Fig. 4 is a schematic diagram of a data mining apparatus according to an embodiment of the present invention, and as shown in fig. 4, the embodiment of the present invention further provides a data mining apparatus, which may include:
an instance deployment module 401 for deploying the data mining program on all instances;
a parameter configuration module 402, configured to configure a data mining parameter file and a data mining time file according to a data mining requirement; wherein, the data mining time file includes: a data mining time point;
the load information obtaining module 403 is configured to monitor a system time, and obtain real-time load information of all instances when the system time reaches a data mining time point;
an actual data mining instance determining module 404, configured to determine an actual data mining instance according to real-time load information of all instances;
the mining information determining module 405 is configured to invoke a data mining program in an actual data mining instance, perform data mining by using a data mining parameter file, and determine mining information;
the report file determining module 406 is configured to determine a report file according to the mining information.
In an embodiment of the present invention, the foregoing parameter configuration module is specifically configured to:
acquiring the time zone in which each instance is located;
according to the data mining requirement, converting the time in the time zone of each instance into Beijing time, determining the data mining time point corresponding to each instance, and writing the data mining time point into a data mining time file.
In an embodiment of the present invention, the foregoing actual data mining instance determining module is specifically configured to:
according to the real-time load information of all the examples, determining the load idle ratio of each example; the load idle ratio is the ratio of the example idle load to the example rated load;
and screening the load idle ratio of each instance according to the set load idle ratio, and determining the instance higher than the set load idle ratio as an actual data mining instance.
In an embodiment of the present invention, the foregoing mining information determining module is specifically configured to:
and calling a data mining program in an actual data mining example, importing a data mining parameter file into the data mining program, operating the mining program by using a big data technology to perform data mining, and determining mining information.
Fig. 5 is a schematic diagram of another embodiment of a data mining apparatus according to the present invention, and as shown in fig. 5, the present invention further provides another embodiment of a data mining apparatus, including: the system comprises an automatic monitoring module, an excavating time calculation module, a load balancing calculation module, a data excavating module and a file automatic generation module.
And the automatic monitoring module is used for calling the system through a one-key setting command according to the setting of the data mining parameter files such as the coding format, the environment parameters and the like, and the system automatically monitors in real time.
And the analysis time calculation module is used for automatically converting the data analysis running time into Beijing time according to the configured time zone, counting the related time into an intermediate file, automatically reading the time in the file by the system, and judging whether the data mining running time point is reached or not, if not, continuing monitoring by the system, and if so, calling the text data mining module to perform data mining.
And the load balancing calculation module is used for operating the load balancing calculation module when the data analysis operation time point is reached, and calling the data analysis module in a more idle example according to the actual load condition of each example so as to avoid affecting the use of clients.
And the data mining module is used for operating a data mining program by utilizing a big data technology when the data mining operation time point is reached, analyzing and analyzing the relevant fields according to the data mining parameter file, and generating mining information required by service personnel.
The file automatic generation module is used for generating report files to be issued after the data mining is completed, and generating the files successfully; compressing the file to prevent the file from being oversized; after compression is completed, encrypting the file to prevent sensitive content from being leaked; after encryption, the file is backed up; and after backup, the file is sent to a reporting system.
In summary, the data mining method and device provided by the embodiment of the invention firstly, the data mining program is deployed on all the examples; then, configuring a data mining parameter file and a data mining time file according to the data mining requirement; wherein, the data mining time file includes: a data mining time point; monitoring system time, and acquiring real-time load information of all the examples when the system time reaches a data mining time point; further determining an actual data mining instance according to the real-time load information of all the instances; further calling a data mining program in the actual data mining example, and performing data mining by using the data mining parameter file to determine mining information; and finally, determining the report file according to the mining information. According to the embodiment of the invention, the data mining is automatically monitored without manual intervention by configuring the data mining parameter file and the data mining time file and monitoring the system time; the invention also screens the examples needing data mining by acquiring the implementation load information of all the examples, avoids occupying the resources normally accessed by clients, realizes the load balancing of the data mining process, and generates report files for business personnel; the invention realizes the data mining of load balancing and ensures the accuracy, timeliness and high efficiency of the data mining. According to the invention, by utilizing a big data analysis technology, idle examples are automatically searched for data mining, and the accuracy, timeliness and high efficiency of data analysis are ensured on the basis of not affecting the use of clients.
The data analysis is rapidly and accurately carried out by utilizing big data mining while fully utilizing resources in the multi-instance and multi-backup deployment environment, and the data result is generated
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. A method of data mining, comprising:
deploying the data mining program on all the instances;
according to the data mining requirement, configuring a data mining parameter file and a data mining time file; wherein, the data mining time file includes: a data mining time point;
monitoring system time, and acquiring real-time load information of all examples when the system time reaches a data mining time point;
determining an actual data mining instance according to the real-time load information of all the instances;
calling a data mining program in an actual data mining example, performing data mining by using a data mining parameter file, and determining mining information;
determining report files according to the mining information;
determining an actual data mining instance according to the real-time load information of all the instances, wherein the real-time load information comprises the following steps:
according to the real-time load information of all the examples, determining the load idle ratio of each example; the load idle ratio is the ratio of the example idle load to the example rated load;
and screening the load idle ratio of each instance according to the set load idle ratio, and determining the instance higher than the set load idle ratio as an actual data mining instance.
2. The method of claim 1, wherein configuring the data mining time file according to data mining requirements comprises:
acquiring the time zone in which each instance is located;
according to the data mining requirement, converting the time in the time zone of each instance into Beijing time, determining the data mining time point corresponding to each instance, and writing the data mining time point into a data mining time file.
3. The method of claim 1, wherein invoking a data mining program in an actual data mining instance, performing data mining using a data mining parameter file, determining mining information, comprises:
and calling a data mining program in an actual data mining example, importing a data mining parameter file into the data mining program, operating the mining program by using a big data technology to perform data mining, and determining mining information.
4. A data mining apparatus, comprising:
an instance deployment module for deploying the data mining program on all instances;
the parameter configuration module is used for configuring a data mining parameter file and a data mining time file according to the data mining requirement; wherein, the data mining time file includes: a data mining time point;
the load information acquisition module is used for monitoring the system time and acquiring real-time load information of all the examples when the system time reaches a data mining time point;
the actual data mining instance determining module is used for determining actual data mining instances according to the real-time load information of all the instances;
the mining information determining module is used for calling a data mining program in an actual data mining example, performing data mining by utilizing a data mining parameter file, and determining mining information;
the report file determining module is used for determining a report file according to the mining information;
the actual data mining instance determining module is specifically configured to:
according to the real-time load information of all the examples, determining the load idle ratio of each example; the load idle ratio is the ratio of the example idle load to the example rated load;
and screening the load idle ratio of each instance according to the set load idle ratio, and determining the instance higher than the set load idle ratio as an actual data mining instance.
5. The apparatus of claim 4, wherein the parameter configuration module is specifically configured to:
acquiring the time zone in which each instance is located;
according to the data mining requirement, converting the time in the time zone of each instance into Beijing time, determining the data mining time point corresponding to each instance, and writing the data mining time point into a data mining time file.
6. The apparatus of claim 4, wherein the mining information determination module is specifically configured to:
and calling a data mining program in an actual data mining example, importing a data mining parameter file into the data mining program, operating the mining program by using a big data technology to perform data mining, and determining mining information.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 3 when executing the computer program.
8. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program for executing the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010825572.9A CN111967961B (en) | 2020-08-17 | 2020-08-17 | Data mining method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010825572.9A CN111967961B (en) | 2020-08-17 | 2020-08-17 | Data mining method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111967961A CN111967961A (en) | 2020-11-20 |
| CN111967961B true CN111967961B (en) | 2023-08-22 |
Family
ID=73389578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010825572.9A Active CN111967961B (en) | 2020-08-17 | 2020-08-17 | Data mining method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111967961B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7328192B1 (en) * | 2002-05-10 | 2008-02-05 | Oracle International Corporation | Asynchronous data mining system for database management system |
| WO2014206063A1 (en) * | 2013-06-28 | 2014-12-31 | 华为技术有限公司 | Data exchange method and device |
| CN105786956A (en) * | 2016-01-08 | 2016-07-20 | 国家电网公司 | Power business data mining method based on business application system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7802197B2 (en) * | 2005-04-22 | 2010-09-21 | Microsoft Corporation | Adaptive systems and methods for making software easy to use via software usage mining |
-
2020
- 2020-08-17 CN CN202010825572.9A patent/CN111967961B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7328192B1 (en) * | 2002-05-10 | 2008-02-05 | Oracle International Corporation | Asynchronous data mining system for database management system |
| WO2014206063A1 (en) * | 2013-06-28 | 2014-12-31 | 华为技术有限公司 | Data exchange method and device |
| CN105786956A (en) * | 2016-01-08 | 2016-07-20 | 国家电网公司 | Power business data mining method based on business application system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111967961A (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107992409B (en) | Test case generation method and device, computer equipment and storage medium | |
| CN110162471B (en) | Pressure testing method and system based on container cloud | |
| CN111147322B (en) | Test system and method for micro service architecture of 5G core network | |
| Syer et al. | Leveraging performance counters and execution logs to diagnose memory-related performance issues | |
| CN111049705A (en) | Method and device for monitoring distributed storage system | |
| CN109150619B (en) | Fault diagnosis method and system based on network flow data | |
| CN110750458A (en) | Big data platform testing method and device, readable storage medium and electronic equipment | |
| US20160224400A1 (en) | Automatic root cause analysis for distributed business transaction | |
| CN110088744B (en) | Database maintenance method and system | |
| US10657028B2 (en) | Method for replicating production behaviours in a development environment | |
| US11449488B2 (en) | System and method for processing logs | |
| US10929258B1 (en) | Method and system for model-based event-driven anomalous behavior detection | |
| US10394793B1 (en) | Method and system for governed replay for compliance applications | |
| US9405657B2 (en) | Application architecture assessment system | |
| CN114168471A (en) | Test method, test device, electronic equipment and storage medium | |
| CN111651342A (en) | HTTP interface automation testing framework based on Charles recording session | |
| CN111967961B (en) | Data mining method and device | |
| CN113360182B (en) | Method and apparatus for system performance diagnostics | |
| CN115408236A (en) | Log data auditing system, method, equipment and medium | |
| CN113434382A (en) | Database performance monitoring method and device, electronic equipment and computer readable medium | |
| CN113535575A (en) | Benchmark testing method and device for basic environment of software and hardware product | |
| CN113360368B (en) | Method and device for testing software performance | |
| CN116954975A (en) | Fault processing method, system and device for software product and storage medium | |
| CN115190008B (en) | Fault processing method, fault processing device, electronic equipment and storage medium | |
| CN116205461A (en) | Service system peak guarantee method, device, equipment, medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |