CN106528795A - Data mining method and apparatus - Google Patents
Data mining method and apparatus Download PDFInfo
- Publication number
- CN106528795A CN106528795A CN201610991856.9A CN201610991856A CN106528795A CN 106528795 A CN106528795 A CN 106528795A CN 201610991856 A CN201610991856 A CN 201610991856A CN 106528795 A CN106528795 A CN 106528795A
- Authority
- CN
- China
- Prior art keywords
- data
- result
- cluster
- mining
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2216/00—Indexing scheme relating to additional aspects of information retrieval not explicitly covered by G06F16/00 and subgroups
- G06F2216/03—Data mining
Abstract
The present invention discloses a data mining method and apparatus that are applicable to a data mining system. The data mining system comprises a first cluster and a second cluster. The first cluster comprises multiple first servers. The second cluster comprises multiple second servers. The first servers are configured with, based on an ILog rule engine, a first mining model. The second servers are configured with, based on a statistical analysis system (SAS), a second mining model. The method comprises the steps of receiving data mining requests comprising request types; classifying the data mining requests; transmitting the data mining requests with the request type of a rapid response type to the first cluster, and performing, by the first servers, data mining processing on data by using the first mining model according to the data mining requests, so as to obtain a first mining result; and transmitting the data mining requests with the data type of a non-rapid response type to the second cluster, and performing, by the second servers, data mining processing on data by using the second mining model according to the data mining requests, so as to obtain a second mining result.
Description
Technical field
The application is related to data mining technology field, more particularly to a kind of data digging method and device.
Background technology
With the development of science and technology, the fast-developing and big data technology of business intelligence is maked rapid progress, big data
Value is increasingly taken seriously, and particularly banking system can accumulate the business number of magnanimity during its routine work is handled
According to carrying out data mining using these big datas, Result can be widely applied to client's marketing, products perfection, wind
The numerous areas such as dangerous management and control, have great importance for core competitiveness is lifted.
Thus, need a kind of implementation that effectively can be excavated to data in real time badly.
The content of the invention
In view of this, the purpose of the application is to provide a kind of data digging method and device, to solve in prior art
The technical problem that effectively data cannot be excavated in real time.
This application provides a kind of data digging method, it is adaptable to which data digging system, the data digging system include
First cluster and the second cluster, first cluster include multiple first servers, and second cluster includes multiple
Two servers, the first server are configured with the first mining model based on ILog regulation engines, and the second server is based on
SAS is configured with the second mining model, and methods described includes：
At least one data mining request is received, in the data mining request, at least includes request type；
Data mining request is classified based on its request type；
Data mining request of the request type for quick response type is transferred to into first cluster, is collected by described first
First server in group is carried out to the data in data source using first mining model based on data mining request
Data mining is processed, and obtains the first Result；
By data type be not quick response type data mining request be transferred to second cluster, by described second
Second server in cluster is entered to the data in data source using second mining model based on data mining request
Row data mining is processed, and obtains the second Result.
Said method, it is preferred that after first Result and second Result is obtained, methods described
Also include：
First Result and second Result are returned.
Said method, it is preferred that after first Result and second Result is obtained, methods described
Also include：
First Result and second Result are stored.
Said method, it is preferred that after first Result and second Result is obtained, methods described
Also include：
First Result and second Result are transferred to into second cluster, by second cluster
In second server first Result and second Result are handed over using second mining model
Fork checking.
Said method, it is preferred that also include：
First mining model is transferred to into second cluster, is utilized by the second server in second cluster
Second mining model carries out model training and checking.
Present invention also provides a kind of data mining device, is connected with data digging system, the data digging system
Including the first cluster and the second cluster, first cluster includes multiple first servers, and second cluster includes many
Individual second server, the first server are configured with the first mining model, the second server based on ILog regulation engines
Second mining model, described device are configured with based on SAS (STATISTICAL ANALYSIS SYSTEM, statistical analysis system)
Including：
Request reception unit, for receiving at least one data mining request, at least includes in the data mining request
Request type；
Requests classification unit, for being classified based on its request type to data mining request；
First transmission unit, is transferred to described first for the data mining request by request type for quick response type
Cluster, by the first server in first cluster based on data mining request using the first mining model logarithm
Data mining is carried out according to the data in source, the first Result is obtained；
Second transmission unit, the data mining request for by data type not being quick response type are transferred to described the
Two clusters, by the second server in second cluster based on data mining request using second mining model pair
Data in data source carry out data mining process, obtain the second Result.
Said apparatus, it is preferred that also include：
As a result returning unit, for after first Result and second Result is obtained, will be described
First Result and second Result are returned.
Said apparatus, it is preferred that also include：
As a result memory element, for first Result and second Result are stored.
Said apparatus, it is preferred that also include：
3rd transmission unit, for after first Result and second Result is obtained, will be described
First Result and second Result are transferred to second cluster, by the second server in second cluster
Cross validation is carried out to first Result and second Result using second mining model.
Said apparatus, it is preferred that also include：
4th transmission unit, for first mining model is transferred to second cluster, by second cluster
In second server carry out model training and checking using second mining model.
From such scheme, a kind of data digging method and device that the application is provided, by by ILog clusters and SAS
Cluster configuration in same system, so as to receive data mining ask when, can according to data mining request request
Type is determining the excavation mode of excavation mode or SAS using Ilog so that the application can be provided simultaneously with the energy of Ilog
The enough data mining characteristic based on expert model and excavations to data model of SAS that quick response is carried out to data mining
And checking etc. data mining duty characteristic, so as to collect two kinds of data mining spies of Ilog and SAS on the basis of originating in identical data
Property, in the case where legacy data mining task disposal ability is not affected, significantly lifted to different response times, different excavations
The response efficiency of complexity task.
Description of the drawings
For the technical scheme being illustrated more clearly that in the embodiment of the present application, below will be to making needed for embodiment description
Accompanying drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the present application, for
For those of ordinary skill in the art, without having to pay creative labor, can be obtaining which according to these accompanying drawings
His accompanying drawing.
Fig. 1 is a kind of flow chart of data digging method that the embodiment of the present application one is provided；
Application exemplary plots of the Fig. 2 for the embodiment of the present application；
Fig. 3 is a kind of flow chart of data digging method that the embodiment of the present application two is provided；
Fig. 4 is a kind of flow chart of data digging method that the embodiment of the present application three is provided；
Fig. 5 is a kind of flow chart of data digging method that the embodiment of the present application four is provided；
Fig. 6 is a kind of partial process view of data digging method that the embodiment of the present application five is provided；
Fig. 7 is a kind of structural representation of data mining device that the embodiment of the present application six is provided；
Fig. 8 is a kind of structural representation of data mining device that the embodiment of the present application seven is provided；
Fig. 9 is a kind of structural representation of data mining device that the embodiment of the present application eight is provided；
Figure 10 is a kind of structural representation of data mining device that the embodiment of the present application nine is provided；
Figure 11 is a kind of structural representation of data mining device that the embodiment of the present application ten is provided.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present application, the technical scheme in the embodiment of the present application is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only some embodiments of the present application, rather than the embodiment of whole.It is based on
Embodiment in the application, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of the application protection.
With reference to Fig. 1, it is a kind of flowchart of data digging method that the embodiment of the present application one is provided, it is adaptable to Fig. 2
Shown data digging system, data digging system are connected with access terminal and data source, as shown in Figure 2.
Wherein, can include in data digging system：First cluster and the second cluster, can include in the first cluster many
Individual first server, can include multiple second servers in the second cluster, first server is configured based on ILog regulation engines
There is the first mining model, the first mining model as quickly can carry out excavating the expert model of response, thus, first to data
It is capable of the data mining demand based on expert model of quick response and deployment user in server based on ILog regulation engines, the
Two servers are configured with the second mining model based on SAS (STATISTICAL ANALYSIS SYSTEM, statistical analysis system),
Second mining model is data model, second server based on SAS can the data mining task higher to complexity carry out
Response.
In the present embodiment, can include having the following steps, realize data mining：
Step 101：Receive at least one data mining request.
Wherein, data mining request characterizes the demand excavated required for user by generating and sending in access terminal,
The request type for characterizing user's request is at least included in each data mining request, type if desired for quick response or
Request type that big data is counted or complexity is higher etc..
Step 102：Data mining request is classified based on its request type.
In the present embodiment, the classification that data mining is asked is referred to, the demand to parsing user in data mining request
Carry out cutting, that is to say, that user generates data mining request by accessing terminal, energy in the data mining request for now generating
Enough characterizing user needs to excavate the data in data source using which kind of mode.
Step 103：Data mining request of the request type for quick response type is transferred to into the first cluster, by the first collection
First server in group carries out data mining to the data in data source using the first mining model based on data mining request
Process, obtain the first Result.
Wherein, after data mining request is transferred to the first cluster, the first cluster can be according to each first server
Present load, determine that one or more first servers carry out data mining, realize the load balance scheduling of data mining.
Step 104：By data type be not quick response type data mining request be transferred to the second cluster, by second
Second server in cluster carries out data digging to the data in data source using the second mining model based on data mining request
Pick is processed, and obtains the second Result.
Wherein, after data mining request is transferred to the second cluster, the second cluster can be according to each second server
Present load, determine that one or more second servers carry out data mining, realize the load balance scheduling of data mining.
That is, in the present embodiment after cutting is carried out according to its request type to data mining request, by difference
The data mining request of type adopts different processing modes, for example：The data mining for needing quick response is asked to be transferred to
First cluster, carries out ageing higher data mining, it would be desirable to which the data mining that data volume is big or complexity demand is higher please
Ask the data mining for being transferred to that the second cluster is more improved or depth is higher.
It should be noted that data source here can be various types of data sources, such as relevant database, Hadoop
Data acquisition system of data base or data file etc..
From such scheme, a kind of data digging method that the embodiment of the present application one is provided, by by ILog clusters with
SAS cluster configurations in same system, so as to receive data mining ask when, can according to data mining request please
Seek type to determine the excavation mode of excavation mode or SAS using Ilog so that the application can be provided simultaneously with Ilog's
The data mining characteristic based on expert model and the digging to data model of SAS of quick response can be carried out to data mining
The characteristic of the data mining duty such as pick and checking, so as to collect two kinds of data minings of Ilog and SAS on the basis of originating in identical data
Characteristic, in the case where legacy data mining task disposal ability is not affected, is significantly lifted to different response times, different diggings
The response efficiency of pick complexity task.
It should be noted that in actual applications, the program code with the methodological function in the present embodiment may operate at
In application server cluster, in application server cluster can contain multiple application servers, data mining can be asked into
Data mining request is carried out processing forward in corresponding first cluster or the second cluster by row response.
And in order to realize load balancing, data mining produced by the access terminal of user request can be sent initially to
In the load-balanced server that application server cluster is connected, balance dispatching commander is carried out by load-balanced server and is forwarded again
To in the application server of suitable application server cluster, and then realize data mining.
In one implementation, with reference to Fig. 3, it is a kind of realization of data digging method that the embodiment of the present application two is provided
Flow chart, after the step 103 and the step 104, methods described can also be comprised the following steps：
Step 105：First Result and second Result are returned.
Specifically, in the present embodiment, first Result and second Result can be returned to user
Access terminal.
In one implementation, with reference to Fig. 4, it is a kind of realization of data digging method that the embodiment of the present application three is provided
Flow chart, after the step 103 and the step 104, methods described can also be comprised the following steps：
Step 106：First Result and second Result are stored.
Specifically, in the present embodiment, can be by first Result and second Result storage to the
In the database storage system of one cluster and the connection of the second cluster.
In one implementation, with reference to Fig. 5, it is that a kind of data digging method that the embodiment of the present application four is provided realizes flow process
Figure, wherein, after the step 103 and the step 104, methods described can also be comprised the following steps：
Step 107：First Result and second Result are transferred to into the second cluster, by the second cluster
In second server first Result and second Result are handed over using second mining model
Fork checking.
That is, the second server in the second cluster is built with the second mining model based on SAS so that second service
Device can carry out the excavation of data model and training checking, thus, in the present embodiment, can obtain the first Result and
The result of the first Result such as expert model result and the second Result such as data model is entered after second Result
Row cross validation.
In one implementation, with reference to Fig. 6, it is a kind of part flow process of data digging method that the embodiment of the present application five is provided
Figure, methods described can also be comprised the following steps：
Step 108：First mining model is transferred in the second cluster, is utilized by the second server in the second cluster
Two mining models carry out model training and checking.
That is, the second server in the second cluster is built with the second mining model based on SAS so that second service
Device can carry out the excavation of data model and training checking, thus, in the present embodiment, can be by first service in the first cluster
First mining model of device such as expert model carries out the training and checking of model in being put into the second cluster, afterwards, the second cluster can
So that model training result is fed back to the first cluster, carry out perfect grade of model and process.
With reference to Fig. 7, be a kind of structural representation of data mining device that the embodiment of the present application six is provided, described device with
Data digging system shown in Fig. 2 is connected, and data mining device is connected with terminal is accessed, data digging system and data source
It is connected.
Wherein, can include in data digging system：First cluster and the second cluster, can include in the first cluster many
Individual first server, can include multiple second servers in the second cluster, first server is configured based on ILog regulation engines
There is the first mining model, the first mining model as quickly can carry out excavating the expert model of response, thus, first to data
It is capable of the data mining demand based on expert model of quick response and deployment user in server based on ILog regulation engines, the
Two servers are configured with the second mining model based on SAS (STATISTICAL ANALYSIS SYSTEM, statistical analysis system),
Second mining model is data model, second server based on SAS can the data mining task higher to complexity carry out
Response.
In the present embodiment, described device can include following structure, realize data mining：
Request reception unit 701, for receiving at least one data mining request, at least wraps in the data mining request
Include request type.
Wherein, data mining request characterizes the demand excavated required for user by generating and sending in access terminal,
The request type for characterizing user's request is at least included in each data mining request, type if desired for quick response or
Request type that big data is counted or complexity is higher etc..
It should be noted that request reception unit 701 can adopt the interface that can carry out data transmission to realize, to connect
Receive the data mining request for accessing that terminal sends.
Requests classification unit 702, for being classified based on its request type to data mining request.
In the present embodiment, the classification that data mining is asked is referred to, the demand to parsing user in data mining request
Carry out cutting, that is to say, that user generates data mining request by accessing terminal, energy in the data mining request for now generating
Enough characterizing user needs to excavate the data in data source using which kind of mode.
It should be noted that requests classification unit 702 can be realized using grader, based on request type by data mining
Request is classified.
First transmission unit 703, it is described for the data mining request that request type is quick response type is transferred to
First cluster, by the first server in first cluster based on data mining request using first mining model
Data mining is carried out to the data in data source, the first Result is obtained.
Wherein, after data mining request is transferred to the first cluster, the first cluster can be according to each first server
Present load, determine that one or more first servers carry out data mining, realize the load balance scheduling of data mining.
It should be noted that the first transmission unit 703 can adopt the interface that can carry out data transmission to realize, to incite somebody to action
Data mining request is transferred to the first cluster.
Second transmission unit 704, the data mining request for by data type not being quick response type are transferred to institute
The second cluster is stated, mould is excavated using described second based on data mining request by the second server in second cluster
Type carries out data mining process to the data in data source, obtains the second Result.
Wherein, after data mining request is transferred to the second cluster, the second cluster can be according to each second server
Present load, determine that one or more second servers carry out data mining, realize the load balance scheduling of data mining.
It should be noted that the second transmission unit 704 can adopt the interface that can carry out data transmission to realize, to incite somebody to action
Data mining request is transferred to the second cluster.
That is, in the present embodiment after cutting is carried out according to its request type to data mining request, by difference
The data mining request of type adopts different processing modes, for example：The data mining for needing quick response is asked to be transferred to
First cluster, carries out ageing higher data mining, it would be desirable to which the data mining that data volume is big or complexity demand is higher please
Ask the data mining for being transferred to that the second cluster is more improved or depth is higher.
It should be noted that data source here can be various types of data sources, such as relevant database, Hadoop
Data acquisition system of data base or data file etc..
From such scheme, a kind of data mining device that the embodiment of the present application six is provided, by by ILog clusters with
SAS cluster configurations in same system, so as to receive data mining ask when, can according to data mining request please
Seek type to determine the excavation mode of excavation mode or SAS using Ilog so that the application can be provided simultaneously with Ilog's
The data mining characteristic based on expert model and the digging to data model of SAS of quick response can be carried out to data mining
The characteristic of the data mining duty such as pick and checking, so as to collect two kinds of data minings of Ilog and SAS on the basis of originating in identical data
Characteristic, in the case where legacy data mining task disposal ability is not affected, is significantly lifted to different response times, different diggings
The response efficiency of pick complexity task.
It should be noted that in actual applications, the program code with the methodological function in the present embodiment may operate at
In application server cluster, in application server cluster can contain multiple application servers, data mining can be asked into
Data mining request is carried out processing forward in corresponding first cluster or the second cluster by row response.
And in order to realize load balancing, data mining produced by the access terminal of user request can be sent initially to
In the load-balanced server that application server cluster is connected, balance dispatching commander is carried out by load-balanced server and is forwarded again
To in the application server of suitable application server cluster, and then realize data mining.
With reference to Fig. 8, it is a kind of structural representation of data mining device that the embodiment of the present application seven is provided, described device is also
Following structure can be included：
As a result returning unit 705, are connected with the first cluster and the second cluster, for obtaining described in the first cluster
After one Result and the second cluster obtain second Result, first Result and described second are excavated
As a result returned.
Specifically, the result returning unit 705 can using with first transmission unit 703 and described second transmission
704 identical coffret of unit, the first Result and the second Result are returned to access terminal.
With reference to Fig. 9, it is a kind of structural representation of data mining device that the embodiment of the present application eight is provided, wherein, it is described
Device can also include following structure：
As a result memory element 706, are connected with the first cluster and the second cluster, and the first cluster is deposited with the second cluster and data
Storage system is connected, and the result memory element 706 is for first Result that obtains the first cluster and the second collection
Second Result that group obtains is stored.
Wherein, the result memory element 706 can be to for example various types of database transmissions of data-storage system
The data-interface of one Result and the second Result.
With reference to Figure 10, it is a kind of structural representation of data mining device that the embodiment of the present application nine is provided, wherein, it is described
Device can also include following structure：
3rd transmission unit 707, is connected with the second cluster, for obtaining first Result and described second
After Result, first Result and second Result are transferred to into second cluster, by described
Second server in two clusters is using second mining model to first Result and second Result
Carry out cross validation.
It should be noted that the 3rd transmission unit 707 can adopt the interface that can carry out data transmission to realize, to incite somebody to action
First Result and the second Result are transferred to the second cluster, carry out intersection by the second server in the second cluster and test
Card.For example, the first Result represents the modeling result of expert model, and the second Result represents the modeling knot of the data evil spirit heart
Really, second server carries out cross one another checking using real data to the result of two models, discovery two is mutually authenticated
Problem and defect that class model is present, in this, as the optimization foundation of two class models, the accuracy of lift scheme.
With reference to Figure 11, it is a kind of structural representation of data mining device that the embodiment of the present application ten is provided, wherein, it is described
Device can also include following structure：
4th transmission unit 708, is connected between the first cluster and the second cluster, for first mining model is passed
It is defeated to second cluster, model training is carried out using second mining model by the second server in second cluster
And checking.
It should be noted that the 4th transmission unit 708 can adopt the interface that can carry out data transmission to realize, by first
The first mining model such as expert model in cluster is transferred to the second cluster, carries out model by the second server in the second cluster
Training and checking.For example, the first mining model (Ilog) can only quickly develop expert model, itself without model training and
Authentication function, and the second mining model (SAS) is, with such function, to therefore, it can for the first mining model to be put into
Two mining models carry out model training and checking.
It should be noted that each embodiment in this specification is described by the way of progressive, each embodiment weight
Point explanation is all difference with other embodiment, between each embodiment identical similar part mutually referring to.
Finally, in addition it is also necessary to explanation, herein, such as first and second or the like relational terms be used merely to by
One entity or operation are made a distinction with another entity or operation, and are not necessarily required or implied these entities or operation
Between there is any this actual relation or order.And, term " including ", "comprising" or its any other variant are anticipated
Covering including for nonexcludability, so that a series of process, method, article or equipment including key elements not only includes that
A little key elements, but also including other key elements being not expressly set out, or also include for this process, method, article or
The intrinsic key element of equipment.In the absence of more restrictions, the key element for being limited by sentence "including a ...", does not arrange
Except also there is other identical element in including the process of the key element, method, article or equipment.
Above a kind of data digging method provided by the present invention and device are described in detail, to disclosed reality
The described above of example is applied, professional and technical personnel in the field is realized or using the present invention.The various of these embodiments are repaiied
Change and will be apparent for those skilled in the art, generic principles defined herein can without departing from
In the case of the spirit or scope of the present invention, realize in other embodiments.Therefore, the present invention is not intended to be limited to this paper institutes
These embodiments shown, and it is to fit to the most wide scope consistent with principles disclosed herein and features of novelty.
Claims (10)
1. a kind of data digging method, it is characterised in that suitable for data digging system, the data digging system includes first
Cluster and the second cluster, first cluster include multiple first servers, and second cluster includes multiple second clothes
Business device, the first server are configured with the first mining model based on ILog regulation engines, and the second server is based on SAS
(STATISTICAL ANALYSIS SYSTEM, statistical analysis system) is configured with the second mining model, and methods described includes：
At least one data mining request is received, in the data mining request, at least includes request type；
Data mining request is classified based on its request type；
Data mining request of the request type for quick response type is transferred to into first cluster, by first cluster
First server based on the data mining request data are carried out to the data in data source using first mining model
Excavation is processed, and obtains the first Result；
By data type be not quick response type data mining request be transferred to second cluster, by second cluster
In second server based on the data mining request line number is entered to the data in data source using second mining model
Process according to excavation, obtain the second Result.
2. method according to claim 1, it is characterised in that excavate first Result and described second is being obtained
As a result, after, methods described also includes：
First Result and second Result are returned.
3. method according to claim 1, it is characterised in that excavate first Result and described second is being obtained
As a result, after, methods described also includes：
First Result and second Result are stored.
4. method according to claim 1, it is characterised in that excavate first Result and described second is being obtained
As a result, after, methods described also includes：
First Result and second Result are transferred to into second cluster, by second cluster
Second server carries out intersection to first Result and second Result using second mining model and tests
Card.
5. method according to claim 1, it is characterised in that also include：
First mining model is transferred to into second cluster, by the second server in second cluster using described
Second mining model carries out model training and checking.
6. a kind of data mining device, it is characterised in that be connected with data digging system, the data digging system includes
One cluster and the second cluster, first cluster include multiple first servers, and second cluster includes multiple second
Server, the first server are configured with the first mining model based on ILog regulation engines, and the second server is based on SAS
(STATISTICAL ANALYSIS SYSTEM, statistical analysis system) is configured with the second mining model, and described device includes：
Request reception unit, for receiving at least one data mining request, at least includes request in the data mining request
Type；
Requests classification unit, for being classified based on its request type to data mining request；
First transmission unit, is transferred to first collection for the data mining request by request type for quick response type
Group, utilizes first mining model to data based on data mining request by the first server in first cluster
Data in source carry out data mining, obtain the first Result；
Second transmission unit, the data mining request for by data type not being quick response type are transferred to second collection
Group, utilizes second mining model to data based on data mining request by the second server in second cluster
Data in source carry out data mining process, obtain the second Result.
7. device according to claim 6, it is characterised in that also include：
As a result returning unit, for after first Result and second Result is obtained, by described first
Result and second Result are returned.
8. device according to claim 6, it is characterised in that also include：
As a result memory element, for first Result and second Result are stored.
9. device according to claim 6, it is characterised in that also include：
3rd transmission unit, for after first Result and second Result is obtained, by described first
Result and second Result are transferred to second cluster, are utilized by the second server in second cluster
Second mining model carries out cross validation to first Result and second Result.
10. device according to claim 6, it is characterised in that also include：
4th transmission unit, for first mining model is transferred to second cluster, by second cluster
Second server carries out model training and checking using second mining model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610991856.9A CN106528795B (en) | 2016-11-10 | 2016-11-10 | Data mining method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610991856.9A CN106528795B (en) | 2016-11-10 | 2016-11-10 | Data mining method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106528795A true CN106528795A (en) | 2017-03-22 |
| CN106528795B CN106528795B (en) | 2023-10-13 |
Family
ID=58351094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610991856.9A Active CN106528795B (en) | 2016-11-10 | 2016-11-10 | Data mining method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106528795B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025288A (en) * | 2017-04-14 | 2017-08-08 | 四川九鼎瑞信软件开发有限公司 | Distributed data digging method and system |
| CN112363831A (en) * | 2020-11-10 | 2021-02-12 | 上海华锐软件有限公司 | Wind control processing method and device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050283459A1 (en) * | 2004-06-22 | 2005-12-22 | Microsoft Corporation | Combining multidimensional expressions and data mining extensions to mine OLAP cubes |
| US20090094174A1 (en) * | 2007-10-09 | 2009-04-09 | International Business Machines Corporation | Method, system and program product for on demand data mining server with dynamic mining models |
| CN103024027A (en) * | 2012-12-07 | 2013-04-03 | 中国电信股份有限公司云计算分公司 | Data mining achieving method and system based on cloud computing |
| CN206209712U (en) * | 2016-11-10 | 2017-05-31 | 中国农业银行股份有限公司 | A kind of data mining device, application server and server cluster |
-
2016
- 2016-11-10 CN CN201610991856.9A patent/CN106528795B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050283459A1 (en) * | 2004-06-22 | 2005-12-22 | Microsoft Corporation | Combining multidimensional expressions and data mining extensions to mine OLAP cubes |
| US20090094174A1 (en) * | 2007-10-09 | 2009-04-09 | International Business Machines Corporation | Method, system and program product for on demand data mining server with dynamic mining models |
| CN103024027A (en) * | 2012-12-07 | 2013-04-03 | 中国电信股份有限公司云计算分公司 | Data mining achieving method and system based on cloud computing |
| CN206209712U (en) * | 2016-11-10 | 2017-05-31 | 中国农业银行股份有限公司 | A kind of data mining device, application server and server cluster |
Non-Patent Citations (1)
| Title |
|---|
| 王春霞;: "基于Web数据挖掘的资源服务系统" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025288A (en) * | 2017-04-14 | 2017-08-08 | 四川九鼎瑞信软件开发有限公司 | Distributed data digging method and system |
| CN112363831A (en) * | 2020-11-10 | 2021-02-12 | 上海华锐软件有限公司 | Wind control processing method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106528795B (en) | 2023-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160335315A1 (en) | System and method for data capture and reporting | |
| CN111787090B (en) | Intelligent treatment platform based on block chain technology | |
| CN107908566A (en) | Automatic test management method, device, terminal device and storage medium | |
| CN106709012A (en) | Method and device for analyzing big data | |
| CN102893275A (en) | Automated social networking graph mining and visualization | |
| KR20200057903A (en) | Artificial intelligence model platform and operation method thereof | |
| CN108038052A (en) | Automatic test management method, device, terminal device and storage medium | |
| CN107872454A (en) | A kind of monitoring of ultra-large type internet platform protection based on security rank threat information and analysis system and method based on big data technology | |
| CN103793399A (en) | Method for integrating information resources of system of politics and law | |
| JP2002032773A (en) | Device and method for processing map data | |
| CN102594607A (en) | Telecommunication application service security test and evaluation universal platform system and method thereof | |
| CN106682206A (en) | Method and system for big data processing | |
| CN106528795A (en) | Data mining method and apparatus | |
| CN206209712U (en) | A kind of data mining device, application server and server cluster | |
| CN104239212B (en) | Query method and device of test case and establishment method and device of test case | |
| CN113326400A (en) | Model evaluation method and system based on depth counterfeit video detection | |
| CN112269324A (en) | Digital exhibition cloud platform data analysis control system | |
| CN107093018A (en) | Communication engineering project information method for visualizing and device based on health model | |
| CN109949175B (en) | User attribute inference method based on collaborative filtering and similarity measurement | |
| CN111581302A (en) | Decision-making assisting system based on data warehouse | |
| CN113449318B (en) | Data classification model training method and device and data classification method and device | |
| CN104462480A (en) | Typicality-based big comment data mining method | |
| CN114548820A (en) | Big data wind control method and server for distance education service | |
| CN101345691A (en) | Method for intelligently generating data analysis report and dispatching through wireless/wired technology | |
| CN116226204A (en) | Scene determination method, device, equipment and storage medium based on joint learning platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |