CN108958726A - A kind of method and apparatus of data mining - Google Patents
A kind of method and apparatus of data mining Download PDFInfo
- Publication number
- CN108958726A CN108958726A CN201710356569.5A CN201710356569A CN108958726A CN 108958726 A CN108958726 A CN 108958726A CN 201710356569 A CN201710356569 A CN 201710356569A CN 108958726 A CN108958726 A CN 108958726A
- Authority
- CN
- China
- Prior art keywords
- script
- data processing
- public
- template
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/33—Intelligent editors
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Stored Programmes (AREA)
Abstract
The embodiment of the present invention provides a kind of method and apparatus of data mining, by defining Script Basics template and designing and developing interface, the sentence that need to often repeat to write is preset in advance in Script Basics template, hardware and software platform and the interface of data mining are realized with this, it the data mining time for reducing developer simultaneously, reduces data mining threshold, improve efficiency.The method of a kind of data mining of the embodiment of the present invention, comprising: define Script Basics template and design and develop interface；The Script Basics template includes the corresponding public head of scripting language and public tail；The parameter Suitable content and data processing statement of public head and public tail are inputted in the development interface；The parameter Suitable content and the data processing statement are fused to the Script Basics template to form executable script, script test is carried out to the executable script.
Description
Technical field
The present invention relates to computer field more particularly to a kind of method and apparatus of data mining.
Background technique
Enter big data era along with internet, the source type of data is varied, carry out data mining when to original
The cleaning of beginning data, processing, processing, excavates the more and more common of change at parsing, more and more important.The skill of existing data mining
Art scheme are as follows: the investigation of data requirements；Data are extracted to be observed；Write the script of data processing；Trial operation script verifies number
According to processing result；Script test is carried out, the online deployment of rear application script is passed through；Configuration schedules task, timing execute foot on line
This.
In realizing process of the present invention, at least there are the following problems in the prior art for inventor's discovery:
(1) height is required to the technical capability of the developer of data mining, developer there should be good reason to business
Solution, has certain program capability again；
(2) the cost is relatively high for data mining: development process complexity repeats, low efficiency, poor operability.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of method and apparatus of data mining, by defining Script Basics mould
Plate and interface is designed and developed, the sentence that need to often repeat to write is preset in advance in Script Basics template, number is realized with this
Hardware and software platform and interface according to exploitation, while reducing the data mining time of developer, it reduces data mining threshold, mention
High efficiency.
To achieve the above object, according to an aspect of an embodiment of the present invention, a kind of method of data mining is provided.
The method of a kind of data mining of the embodiment of the present invention, comprising: define Script Basics template and design and develop interface；
The Script Basics template includes the corresponding public head of scripting language and public tail；Inputted in the development interface public head and
The parameter Suitable content and data processing statement of public tail；The parameter Suitable content and the data processing statement are merged
To the Script Basics template to form executable script, script test is carried out to the executable script.
Optionally, the method also includes the online deployment of the application executable scripts, can described in setting after application passes through
The execution time of perform script, and the executable script is executed according to setting time.
Optionally, the method also includes the incoming task title in development interface, using the task names as can hold
The script name for sheet of traveling far and wide；The task names are for uniquely identifying task.
It optionally, further include the parameter Suitable content and data that public head and public tail are inputted in the development interface
After processing sentence: carrying out data verification to the data processing statement.
Optionally, carrying out data verification to the data processing statement includes: calling actuator, and data processing statement is defeated
Enter to actuator and carries out data verification.
Optionally, when the parameter of adjustment in need in public head and public tail, the attribute value of identical parameters is to develop boundary
Subject to the input in face.
Optionally, the data processing statement is SQL statement, and the actuator is SQL actuator.
To achieve the above object, according to another aspect of an embodiment of the present invention, a kind of device of data mining is provided.
A kind of device of data mining of the embodiment of the present invention, comprising: template definition and interface module, for defining
Script Basics template simultaneously designs and develops interface；The Script Basics template includes the corresponding public head of scripting language and public tail；
Parameter and data processing statement input module, for being inputted in the development interface in the parameter adjustment of public head and public tail
Appearance and data processing statement；Script test module, for merging the parameter Suitable content and the data processing statement
To the Script Basics template to form executable script, script test is carried out to the executable script.
Optionally, described device further include: online deployment and scheduling configuration module can be held described in online deployment for applying
This is traveled far and wide, application sets the execution time of the executable script after passing through, and executes the executable foot according to setting time
This.
Optionally, described device further includes task names input module:, will for the incoming task title in development interface
Script name of the task names as executable script；The task names are for uniquely identifying task.
Optionally, described device further includes Data Verification module: for carrying out data verification to the data processing statement.
Optionally, the Data Verification module is also used to: call actuator, by data processing statement be input to actuator into
Row data verification.
Optionally, when the parameter of adjustment in need in public head and public tail, the attribute value of identical parameters is to develop boundary
Subject to the input in face.
Optionally, the data processing statement is SQL statement, and the actuator is SQL actuator.
To achieve the above object, according to an embodiment of the present invention in another aspect, providing a kind of electronic equipment.
The a kind of electronic equipment of the embodiment of the present invention, comprising: one or more processors；Storage device, for storing one
A or multiple programs, when one or more of programs are executed by one or more of processors, so that one or more
The method that a processor realizes a kind of data mining of the embodiment of the present invention.
To achieve the above object, according to an embodiment of the present invention in another aspect, providing a kind of computer-readable medium.
A kind of computer-readable medium of the embodiment of the present invention, is stored thereon with computer program, and described program is processed
A kind of method of data mining of the embodiment of the present invention is realized when device executes.
According to the technique and scheme of the present invention, one embodiment in foregoing invention has the following advantages that or the utility model has the advantages that leads to
It crosses definition Script Basics template and designs and develops interface, the sentence that need to often repeat to write is preset to Script Basics template in advance
In, hardware and software platform and the interface of data mining are realized with this, while reducing the data mining time of developer, reduce
Data mining threshold improves efficiency；It directly will be through hardware and software platform and interfaceization treated script carry out portion in script deployment
Administration does not need to do the efficiency for improving script deployment across multi-platform or software processing.
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment
With explanation.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is the schematic diagram of the key step of data mining method according to an embodiment of the present invention；
Fig. 2 is the schematic diagram of the main modular of data mining device according to an embodiment of the present invention；
Fig. 3 is the composition schematic diagram of development interface according to an embodiment of the present invention；
Fig. 4 is the structural schematic diagram for being suitable for the computer system of the electronic equipment to realize the embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
In the embodiment of the present invention, data mining method before improving realizes a kind of data mining of convenient and efficient
The solution of hardware and software platform.When carrying out data mining, it is necessary first to carry out data requirements investigation, and extract data and seen
It examines, carries out data mining processing according still further to data mining method of the present invention later.
Fig. 1 is the schematic diagram of the key step of the data mining method of the embodiment of the present invention.
As shown in Figure 1, the method for the data mining of the embodiment of the present invention mainly includes the following steps:
Step S11: it defines Script Basics template and designs and develops interface；The Script Basics template includes scripting language pair
The public head and public tail answered.Wherein, public head and public tail refer to do data mining write when script commonly used head and
Commonly used head and tail is fixed up the content as Script Basics template by tail, saves and requires when writing script every time
Operation one time.Public head may include the introducing of some common mathematical function packets, such as: C language Plays input-output function draws
Enter: #include<stdoi.h>, system function introduces in Python: import sys.Public tail is similar to public head,
Also it may include the introducing (the function packet of introducing is depending on specific programming) of some common mathematical function packets, but in Script Basics template
What public tail was not required, by these sentences to should be used as if having commonly used sentence in used scripting language
Public tail.Python is a kind of explanation type computer programming language of object-oriented.
It may include the default setting for some parameters being often used in public head and public tail, it is therefore an objective to reduce exploitation
The modification of personnel, but parameter setting is not necessarily, but determine whether to carry out according to actual needs the setting of Common Parameters with
And specifically which parameter is configured.Such as it can be and whether maximum dynamic partition number, output file are compressed, compress mode
Setting etc..With Python for example: set hive.exec.max.dynamic.partitions=100000；
Set hive.exec.max.dynamic.partitions.pernode=10000.Define Script Basics template effect be
The sentence for being frequently necessary to repeat to write is preset in advance in Script Basics template, carries out data mining to reduce developer
Time improves efficiency.
It include that parameter adjustment text box, data processing text box, data verification button and script test are pressed in development interface
Button.The effect of parameter adjustment text box is: when needing to be adjusted the parameter in public head and public tail, adjusting in parameter
Related content is inputted in text box, in subsequent progress script test, parameter adjusted can be fused in Script Basics template,
Subject to input of the attribute value of identical parameters according to parameter adjustment text box, such as Python script, by set
Mapred.output.compress=true；Adjustment are as follows: set mapred.output.compress=false；If no
It needs to be adjusted parameter, is not then inputted in parameter adjustment text box.Data processing text box is for input data processing
Sentence, mainly SQL (Structured Query Language, structured query language) sentence, also may include
The sentence of Python processing, such as defined variable, initializing variable create tables of data, more new data table etc..Click exploitation circle
Face data verifies button, can execute the data processing statement in data processing text box, automatically to verify the data processing write
Whether sentence is reasonable, obtains data verification as a result, data verification result is directly shown, even with Excel table, text, interface
It is provided in such a way that data form is shown.
After step S11 completes template definition good person interface, data verification is carried out since step S12 and script is surveyed
Examination.
Step S12: parameter Suitable content and the data processing of public head and public tail are inputted in the development interface
Sentence.Wherein, data verification can be carried out to the data processing statement after input data processing sentence, specifically included: programming
Actuator is called, data processing statement is input to actuator and carries out data verification.By taking SQL statement as an example, actuator is corresponded to
SQL statement is input to SQL actuator (SQL actuator herein is MySQL query engine) with realization pair by SQL actuator
The inquiry of data and the output of query result.If it is improper that data verification does not pass through the data processing statement write, repair
Change data processing statement to executing step S13 again after suitable.MySQL is a kind of relational database management system.
Step S13: by the parameter Suitable content and the data processing statement be fused to the Script Basics template with
Executable script is formed, script test is carried out to the executable script.Because public head and public tail contain the default of parameter
Setting, thus need by development interface parameter Suitable content and newly-increased data processing statement all by being programmed into foot
This foundation forms ultimately forms an executable script to be finely adjusted to Script Basics template.Foot is carried out to executable script
Test result is output to Excel table, is popped up if reporting an error with text mode by this test.
In addition, the application of the executable online deployment of script is carried out in the embodiment of the present invention after the completion of script test execution,
The examination & approval of online deployment application set execution time of the executable script after passing through, and according to setting time execute described in can
Perform script.
In addition, development interface can also include task names text box in the embodiment of the present invention, it is defeated in development interface
Enter task names, task names can uniquely identify task, by programming using the task names as the foot of executable script
This title.Task names are such as: exe_analyze_ip, are a tasks for parsing IP address.
In addition, the execution that script can be performed described in time setting interface setting is executed in the embodiment of the present invention by design
Time, executing time setting interface includes executing time text box, ACK button and close button, executes and fills out in time text box
The data format write are as follows: Minute Hour Day Month Dayofweek (minute hour day month day is per week).User according to
Data format, which is manually entered, executes the time, and executable script is periodically executed according to the time of setting.
The method of data mining through the embodiment of the present invention can be seen that be opened by defining Script Basics template and design
Interface is sent out, the sentence that need to often repeat to write is preset in advance in Script Basics template, the flat of data mining is realized with this
Platform and interface, while reducing the data mining time of developer, it reduces data mining threshold, improve efficiency；
In script deployment directly will through hardware and software platform and interfaceization, treated that script is disposed, do not need to do across multi-platform or software
Processing, improve script deployment efficiency.
Fig. 2 is the schematic diagram of the main modular of data mining device according to an embodiment of the present invention；
As shown in Fig. 2, the data mining device 20 of the embodiment of the present invention specifically includes that template definition and interface module
201, for defining Script Basics template and designing and developing interface；The Script Basics template includes that scripting language is corresponding public
Head and public tail.Parameter and data processing statement input module 202, for inputting public head and public in the development interface
The parameter Suitable content and data processing statement of tail.Script test module 203, for by the parameter Suitable content and described
Data processing statement is fused to the Script Basics template to form executable script, carries out script survey to the executable script
Examination.
Wherein, public head and public tail refer to that doing data mining writes commonly used head and tail when script.Public head can
With the introducing comprising some common mathematical function packets, such as C language #include<stdio.h>, Python import sys,
Import os etc..Public tail is similar to public head, also may include the introducing of some common mathematical function packets, but Script Basics mould
Public tail is not required in plate, if having commonly used sentence in used scripting language that these sentences is corresponding
As public tail.It may include the default setting for some parameters being often used in public head and public tail, such as can be
Whether maximum dynamic partition number, output file are compressed, the setting of compress mode etc..Such as Python script: set
Hive.exec.max.dynamic.partitions=100000；set
Hive.exec.max.dynamic.partitions.pernode=10000；Defining Script Basics template action is often
The sentence for repeating to write is needed to be preset in Script Basics template in advance, to reduce the time that developer carries out data mining,
It improves efficiency.
It wherein, include that parameter adjustment text box, data processing text box, data verification button and script are surveyed in development interface
Try button.The effect of parameter adjustment text box is: when needing to be adjusted the parameter in public head and public tail, in parameter
Related content is inputted in adjustment text box, in subsequent progress script test, parameter adjusted can be fused to Script Basics mould
In plate, subject to input of the attribute value of identical parameters according to parameter adjustment text box, such as Python script, by set
Mapred.output.compress=true；Adjustment are as follows: set mapred.output.compress=false；If no
It needs to be adjusted parameter, is not then inputted in parameter adjustment text box.Data processing text box is for input data processing
Sentence, mainly SQL statement also may include the sentence of some Python processing, such as defined variable, initializing variable, wound
Build tables of data, more new data table etc..Development interface data verification button is clicked, can be executed in data processing text box automatically
Data processing statement, it is whether reasonable to verify the data processing statement write, obtain data verification as a result, data verification result with
Excel table, text, interface directly show, even provide in such a way that data form is shown.
In addition, the data mining device of the embodiment of the present invention further includes Data Verification module, for the data processing
Sentence carries out data verification, specifically includes: actuator is called in programming, and data processing statement is input to actuator progress data and is tested
Card.By taking SQL statement as an example, actuator corresponds to SQL actuator, and SQL statement is input to SQL actuator, and (SQL herein is executed
Device is MySQL query engine) to realize to the inquiry of data and the output of query result.It is compiled if data verification does not pass through
The data processing statement write is improper, then modifies data processing statement to carrying out script test again after suitable.
Wherein, script test module 203 is used for the parameter Suitable content and newly-increased data processing language in development interface
Sentence ultimately forms an executable script by being programmed into Script Basics template.Script test is carried out to executable script,
Test result is output to Excel table, is popped up if reporting an error with text mode.Because public head and public tail contain parameter
Default setting, so need by development interface parameter Suitable content and newly-increased data processing statement by being programmed into
To Script Basics template, an executable script is ultimately formed to be finely adjusted to Script Basics template.
In addition, the data mining device of the embodiment of the present invention further includes online deployment and scheduling configuration module, it is used for
After the completion of script test execution, the application of the executable online deployment of script is carried out, the examination & approval of online deployment application are set after passing through
The execution time of the executable script, and the executable script is executed according to setting time.
In addition, the data mining device of the embodiment of the present invention can also include task names input module, for opening
Incoming task title on interface is sent out, by programming using the task names as the script name of executable script.The task
Title can uniquely identify task.Task names are such as: exe_analyze_ip, are a tasks for parsing IP address.
In addition, the execution that script can be performed described in time setting interface setting is executed in the embodiment of the present invention by design
Time, executing time setting interface includes executing time text box, ACK button and close button, executes and fills out in time text box
The data format write are as follows: Minute Hour Day Month Dayofweek (minute hour day month day is per week).User according to
Data format, which is manually entered, executes the time, and executable script is periodically executed according to the time of setting.
From the above, it can be seen that Script Basics template and design and develop interface by defining, need to often repeat volume
The sentence write is preset in advance in Script Basics template, and hardware and software platform and the interface of data mining are realized with this, is reduced simultaneously
It the data mining time of developer, reduces data mining threshold, improve efficiency；It directly will be through flat in script deployment
Treated that script is disposed for platform and interfaceization, does not need to do across multi-platform or software processing, improves script deployment
Efficiency.
Fig. 3 is the composition schematic diagram of the development interface of the embodiment of the present invention.
As shown in figure 3, the composition of the development interface of the embodiment of the present invention includes task names text box, parameter adjustment text
Frame, data processing text box, data verification button, script testing button and the online button of submission.User is in task name text
Incoming task title in frame, input needs the parameter Suitable content of adjusting parameter in parameter adjustment text box, in data processing
Input carries out the SQL statement of data processing in text box；Then click data verifying button carries out SQL statement and writes verifying,
Verification result is provided in the form of Excel；Script testing button is clicked after being verified, and parameter Suitable content and SQL statement are melted
It closes Script Basics template and obtains executable script, executable script is named according to task names and executable script is carried out
Script test, script test result is exported into Excel table and (is popped up if any reporting an error with text mode)；Script test is completed
The laggard online application of this deployment of traveling far and wide, application can arrive the interface for executing time setting after passing through；The interface set between when being executed
It is set according to predetermined format and executes the time, click the ACK button on interface, complete the online deployment of executable script and exited
The interface, executable script can periodically be disposed according to the execution time that user inputs, click close button, then directly exit execution
The interface of time setting.
According to an embodiment of the invention, the present invention also provides a kind of electronic equipment and a kind of computer-readable medium.
Electronic equipment of the invention includes: one or more processors；Storage device, for storing one or more journeys
Sequence, when one or more of programs are executed by one or more of processors, so that one or more of processors are real
A kind of method of data mining of the existing embodiment of the present invention.
Computer-readable medium of the invention is stored thereon with computer program, real when described program is executed by processor
A kind of method of data mining of the existing embodiment of the present invention.
Below with reference to Fig. 4, it illustrates the computer systems 400 being suitable for realize the electronic equipment of the embodiment of the present invention
Structural schematic diagram.Electronic equipment shown in Fig. 4 is only an example, function to the embodiment of the present invention and should not use model
Shroud carrys out any restrictions.
As shown in figure 4, computer system 400 includes central processing unit (CPU) 401, it can be read-only according to being stored in
Program in memory (ROM) 402 or be loaded into the program in random access storage device (RAM) 403 from storage section 408 and
Execute various movements appropriate and processing.In RAM 403, also it is stored with computer system 400 and operates required various programs
And data.CPU 401, ROM 402 and RAM 403 are connected with each other by bus 404.Input/output (I/O) interface 405
It is connected to bus 404.
I/O interface 405 is connected to lower component: the importation 406 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 407 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 408 including hard disk etc.；
And the communications portion 409 of the network interface card including LAN card, modem etc..Communications portion 409 via such as because
The network of spy's net executes communication process.Driver 410 is also connected to I/O interface 405 as needed.Detachable media 411, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 410, in order to read from thereon
Computer program be mounted into storage section 408 as needed.
Particularly, disclosed embodiment, the process of key step figure description above may be implemented as counting according to the present invention
Calculation machine software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable
Computer program on medium, the computer program include the program code for executing method shown in key step figure.?
In such embodiment, which can be downloaded and installed from network by communications portion 409, and/or from can
Medium 411 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 401, system of the invention is executed
The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in module involved in the embodiment of the present invention can be realized by way of software, can also be by hard
The mode of part is realized.Described module also can be set in the processor, for example, can be described as: a kind of processor packet
Include template definition and interface module, parameter and data processing statement input module and script test module.Wherein, these lists
The title of member does not constitute the restriction to the unit itself under certain conditions, for example, template definition and interface module are also
It can be described as " define Script Basics template and design and develop the module at interface ".
As on the other hand, the present invention also provides a kind of computer-readable medium, which be can be
Included in equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying equipment.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the equipment, makes
Obtaining the equipment includes: to define Script Basics template and design and develop interface；The Script Basics template includes that scripting language is corresponding
Public head and public tail；Parameter Suitable content and the data processing of public head and public tail are inputted in the development interface
Sentence；The parameter Suitable content and the data processing statement are fused to the Script Basics template to form executable foot
This, carries out script test to the executable script.
According to the technique and scheme of the present invention, Script Basics template and interface is designed and developed by defining, often needing to repeat
The sentence write is preset in advance in Script Basics template, and hardware and software platform and the interface of data mining are realized with this, is subtracted simultaneously
The data mining time for having lacked developer reduces data mining threshold, improves efficiency；It directly will be through in script deployment
Treated that script is disposed for hardware and software platform and interfaceization, does not need to do across multi-platform or software processing, improves script portion
The efficiency of administration.
Method provided by the embodiment of the present invention can be performed in the said goods, has the corresponding functional module of execution method and has
Beneficial effect.The not technical detail of detailed description in the present embodiment, reference can be made to method provided by the embodiment of the present invention.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (16)
1. a kind of method of data mining characterized by comprising
It defines Script Basics template and designs and develops interface；The Script Basics template include the corresponding public head of scripting language and
Public tail；
The parameter Suitable content and data processing statement of public head and public tail are inputted in the development interface；
The parameter Suitable content and the data processing statement are fused to the Script Basics template to form executable foot
This, carries out script test to the executable script.
2. the method according to claim 1, wherein the method also includes: apply to hold described in online deployment
This is traveled far and wide, application sets the execution time of the executable script after passing through, and executes the executable foot according to setting time
This.
3. the method according to claim 1, wherein the method also includes in development interface incoming tasks
Title, using the task names as the script name of executable script；The task names are for uniquely identifying task.
4. the method according to claim 1, wherein further including inputting public head and public affairs in the development interface
After the parameter Suitable content and data processing statement of cofinal: carrying out data verification to the data processing statement.
5. according to the method described in claim 4, it is characterized in that, including: to data processing statement progress data verification
Actuator is called, data processing statement is input to actuator and carries out data verification.
6. -5 any method according to claim 1, which is characterized in that when adjustment in need in public head and public tail
When parameter, the attribute value of identical parameters is subject to the input of development interface.
7. according to the method described in claim 5, it is characterized in that, the data processing statement is SQL statement, the actuator
For SQL actuator.
8. a kind of device of data mining, which is characterized in that including,
Template definition and interface module, for defining Script Basics template and designing and developing interface；The Script Basics mould
Plate includes the corresponding public head of scripting language and public tail；
Parameter and data processing statement input module, for inputting the parameter tune of public head and public tail in the development interface
Whole content and data processing statement；
Script test module, for the parameter Suitable content and the data processing statement to be fused to the Script Basics mould
Plate carries out script test to form executable script, to the executable script.
9. device according to claim 8, which is characterized in that described device further include: online deployment and scheduling configuration mould
Block, for applying for the online deployment executable script, application sets the execution time of the executable script after passing through, and presses
The executable script is executed according to setting time.
10. device according to claim 8, which is characterized in that described device further includes task names input module: being used for
The incoming task title in development interface, using the task names as the script name of executable script；The task names
For uniquely identifying task.
11. device according to claim 8, which is characterized in that described device further includes Data Verification module: for institute
It states data processing statement and carries out data verification.
12. device according to claim 11, which is characterized in that the Data Verification module is also used to: actuator is called,
Data processing statement is input to actuator and carries out data verification.
13. according to any device of claim 8-12, which is characterized in that when adjustment in need in public head and public tail
Parameter when, the attribute value of identical parameters is subject to the input of development interface.
14. device according to claim 12, which is characterized in that the data processing statement is SQL statement, the execution
Device is SQL actuator.
15. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
The now method as described in any in claim 1-7.
16. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
The method as described in any in claim 1-7 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710356569.5A CN108958726B (en) | 2017-05-19 | 2017-05-19 | Data development method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710356569.5A CN108958726B (en) | 2017-05-19 | 2017-05-19 | Data development method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108958726A true CN108958726A (en) | 2018-12-07 |
| CN108958726B CN108958726B (en) | 2022-01-28 |
Family
ID=64462905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710356569.5A Active CN108958726B (en) | 2017-05-19 | 2017-05-19 | Data development method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108958726B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111291990A (en) * | 2020-02-04 | 2020-06-16 | 浙江大华技术股份有限公司 | Quality monitoring processing method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102164045A (en) * | 2010-02-20 | 2011-08-24 | 微软公司 | Parameterized computer monitoring system |
| CN103135976A (en) * | 2011-11-30 | 2013-06-05 | 阿里巴巴集团控股有限公司 | Code automatic generation method and device |
| US20140033175A1 (en) * | 2012-07-27 | 2014-01-30 | Hon Hai Precision Industry Co., Ltd. | Computing device and method for creating test cases of software program |
| CN103942140A (en) * | 2013-12-19 | 2014-07-23 | 江苏锐天信息科技有限公司 | Automatic testing program conversion method |
| CN104268072A (en) * | 2014-09-03 | 2015-01-07 | 武汉烽火网络有限责任公司 | Graphical script editor and method for fast developing communication equipment testing script |
-
2017
- 2017-05-19 CN CN201710356569.5A patent/CN108958726B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102164045A (en) * | 2010-02-20 | 2011-08-24 | 微软公司 | Parameterized computer monitoring system |
| CN103135976A (en) * | 2011-11-30 | 2013-06-05 | 阿里巴巴集团控股有限公司 | Code automatic generation method and device |
| US20140033175A1 (en) * | 2012-07-27 | 2014-01-30 | Hon Hai Precision Industry Co., Ltd. | Computing device and method for creating test cases of software program |
| CN103942140A (en) * | 2013-12-19 | 2014-07-23 | 江苏锐天信息科技有限公司 | Automatic testing program conversion method |
| CN104268072A (en) * | 2014-09-03 | 2015-01-07 | 武汉烽火网络有限责任公司 | Graphical script editor and method for fast developing communication equipment testing script |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111291990A (en) * | 2020-02-04 | 2020-06-16 | 浙江大华技术股份有限公司 | Quality monitoring processing method and device |
| CN111291990B (en) * | 2020-02-04 | 2023-11-07 | 浙江大华技术股份有限公司 | Quality monitoring processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108958726B (en) | 2022-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109446099A (en) | Automatic test cases generation method, device, medium and electronic equipment | |
| US20210255842A1 (en) | Low-Code Development Platform | |
| CN107861871B (en) | Gray scale publishing method, device, server and storage medium | |
| CN108255653A (en) | The test method and its terminal of a kind of product | |
| CN109558134A (en) | The method and apparatus of business datum verification | |
| CN107506434A (en) | Method and apparatus based on artificial intelligence classification phonetic entry text | |
| CN110166276A (en) | A kind of localization method, device, terminal device and the medium of frame synchronization exception | |
| CN109359194A (en) | Method and apparatus for predictive information classification | |
| CN109992496A (en) | A kind of data processing method and device for automatic test | |
| CN110059011A (en) | Interface test method, device, medium and electronic equipment | |
| CN110134604A (en) | A kind of unit or interface test method, device and computer equipment without code | |
| CN109739526A (en) | Code update method and device | |
| CN109002289A (en) | A kind of method and apparatus constructing data model | |
| CN109284342A (en) | Method and apparatus for output information | |
| CN109992494A (en) | A kind of automatic test execution method and apparatus | |
| CN109947423A (en) | Code generating method, device, terminal and medium | |
| CN108958726A (en) | A kind of method and apparatus of data mining | |
| CN112685291A (en) | System joint test method and related device | |
| CN108733691A (en) | Data preprocessing method and device | |
| CN116795701A (en) | Method and device for generating universal test case of interface program | |
| CN109933727A (en) | User's portrait generation method and system, user's portrait application method and system | |
| CN109977011A (en) | Automatic generation method, device, storage medium and the electronic equipment of test script | |
| US9569187B2 (en) | Irreducible modules | |
| CN109634866A (en) | Data test method, apparatus, medium and electronic equipment | |
| CN112256978B (en) | Data processing method, device and medium based on data model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |