CN108241706A - A kind of data storage method and device - Google Patents
A kind of data storage method and device Download PDFInfo
- Publication number
- CN108241706A CN108241706A CN201611223523.8A CN201611223523A CN108241706A CN 108241706 A CN108241706 A CN 108241706A CN 201611223523 A CN201611223523 A CN 201611223523A CN 108241706 A CN108241706 A CN 108241706A
- Authority
- CN
- China
- Prior art keywords
- data
- source
- source data
- database
- field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
An embodiment of the present invention provides a kind of data storage method and device, this method obtains the source data for meeting preset source data model first, by the source data typing source database of acquisition；The source data is entered into application database by the default correspondence based on first kind field Yu the second class field；Then data inputting order is obtained；From the application database, the destination application data of the corresponding data inputting order of extraction；Based on the destination application data, generation target evaluation and test data；Most the target evaluates and tests data inputting to evaluation and test database at last.In scheme provided in an embodiment of the present invention, from the acquisition of source data until most evaluating and testing data inputting at last to evaluation and test database, all it is to perform automatically, does not need to that the application data obtained manually is imported into evaluation and test database, ensure that the high efficiency and high-accuracy of data loading.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of data storage method and device.
Background technology
Application database is the database rich in extensive application contents of program specially made for application program,
The characteristics of maximum is exactly comprehensive data content, can be stored in application database application program userspersonal information,
The application datas such as user activity, user gradation information.The provider or operator of application program can utilize above-mentioned answer
With program data generate evaluation and test data, come application program is evaluated and tested, assess and the later stage planning.
At present there are mainly two types of application databases：One kind is manual collection type database, that is, passes through manually mode
Artificial interpolation data, and then make complete application database；Another kind is to crack type database, i.e., by game visitor
Family end is cracked, and so as to extract accurate true application data, then application data is entered into application program
Database.It is applied after program data base, it is artificial to search the generation evaluation and test required application data of data, and use meter
The methods of calculating, counting, which obtains, evaluates and tests data, in the evaluation and test data inputting most obtained at last to evaluation and test database.
In the prior art, the storage process needs for evaluating and testing data are manually operated, and need the application that will manually be obtained
Program data imported into evaluation and test database, causes the warehouse-in efficiency for evaluating and testing data low.
Invention content
The embodiment of the present invention is designed to provide a kind of data storage method and device, to realize the automatic of evaluation and test data
Storage ensures the high efficiency and high-accuracy of data loading.Specific technical solution is as follows：
In order to achieve the above object, in a first aspect, an embodiment of the present invention provides a kind of data storage method, the method packet
It includes：
The source data for meeting preset source data model is obtained, by the source data typing source database of acquisition；
The source data is entered into application program number by the default correspondence based on first kind field Yu the second class field
According to library, wherein, the first kind field is the field in the source database, and the second class field is the application program number
According to the field in library；
Obtain data inputting order；
From the application database, the destination application data of the corresponding data inputting order of extraction；
Based on the destination application data, generation target evaluation and test data；
The target is evaluated and tested into data inputting to evaluation and test database.
Preferably, it is described to obtain the source data for meeting preset source data model, by the source data typing source data of acquisition
The step of library, including：
Acquire the daily record of client operation and/or the daily record of server operation；
Based on preset source data model, from the daily record of acquisition, the source data for meeting preset source data model is obtained；
By the source data typing source database of acquisition.
Preferably, it is described to obtain the source data for meeting preset source data model, by the source data typing source data of acquisition
The step of library, including：
Obtain the source data for meeting preset source data model；
The source data of acquisition is filtered according to preset filtering rule；
The source data being obtained by filtration is entered into source database.
Preferably, the default correspondence based on first kind field Yu the second class field, by the source data typing
The step of to application database, including：
For each second class field, the default correspondence based on first kind field Yu the second class field, judge this
Whether the quantity of the first kind field corresponding to two class fields is more than 1；
If so, recombinate the source data in the corresponding all first kind fields of the second class field, and by the source after recombination
Under data inputting to the second class field；
If not, the source data in the corresponding first kind field of the second class field is directly entered into the second class field
Under.
Preferably, the step of acquisition data inputting order, including：
Receive the instruction that user is inputted by user interface；
The instruction received is combined, obtains data inputting order.
Preferably, it is described that the instruction received is combined, obtain data inputting order the step of, including：
The instruction received is combined, obtains the data inputting order for performing file with script and presenting.
Second aspect, an embodiment of the present invention provides a kind of data loading device, described device includes：
First recording module, for obtaining the source data for meeting preset source data model, by the source data typing of acquisition
Source database；
Second recording module, for the default correspondence based on first kind field Yu the second class field, by the source number
According to being entered into application database, wherein, the first kind field is the field in the source database, the second class word
Section is the field in the application database；
Module is obtained, for obtaining data inputting order；
Extraction module, for from the application database, the target of the corresponding data inputting order of extraction should
Use program data；
Generation module, for being based on the destination application data, generation target evaluation and test data；
Third recording module, for the target to be evaluated and tested data inputting to evaluation and test database.
Preferably, first recording module, including：
Submodule is acquired, for acquiring the daily record of the daily record of client operation and/or server operation；
First obtains submodule, and for being based on preset source data model, from the daily record of acquisition, acquisition meets preset
The source data of source data model；
First typing submodule, for the source data typing source database that will be obtained.
Preferably, first recording module, including：
Second obtains submodule, for obtaining the source data for meeting preset source data model；
Filter submodule, for being filtered according to preset filtering rule to the source data of acquisition；
Second typing submodule, for the source data being obtained by filtration to be entered into source database.
Preferably, second recording module, including：
Judging submodule, for being directed to each second class field, default pair based on first kind field and the second class field
It should be related to, judge whether the quantity of the first kind field corresponding to the second class field is more than 1；
Third typing submodule, for the judging result of the judging submodule in the case of being, recombinate this second
Source data in the corresponding all first kind fields of class field, and the source data after recombination is entered under the second class field；
4th typing submodule, in the case of being no in the judging result of the judging submodule, by second class
Source data in the corresponding first kind field of field is directly entered under the second class field.
Preferably, the acquisition module, including：
Receiving submodule, for receiving the instruction that user is inputted by user interface；
Third obtains submodule, for being combined to the instruction received, obtains data inputting order.
Preferably, the third obtains submodule, is specifically used for：
The instruction received is combined, obtains the data inputting order for performing file with script and presenting.
As seen from the above, it in scheme provided in an embodiment of the present invention, obtains meet preset source data model first
Source data, by the source data typing source database of acquisition；Default correspondence based on first kind field Yu the second class field, will
The source data is entered into application database；Then data inputting order is obtained；From the application database, carry
Take the destination application data of the corresponding data inputting order；Based on the destination application data, generation target is commented
Measured data；Most the target evaluates and tests data inputting to evaluation and test database at last.Compared with prior art, the embodiment of the present invention provides
Scheme in, from the acquisition of source data until most evaluation and test data inputting to evaluation and test database, is all to perform automatically, is not required at last
The application data obtained is manually imported into evaluation and test database, ensure that the high efficiency and high precision of data loading
Rate.
Description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention, for those of ordinary skill in the art, without creative efforts, can be with
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is a kind of the first flow diagram of data storage method provided in an embodiment of the present invention；
Fig. 2 is a kind of second of flow diagram of data storage method provided in an embodiment of the present invention；
Fig. 3 is a kind of the third flow diagram of data storage method provided in an embodiment of the present invention；
Fig. 4 is a kind of the 4th kind of flow diagram of data storage method provided in an embodiment of the present invention；
Fig. 5 is a kind of the 5th kind of flow diagram of data storage method provided in an embodiment of the present invention；
Fig. 6 is a kind of the first structure diagram of data loading device provided in an embodiment of the present invention；
Fig. 7 is a kind of second of structure diagram of data loading device provided in an embodiment of the present invention；
Fig. 8 is a kind of the third structure diagram of data loading device provided in an embodiment of the present invention；
Fig. 9 is a kind of the 4th kind of structure diagram of data loading device provided in an embodiment of the present invention；
Figure 10 is a kind of the 5th kind of structure diagram of data loading device provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without making creative work
Embodiment shall fall within the protection scope of the present invention.
In the following, first to this specification relates to technical term simply introduced.
Data model is one group of well-defined notion set, these concepts describe accurately the data knot of database
Structure, data manipulation and data integrity constraint.Database is established according to data model, thus data model is data
The basis of library system.
The three elements of data model are data structure described herein, data manipulation and integrity constraint.Wherein, number
It is the set of studied object type according to structure, the inside composition and externally contact that it includes data；Data manipulation refers to pair
Various data objects allow the operational set performed in database, it is included in operation object and related operation rules two parts
Hold；Data constraint condition is the set of one group of data integrity rule, it is that data in data model and its contact are had
Restriction and interdependent rule.
It is appreciated that source data model involved in the embodiment of the present invention is the data mould corresponding to source database
Type, it is well known by those skilled in the art that after data model is determined, system can be obtained to meet by the prior art and be determined
Data model data.
Application database is the database rich in extensive application program data for application program generation, such as
Common game database；And application data can include：Application program self information, for example, it is OS Type, interior
Deposit demand etc.；And the operation data that user data and user are generated using application program process, such as userspersonal information,
User activity and user gradation information etc..
In various embodiments of the present invention, involved application program can be that office class application program, social activity are chatted
Its class application program and game etc..For example, application data and application database, can be respectively game data and
Game database.
Data are evaluated and tested, are the data for being evaluated and tested to application program, assessing, being compared with other applications,
Either operator can estimate the later stage market manifestation or progress of application program to the developer of application program by evaluating and testing data
Improve application program.Such as common application program report of accessment and test, wherein multiple evaluation and test items can be included, each evaluate and test in item can be with
The straight of item is respectively evaluated and tested with the evaluation and test data exhibiting such as static images, constant data, document snippet, statistical analysis figure and two-dimensional table
See evaluation result.
It should be noted that in the prior art, since the development teams of current application domain usually regard database
For the corpus separatum completely disengaged with application program, database team detaches with application development team.It is so above-mentioned
Various databases are typically by changing the data in database by hand.
It is described in detail again by the way that example is embodied to the present invention below.
It should be noted that scheme provided in an embodiment of the present invention can be applied to destination server, which corresponds to
The present embodiments relate to source database, application database and evaluation and test database, that is, destination server can be
The database manipulations such as reading and writing are performed in source database, application database and evaluation and test database.
Fig. 1 is a kind of the first structure diagram of data storage method provided in an embodiment of the present invention.
As shown in Figure 1, a kind of data storage method provided in an embodiment of the present invention, including：
S101：The source data for meeting preset source data model is obtained, by the source data typing source database of acquisition.
It is well known by those skilled in the art that need the data institute of typing source database right since source data model determines
Data structure, data manipulation and the integrity constraint answered, on the basis of source data model is determined, above-mentioned destination server
The source data for matching the source data model can be obtained from a large amount of data information by the prior art.
It is appreciated that above-mentioned a large amount of data information can include the data letter that application client generates at runtime
Breath can also include the data information that apps server generates at runtime；In addition, institute in above-mentioned a large amount of data information
Comprising all data not be required for being entered into source database, it is therefore desirable to pre-set source data model, and then
It will be in the data inputting to source database that the source data model be matched.
In embodiments of the present invention, can preset source data model be met to acquire by existing information acquisition system
Source data, and then obtain above-mentioned source data；For example, an information acquisition system, the information acquisition system can specially be configured
In the data generated when directly connecting with the client communication of application program, and being run from each client, above-mentioned source number is acquired
According to.
In addition, due to the presence for the problems such as data storage place disperses, for example, being obtained from each client of application program
When obtaining source data, since same data information may be stored in each client of dispersion, so above-mentioned destination server
A large amount of repetition, redundancy may be included in the source data obtained.
Therefore in embodiments of the present invention, in order to exclude above-mentioned repetition, redundancy；As shown in Fig. 2, above-mentioned met
The source data of preset source data model the step of by the source data typing source database (S101) of acquisition, can include：
S1014：Obtain the source data for meeting preset source data model；
S1015：The source data of acquisition is filtered according to preset filtering rule；
S1016：The source data being obtained by filtration is entered into source database.
It is appreciated that in above-mentioned steps S1014~S1016, destination server, which can be obtained first, meets preset source data
Then the source data of model filters obtained institute's active data by preset filtering rule, remove the data of redundance
The source data being obtained by filtration finally is entered into source database by information, and then ensures there is smaller redundancy in the source database
Degree, so as to reduce the degree of difficulty of the source data in Select input to source database.
For example, destination server obtains all source datas, including data a, b, c, d, a, c, e and f are then finally needed
Being entered into the data of source database can be：Data a, b, c, d, e and f.
It should be noted that in actual application, which can be set pair according to actual needs
The filtering rule answered；Moreover, it is also possible to corresponding information filtering system is configured in above-mentioned destination server, by the information filtering
System, which performs, completes above-mentioned filter operation.
S102：Default correspondence based on first kind field Yu the second class field, application program is entered by source data
Database.Wherein, first kind field is the field in source database, and the second class field is the field in application database.
It is appreciated that in most of databases, " row " of database table are known as " field ", and each field includes a certain special
The information of topic.For example, in " address list " database, these are all rows share in table attributes for " name ", " telephone number ",
So these row are known as " name " field and " telephone number " field.
It should be noted that in source database for store certain class data field and application database in be used for
The field of such data is stored, it may be identical, it is also possible to it is different, for example, there are a certain first kind field " users in source database
Name ", the second class field that the first kind field " user name " is corresponded in application database are similarly " user name "；Source data
There are a certain first kind field " game name " in library, the first kind field " game name " is corresponded in application database
Second class field is " Game Name ".
Certainly, it should be emphasised that, in the process by the data inputting in source database to above application program data base
In, if the data format of the corresponding data format of first kind field the second class field corresponding from the first kind field is different,
Should also format conversion be carried out to the data under the first kind field, to obtain meeting the data of the data under the second class field
Then source data after progress format conversion is entered into application database by form again.
For example, the data format corresponding to data in first kind field is：Data type is Char, maximum data length
For 255 characters, corresponding to the data format corresponding to the data under the second class field of the first kind field is：Data type is
Varchar, maximum data length are 255 characters.Then into the data inputting being about in source database to above application program
During database, can format conversion be carried out to the data extracted from source database first, i.e., it will be under first kind field
Data be converted into：Data type is Varchar, and maximum data length is the data of 255 characters.
In practical applications, same application database may be corresponding with multiple source datas library, each source data
Can all there be the first kind field of corresponding second class field in library, so, on the basis of either method embodiment shown in Fig. 1 or Fig. 2
On, as shown in figure 3, the above-mentioned default correspondence based on first kind field Yu the second class field, the source data is entered into
The step of application database (S102), it can include：
S1021：For each second class field, the default correspondence based on first kind field Yu the second class field is sentenced
Whether the quantity of the first kind field corresponding to the second class field of breaking is more than 1.
If the judging result of step S1021 is yes, step S1022 is performed：It is corresponding all to recombinate the second class field
Source data in first kind field, and the source data after recombination is entered under the second class field.
It should be noted that recombination described herein can refer to that institute's active data by the corresponding second class field is combined to one
It rises, while the source data of repetition should be removed.
For example, application database M corresponds to source database X, source database Y and source database Z, and in source database X
First kind field x, source database Y in first kind field y and source database Z in first kind field z both correspond to apply
The second class field m in program data base M, it is assumed that data 1,2,3 and 4 are stored under first kind field x, under first kind field y
Data 1,6,7 and 9 are stored with, data 5,6,7 and 8 are stored under first kind field z, then are finally entered under the second class field m
Data can be 1,2,3,4,5,6,7,8 and 9.
If the judging result of step S1021 is no, step S1023 is performed：By the corresponding first kind of the second class field
Source data in field is directly entered under the second class field.
For example, the first kind field x in source database X corresponds to the second class field m in application database M, it is false
If being stored with data 1,2,3 and 4 under first kind field x, then the data that are finally entered under the second class field m can be 1,2,3,
4。
It is appreciated that step S102 can be regarded as according to above-mentioned second class field to the first kind field in source database
Under data carry out data hierarchy and recombination, such as according to attribute fields such as data name, type and special topics.
In practical application, above-mentioned first kind field is advised with the default correspondence of the second class field or for recombination
Then, depending on can be according to actual conditions, for different source databases and application database or for different applications
Program, user can define the reformulation rule of oneself needs according to their needs, realize the versatility of the embodiment of the present invention.
For example, corresponding to certain application program, user defines first kind field x corresponding to the second class field m, first kind word
Section y corresponds to the second class field n, and first kind field z equally corresponds to the second class field m.And apply journey corresponding to another
Sequence, user define first kind field o and correspond to the second class field n, first kind word corresponding to the second class field m, first kind field p
Section q equally corresponds to the second class field n.
For another example, for the source data of different data structure, different reformulation rules is formulated, the data of different structure respectively should
Application database is entered into adaptable reformulation rule, so as to reach the integrated requirement of isomeric data.
S103：Obtain data inputting order.
In actual use, which can be pre-set data inputting order, from application program
Before extracting destination application data in database, the data inputting order pre-set is directly transferred.
On the other hand, in order to complete the purpose of downward commentary measured data library logging data according to the actual demand of user,
In the embodiment of the present invention, on the basis of either method embodiment shown in Fig. 1~3, as shown in figure 4, above-mentioned acquisition data inputting
The step of order (S103), it can include：
S1031：Receive the instruction that user is inputted by user interface.
It should be noted that the user interface is configured for the destination server, in the case of a kind of, shown in the user interface
It is shown with multiple optional order buttons, user can click the corresponding order button of selection, then user clicks the instruction of selection and presses
Instruction corresponding to button, the instruction that as above-mentioned user is inputted by user interface, for example, showing instruction in the user interface
Button 1~9, order button 1~9 distinguish corresponding instruction a~j, user click has selected instruction 2,3,5,7, then user by using
The instruction of family interface input is b, c, e and g.
In the case of another, user can directly be inputted by way of voice or word input by user interface
Instruction；For example, there are one text box, text box is arranged with " submission " button for display in the user input interface, user can be with
Corresponding instruction is written in text box, then clicks " submission " button, then user can be written at this time for the destination server
And the instruction submitted is as above-mentioned " instruction that user is inputted by user interface ".
S1032：The instruction received is combined, obtains data inputting order.
It should be noted that it is input by user instruction may be multiple, so need to multiple instruction input by user into
Row combination, to form a complete, sequential order that machine is facilitated to perform.For example, instruction input by user includes：Refer to
1 is enabled, the corresponding application data of extraction A class users；Instruction 2 is extracted when in November, 2016, user used application program
The application data of generation.The data inputting then obtained instructs：Extraction A classes user uses in November, 2016
The application data generated during application program.
In embodiments of the present invention, it is above-mentioned that the instruction received is combined, obtain data inputting order (S1032)
Step can include：
The instruction received is combined, obtains the data inputting order for performing file with script and presenting.
It is well known by those skilled in the art that above-mentioned data inputting order is the journey embodied in the form of script performs file
It during sequence order, can be handled in order to mass, and convenient for the quick exploitation of developer.
S104：From application database, the destination application data of the corresponding data inputting order are extracted.
It represents, which performs the data inputting order, from above application program data base, needed for extraction
Destination application data.
It should be noted that when the data inputting order performs file with script and presents, the data inputting order is performed
It can be understood as performing corresponding script file.It is well known by those skilled in the art that ensure the consistent of data-base content, just
Using one group of operation of database as an entirety come carry out or all successfully complete or all unsuccessfully exit, so
In embodiments of the present invention, transaction schema may be used in the execution pattern of script execution file.In addition, in order to avoid different
Normal terminal can also use breakpoint transmission technology.
Furthermore it is also possible to the control file of the script file is configured to control, so that user can be based on the control
File interrupts the execution of script file at any time, controls the implementation procedure of script file completely.
It is appreciated that above-mentioned steps S1031~1032 and control file herein, for user, to following
The process for evaluating and testing Input of Data data is visual, which is presented to user in a manner of modeling, and can reduce this
The complexity of Data Input Process, and the reusability when evaluation and test database redesigns is improved, reduce evaluation and test database
Development cycle.
S105：Based on destination application data, generation target evaluation and test data.
The application data extracted from application database, a part can evaluate and test number directly as the target
According to certainly, some cannot evaluate and test data directly as target, need through corresponding data statistic analysis, ability
Obtain target evaluation and test data described herein.It will be clear to someone skilled in the art that evaluating and testing data for different targets, use
Different data analysing methods, the embodiment of the present invention not described in detail herein, are only done with following two kinds of targets evaluation and test data and shown
Example property introduction.
User's retention ratio：In internet industry, user begins to use application program within certain time, during by one section
Between after, continue to the user using the application program, be remembered as retaining user, this certain customers accounts for what is Added User at that time
Ratio is user's retention ratio.User's retention ratio embodies the quality of application program and retains the ability of user, is particularly significant
A target evaluation and test data.
The calculation formula of user's retention ratio is generally as follows：
User's retention ratio=Add User middle login user number/Adds User several * 100%；
Wherein, the number that Adds User is in some period, generally first all day, the number of users of new login application；It logs in
Number of users is to current time, at least logged primary number of users after login is applied.
Common user's retention ratio includes：Next day retention ratio, retention ratio on the 3rd, retention ratio on the 7th and the 30th day are retained
Rate.Assuming that it in embodiments of the present invention, for a certain tool-class application program, needs to acquire retention ratio on the 7th, then stays within 7th
The calculation formula for depositing rate is as follows：
Retention ratio on the 7th=(in first day newly-increased user, in the 7th day number of users for also having login of registration)/first
It increases total number of users newly.
Conversion ratio (Take Rates are called Conversions Rates)：Refer in a measurement period, complete to turn
The number of change behavior accounts for the ratio of total number of clicks.Its calculation formula can be：The visit of the corresponding action of conversion ratio=carried out
The amount of asking/total visit capacity；The index meaning of conversion ratio with：Weigh attraction degree of the application content to user.
By taking the payment conversion ratio of game as an example, the calculation formula of the conversion ratio can be：
Conversion ratio=paying customer's sum/game registration total number of users × 100%.
Can be seen that above-mentioned target evaluation and test data from above-mentioned two example may not be able to be directly from application database
In extract, but will it is for statistical analysis to the application data extracted from application database after,
It can obtain.
S106：Target is evaluated and tested into data inputting to evaluation and test database.
It should be noted that the evaluation and test data for being entered into evaluation and test database if necessary are not entered into evaluation and test data successfully
Library should also perform rolling back action.In addition, for any one evaluation and test data, when by the evaluation and test data to evaluation and test database in
During typing, if the evaluation and test database, there are during corresponding evaluation and test data, only specifically the value of evaluation and test data is different, then at this time
The value of the evaluation and test data should be replaced with to the value of obtained evaluation and test data, complete the automatic revision of evaluation and test database.
In the actual use process, it can include in above-mentioned data inputting order and instruct above-mentioned steps S105~S106
The command information of execution, so that the specific implementation procedure of step S105~S106 goes to perform according to the data inputting order.
As seen from the above, in scheme provided in this embodiment, the source number for meeting preset source data model is obtained first
According to by the source data typing source database of acquisition；Default correspondence based on first kind field Yu the second class field, by described in
Source data is entered into application database；Then data inputting order is obtained；From the application database, extraction pair
Answer the destination application data of the data inputting order；Based on the destination application data, generation target evaluation and test number
According to；Most the target evaluates and tests data inputting to evaluation and test database at last.Compared with prior art, scheme provided in this embodiment
In, from the acquisition of source data until most evaluation and test data inputting to evaluation and test database, is all to perform automatically, does not need to be artificial at last
The application data obtained is imported into evaluation and test database, ensure that the high efficiency and high-accuracy of data loading.
On the basis of either method embodiment shown in Fig. 1~4, as shown in figure 5, above-mentioned acquisition meets preset source data
The source data of model the step of by the source data typing source database (S101) of acquisition, can include：
S1011：Acquire the daily record of client operation and/or the daily record of server operation.
The network equipment, system and service routine etc. can all generate one and are the log logouts of (daily record) in running；
All recite the description of the relevant operations such as date, time, user and action per a line daily record.For example, in Windows systems
(a kind of network operating system) is all designed with various journal files, such as application log, security log, system log
Deng.
It is appreciated that client described herein and server are the client and server to application program, example
Such as, for application program " Kingsoft bodyguard ", which is respectively the client and server of " Kingsoft bodyguard ".
Certain log informations have timeliness, so in embodiments of the present invention, destination server can be real-time
The log stream of client operation and/or the log stream of server operation are intercepted, for example, at interval of 1 hour, the destination server
The log stream of client operation of interception and/or the log stream of server operation.
S1012：Based on preset source data model, from the daily record of acquisition, acquisition meets preset source data model
Source data.
S1013：By the source data typing source database of acquisition.
Client and server are to run on one side, log information are generated on one side, so included in log information
The timeliness of source data is higher.
As seen from the above, in scheme provided in this embodiment, the source number for meeting preset source data model is obtained first
According to by the source data typing source database of acquisition；Default correspondence based on first kind field Yu the second class field, by described in
Source data is entered into application database；Then data inputting order is obtained；From the application database, extraction pair
Answer the destination application data of the data inputting order；Based on the destination application data, generation target evaluation and test number
According to；Most the target evaluates and tests data inputting to evaluation and test database at last.Compared with prior art, scheme provided in this embodiment
In, from the acquisition of source data until most evaluation and test data inputting to evaluation and test database, is all to perform automatically, does not need to be artificial at last
The application data obtained is imported into evaluation and test database, ensure that the high efficiency and high-accuracy of data loading.
The embodiment of the present invention is simply introduced below by a specific example.
For a certain application program " Jinshan anti-virus software ", destination server acquisition obtains the server operation of " Jinshan anti-virus software "
Daily record is then based on preset source data model, from the daily record of acquisition, obtains the source number for meeting preset source data model
According to being filtered according still further to preset filtering rule to the source data of acquisition, and the source data being obtained by filtration is entered into source number
According to library.
The destination server default correspondence based on first kind field Yu the second class field again, source data is entered into should
Use program data base.In addition, user has input instruction 1~5 by user interface, which is combined by destination server
One complete data inputting order, then destination server is from application database, the corresponding data inputting life of extraction
The destination application data of order, then based on destination application data, generation target evaluation and test data, the mesh that will finally be generated
Mark evaluates and tests data inputting to evaluation and test database.
Corresponding to embodiment of the method shown in Fig. 1, the embodiment of the present invention additionally provides a kind of data loading device, such as Fig. 6 institutes
Show, described device includes：
For obtaining the source data for meeting preset source data model, the source data of acquisition is recorded for first recording module 110
Enter source database；
Second recording module 120, for the default correspondence based on first kind field Yu the second class field, by the source
Data inputting to application database, wherein, the first kind field be the source database in field, second class
Field is the field in the application database；
Module 130 is obtained, for obtaining data inputting order；
Extraction module 140, for from the application database, extraction to correspond to the target of the data inputting order
Application data；
Generation module 150, for being based on the destination application data, generation target evaluation and test data；
Third recording module 160, for the target to be evaluated and tested data inputting to evaluation and test database.
Corresponding to embodiment of the method shown in Fig. 2, as shown in fig. 7, specifically, first recording module 110, can also wrap
It includes：
Second obtains submodule 1104, for obtaining the source data for meeting preset source data model；
Filter submodule 1105, for being filtered according to preset filtering rule to the source data of acquisition；
Second typing submodule 1106, for the source data being obtained by filtration to be entered into source database.
Corresponding to embodiment of the method shown in Fig. 3, as shown in figure 8, specifically, second recording module 120, can wrap
It includes：
Judging submodule 1201, it is pre- based on first kind field and the second class field for being directed to each second class field
If correspondence, judge whether the quantity of the first kind field corresponding to the second class field is more than 1；
Third typing submodule 1202, in the case where the judging result of the judging submodule 1201 is to be, weighing
Source data in the corresponding all first kind fields of group the second class field, and the source data after recombination is entered into second class
Under field；
4th typing submodule 1203, will in the case of being no in the judging result of the judging submodule 1201
Source data in the corresponding first kind field of second class field is directly entered under the second class field.
Corresponding to embodiment of the method shown in Fig. 4, as shown in figure 9, specifically, the acquisition module 130, can include：
Receiving submodule 1301, for receiving the instruction that user is inputted by user interface；
Third obtains submodule 1302, for being combined to the instruction received, obtains data inputting order.
In practical applications, the third obtains submodule 1302, can be specifically used for：
The instruction received is combined, obtains the data inputting order for performing file with script and presenting.
As seen from the above, in scheme provided in this embodiment, the source number for meeting preset source data model is obtained first
According to by the source data typing source database of acquisition；Default correspondence based on first kind field Yu the second class field, by described in
Source data is entered into application database；Then data inputting order is obtained；From the application database, extraction pair
Answer the destination application data of the data inputting order；Based on the destination application data, generation target evaluation and test number
According to；Most the target evaluates and tests data inputting to evaluation and test database at last.Compared with prior art, scheme provided in this embodiment
In, from the acquisition of source data until most evaluation and test data inputting to evaluation and test database, is all to perform automatically, does not need to be artificial at last
The application data obtained is imported into evaluation and test database, ensure that the high efficiency and high-accuracy of data loading.
Corresponding to embodiment of the method shown in Fig. 5, on the basis of any device embodiment shown in Fig. 6~9, such as Figure 10 institutes
Show, first recording module 110 can include：
Submodule 1101 is acquired, for acquiring the daily record of the daily record of client operation and/or server operation；
First obtains submodule 1102, for being based on preset source data model, from the daily record of acquisition, obtains and meets in advance
If source data model source data；
First typing submodule 1103, for the source data typing source database that will be obtained.
As seen from the above, in scheme provided in this embodiment, the source number for meeting preset source data model is obtained first
According to by the source data typing source database of acquisition；Default correspondence based on first kind field Yu the second class field, by described in
Source data is entered into application database；Then data inputting order is obtained；From the application database, extraction pair
Answer the destination application data of the data inputting order；Based on the destination application data, generation target evaluation and test number
According to；Most the target evaluates and tests data inputting to evaluation and test database at last.Compared with prior art, scheme provided in this embodiment
In, from the acquisition of source data until most evaluation and test data inputting to evaluation and test database, is all to perform automatically, does not need to be artificial at last
The application data obtained is imported into evaluation and test database, ensure that the high efficiency and high-accuracy of data loading.
It should be noted that herein, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any this practical relationship or sequence.Moreover, term " comprising ", "comprising" or its any other variant are intended to
Non-exclusive inclusion, so that process, method, article or equipment including a series of elements not only will including those
Element, but also including other elements that are not explicitly listed or further include as this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
Also there are other identical elements in process, method, article or equipment including the element.
Each embodiment in this specification is described using relevant mode, identical similar portion between each embodiment
Point just to refer each other, and the highlights of each of the examples are difference from other examples.Especially for system reality
For applying example, since it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention
It is interior.
Claims (12)
1. a kind of data storage method, which is characterized in that the method includes：
The source data for meeting preset source data model is obtained, by the source data typing source database of acquisition；
The source data is entered into application data by the default correspondence based on first kind field Yu the second class field
Library, wherein, the first kind field is the field in the source database, and the second class field is the application data
Field in library；
Obtain data inputting order；
From the application database, the destination application data of the corresponding data inputting order of extraction；
Based on the destination application data, generation target evaluation and test data；
The target is evaluated and tested into data inputting to evaluation and test database.
2. according to the method described in claim 1, it is characterized in that, described obtain the source number for meeting preset source data model
According to, the step of by the source data typing source database of acquisition, including：
Acquire the daily record of client operation and/or the daily record of server operation；
Based on preset source data model, from the daily record of acquisition, the source data for meeting preset source data model is obtained；
By the source data typing source database of acquisition.
3. according to the method described in claim 1, it is characterized in that, described obtain the source number for meeting preset source data model
According to, the step of by the source data typing source database of acquisition, including：
Obtain the source data for meeting preset source data model；
The source data of acquisition is filtered according to preset filtering rule；
The source data being obtained by filtration is entered into source database.
It is 4. according to the method described in claim 1, it is characterized in that, described default based on first kind field and the second class field
Correspondence, the step of source data is entered into application database, including：
For each second class field, the default correspondence based on first kind field Yu the second class field judges second class
Whether the quantity of the first kind field corresponding to field is more than 1；
If so, recombinate the source data in the corresponding all first kind fields of the second class field, and by the source data after recombination
It is entered under the second class field；
If not, the source data in the corresponding first kind field of the second class field is directly entered under the second class field.
5. according to the described method of any one of claim 1-4, which is characterized in that the step for obtaining data inputting order
Suddenly, including：
Receive the instruction that user is inputted by user interface；
The instruction received is combined, obtains data inputting order.
6. according to the method described in claim 5, it is characterized in that, described be combined the instruction received, data are obtained
The step of typing order, including：
The instruction received is combined, obtains the data inputting order for performing file with script and presenting.
7. a kind of data loading device, which is characterized in that described device includes：
First recording module, for obtaining the source data for meeting preset source data model, by the source data typing source number of acquisition
According to library；
For the default correspondence based on first kind field Yu the second class field, the source data is recorded for second recording module
Enter to application database, wherein, the first kind field is the field in the source database, and the second class field is
Field in the application database；
Module is obtained, for obtaining data inputting order；
Extraction module, for from the application database, extraction to correspond to the intended application journey of the data inputting order
Ordinal number evidence；
Generation module, for being based on the destination application data, generation target evaluation and test data；
Third recording module, for the target to be evaluated and tested data inputting to evaluation and test database.
8. device according to claim 7, which is characterized in that first recording module, including：
Submodule is acquired, for acquiring the daily record of the daily record of client operation and/or server operation；
First obtains submodule, and for being based on preset source data model, from the daily record of acquisition, acquisition meets preset source number
According to the source data of model；
First typing submodule, for the source data typing source database that will be obtained.
9. device according to claim 7, which is characterized in that first recording module, including：
Second obtains submodule, for obtaining the source data for meeting preset source data model；
Filter submodule, for being filtered according to preset filtering rule to the source data of acquisition；
Second typing submodule, for the source data being obtained by filtration to be entered into source database.
10. device according to claim 7, which is characterized in that second recording module, including：
Judging submodule, for being directed to each second class field, the default corresponding pass based on first kind field with the second class field
System, judges whether the quantity of the first kind field corresponding to the second class field is more than 1；
Third typing submodule, in the case where the judging result of the judging submodule is to be, recombinating the second class word
Source data in the corresponding all first kind fields of section, and the source data after recombination is entered under the second class field；
4th typing submodule, in the case of being no in the judging result of the judging submodule, by the second class field
Source data in corresponding first kind field is directly entered under the second class field.
11. according to the device described in any one of claim 7-10, which is characterized in that the acquisition module, including：
Receiving submodule, for receiving the instruction that user is inputted by user interface；
Third obtains submodule, for being combined to the instruction received, obtains data inputting order.
12. according to the devices described in claim 11, which is characterized in that the third obtains submodule, is specifically used for：
The instruction received is combined, obtains the data inputting order for performing file with script and presenting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611223523.8A CN108241706B (en) | 2016-12-27 | 2016-12-27 | Data warehousing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611223523.8A CN108241706B (en) | 2016-12-27 | 2016-12-27 | Data warehousing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108241706A true CN108241706A (en) | 2018-07-03 |
| CN108241706B CN108241706B (en) | 2020-12-25 |
Family
ID=62701613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611223523.8A Active CN108241706B (en) | 2016-12-27 | 2016-12-27 | Data warehousing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108241706B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101895426A (en) * | 2010-07-20 | 2010-11-24 | 中兴通讯股份有限公司 | Data acquisition method and system in network resource evaluation |
| CN102346775A (en) * | 2011-09-26 | 2012-02-08 | 苏州博远容天信息科技有限公司 | Method for synchronizing multiple heterogeneous source databases based on log |
| CN103914564A (en) * | 2014-04-18 | 2014-07-09 | 山东泰华电讯有限责任公司 | Method and system for updating and warehousing urban pipelines on basis of ArcGIS (geographic information system) |
| CN104503989A (en) * | 2014-12-03 | 2015-04-08 | 中建材国际贸易有限公司 | Method for completing collection of isomerous database |
| WO2015184782A1 (en) * | 2014-10-22 | 2015-12-10 | 中兴通讯股份有限公司 | Data query method and device |
-
2016
- 2016-12-27 CN CN201611223523.8A patent/CN108241706B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101895426A (en) * | 2010-07-20 | 2010-11-24 | 中兴通讯股份有限公司 | Data acquisition method and system in network resource evaluation |
| CN102346775A (en) * | 2011-09-26 | 2012-02-08 | 苏州博远容天信息科技有限公司 | Method for synchronizing multiple heterogeneous source databases based on log |
| CN103914564A (en) * | 2014-04-18 | 2014-07-09 | 山东泰华电讯有限责任公司 | Method and system for updating and warehousing urban pipelines on basis of ArcGIS (geographic information system) |
| WO2015184782A1 (en) * | 2014-10-22 | 2015-12-10 | 中兴通讯股份有限公司 | Data query method and device |
| CN104503989A (en) * | 2014-12-03 | 2015-04-08 | 中建材国际贸易有限公司 | Method for completing collection of isomerous database |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108241706B (en) | 2020-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kim et al. | The emerging role of data scientists on software development teams | |
| Da Costa et al. | A framework for evaluating the results of the szz approach for identifying bug-introducing changes | |
| Hornix | Performance analysis of business processes through process mining | |
| Zerouali et al. | On the diversity of software package popularity metrics: An empirical study of npm | |
| Bellomo et al. | Got technical debt? Surfacing elusive technical debt in issue trackers | |
| Franco-Bedoya et al. | Queso a quality model for open source software ecosystems | |
| Syeed et al. | Evolution of open source software projects: A systematic literature review. | |
| US20050288956A1 (en) | Systems and methods for integrating business process documentation with work environments | |
| Bischofberger et al. | Sotograph–a pragmatic approach to source code architecture conformance checking | |
| Ali et al. | Testing highly complex system of systems: an industrial case study | |
| Becker et al. | Plato: A service oriented decision support system for preservation planning | |
| CN108228441A (en) | A kind of generation method, device and the report of accessment and test of application program report of accessment and test | |
| Verhulst | Evaluating quality of event data within event logs: an extensible framework | |
| Jiang et al. | Towards a framework for understanding the relationships between classical software engineering and agile methodologies | |
| Izquierdo-Cortazar et al. | Towards automated quality models for software development communities: The QualOSS and FLOSSMetrics case | |
| Király | A metadata quality assurance framework | |
| Vidoni | Understanding roxygen package documentation in R | |
| CN111930611B (en) | Statistical method and device for test data | |
| Li | Jira 7 Essentials | |
| Li | Jira 8 Essentials: Effective issue management and project tracking with the latest Jira features | |
| CN108241706A (en) | A kind of data storage method and device | |
| Sneed | Testing web services in the cloud | |
| US20080195453A1 (en) | Organisational Representational System | |
| Meyen et al. | A Data Quality Engineering Framework. | |
| Ferreira et al. | Dependent or not: Detecting and understanding collections of refactorings |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |