CN110309127A - A kind of data processing method, device and electronic equipment - Google Patents
A kind of data processing method, device and electronic equipment Download PDFInfo
- Publication number
- CN110309127A CN110309127A CN201910596822.3A CN201910596822A CN110309127A CN 110309127 A CN110309127 A CN 110309127A CN 201910596822 A CN201910596822 A CN 201910596822A CN 110309127 A CN110309127 A CN 110309127A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- target data
- condition
- handles condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Abstract
This application discloses a kind of data processing method, device and electronic equipments, this method comprises: obtaining the data characteristics of target data to be processed at least one data dimension；Obtain the corresponding object handles condition of target data；Based on data characteristics and object handles condition, the target object for being directed to target data is determined；Target data is handled with target object, so that the treatment effeciency of target data meets object handles condition.The application can carry out the analysis of various dimensions to target data, obtain treatment effeciency with this and meet the target object of object handles condition and handle target data, this can be for the most suitable migration scheme of target data selection.
Description
Technical field
This application involves tables of data migrating technology field more particularly to a kind of data processing methods, device and electronic equipment.
Background technique
With the arrival of big data era, the data volume that user generates is increasing, how it is extracted and be migrated
Just at a urgent problem needed to be solved.
Currently, being used as the technological means of data pick-up and migration has very much, such as Sqoop, Talend and Kettle
Deng.When selecting migration scheme for tables of data to be migrated, usually require it is artificial go to select suitable scheme, than
The size of data volume is such as relied on, but is not only only that the difference of data volume between tables of data, therefore, causes the scheme chosen can
It can not be most suitable migration scheme.
Therefore, the selection accuracy rate to migration scheme is needed to improve.
Summary of the invention
In view of this, the application provides the following technical solutions:
A kind of data processing method, comprising:
Obtain the data characteristics of target data to be processed at least one data dimension；
Obtain the corresponding object handles condition of the target data；
Feature and the object handles condition based on the data determine the target object for being directed to the target data；
The target data is handled with the target object, so that the treatment effeciency of the target data meets
The object handles condition.
Preferably, feature and the object handles condition, determination are directed to the target of the target data based on the data
Object, comprising:
Obtain the corresponding disaggregated model of the object handles condition, wherein using multiple with preset object tag
Disaggregated model described in sample training；
The data characteristics is inputted in the disaggregated model, with output category result；
In at least one corresponding process object of the classification results, the mesh that the object handles condition matches is determined
Mark object.
It is preferably, described to utilize disaggregated model described in multiple sample trainings with preset object tag, comprising:
At least one data sample is obtained, the data sample has data characteristics at least one data dimension,
And the data sample has preset object tag, the object tag characterization is with the corresponding process object of the object tag
The efficiency handled the data sample meets corresponding object handles condition；
The data characteristics and its object tag of sample based on the data carries out the disaggregated model based on decision Tree algorithms
Training.
Preferably, the data dimension, comprising:
The tables of data dimension of the target data, wherein the tables of data dimension include: line number, columns, data type and
One of tables of data source or a variety of dimensions.
Preferably, the treatment effeciency of the target data meets the object handles condition, comprising:
The treatment effeciency of the target data is higher than the target treatment effeciency value in the object handles condition.
A kind of data processing equipment, comprising:
Obtaining unit, for obtaining the data characteristics of target data to be processed at least one data dimension；And it obtains
Obtain the corresponding object handles condition of the target data；
Determination unit determines for feature and the object handles condition based on the data and is directed to the target data
Target object；
Processing unit, for being handled with the target object the target data, so that the target data
Treatment effeciency meet the object handles condition.
A kind of electronic equipment, comprising:
Memory runs generated data for storing application program and the application program；
Processor, for executing the application program, to realize function: obtaining target data to be processed at least one
Data characteristics on data dimension；Obtain the corresponding object handles condition of the target data；Feature and institute based on the data
Object handles condition is stated, determines the target object for being directed to the target data；With the target object to the target data into
Row processing, so that the treatment effeciency of the target data meets the object handles condition.
It can be seen via above technical scheme that the embodiment of the present application provides a kind of data processing method, by obtain to
Data characteristics and corresponding object handles condition of the target data of processing at least one data dimension, to determine for mesh
The target object for marking data, the treatment effeciency satisfaction pair that target data is handled with the target object to realize target data
As treatment conditions.It can be seen that the application can carry out the analysis of various dimensions to target data, treatment effeciency is obtained with this and is met
The target object of object handles condition handles target data, this can be to choose most suitable migration side for target data
Case.
Detailed description of the invention
In order to more clearly explain the technical solutions in the embodiments of the present application, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, the drawings in the following description are only some examples of the present application, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is the system architecture diagram of server cluster disclosed in the embodiment of the present application；
Fig. 2 is the hardware block diagram of electronic equipment disclosed in the embodiment of the present application；
Fig. 3 is the method flow diagram of data processing method disclosed in the embodiment of the present application one；
Fig. 4 is the schematic diagram of tables of data disclosed in the embodiment of the present application；
Fig. 5 is the method flow diagram of data processing method disclosed in the embodiment of the present application two；
Fig. 6 is the method flow diagram of data processing method disclosed in the embodiment of the present application three；
Fig. 7 is the schematic diagram of Decision-Tree Classifier Model disclosed in the embodiment of the present application；
Fig. 8 is the structural schematic diagram of data processing equipment disclosed in the embodiment of the present application；
Fig. 9 is the method flow diagram of data processing method disclosed in the application scene embodiment.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on
Embodiment in the application, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall in the protection scope of this application.
A kind of data processing method provided by the embodiments of the present application can be applied to the server cluster of cloud computing.Fig. 1 is
A kind of system architecture diagram of server cluster provided by the embodiments of the present application, referring to Fig.1, Data Migration can occur in server
Between, for example data move on server 2 by server 1, can also occur at server internal, such as data by disk 1
(not shown in figure 1) moves in 2 (not shown in figure 1) of disk.
It should be noted that above description is only a kind of application scenarios of Data Migration, it is to be understood that for not arranging
Other lifted are related to the electronic equipment of Data Migration, are in the protection scope of the embodiment of the present application.
Hardware block diagram of the Fig. 2 for a kind of electronic equipment provided by the embodiments of the present application, reference Fig. 2, electronic equipment
Hardware configuration may include: memory 11, processor 12, communication interface 13 and communication bus 14；
In the embodiment of the present application, the quantity of memory 11, processor 12, communication interface 13 and communication bus 14 is at least
One, and memory 11, processor 12, communication interface 13 complete mutual communication by communication bus 14.
Memory 11 may include high speed RAM memory, it is also possible to further include nonvolatile memory (non-volatile
Memory) etc., a for example, at least magnetic disk storage；Wherein, produced by memory storage application program and application program are run
Data.
Processor 12 may be a central processor CPU, GPU (Graphics Processing Unit, graphics process
Device) specific integrated circuit ASIC (Application Specific Integrated Circuit) or quilt
It is configured to implement one or more integrated circuits etc. of the embodiment of the present invention；Wherein, processor 12 is used for executing application, with
Realize function:
Obtain the data characteristics of target data to be processed at least one data dimension；It is corresponding to obtain target data
Object handles condition；Based on data characteristics and object handles condition, the target object for being directed to target data is determined；With target object
Target data is handled, so that the treatment effeciency of target data meets object handles condition.
The refinement function and extension function of use above program can refer to and be described below.
In a kind of embodiment one of data processing method disclosed in the present application, as shown in figure 3, this method includes following step
It is rapid:
Step S101: the data characteristics of target data to be processed at least one data dimension is obtained.
In the embodiment of the present application, target data to be processed can be stored in the form of tables of data, document etc..It is deposited for difference
Storage form, the data dimension of target data be not also identical.For example, for tables of data, data dimension may include line number,
Columns, data type and tables of data source；For another example, for document, data dimension may include number of characters, paragraph
Number, data type and document source.
For convenience of understanding, the embodiment of the present application is illustrated data characteristics by taking tables of data as an example:
The tables of data dimension of target data can be obtained, which includes line number, columns, data type and data
One of table source or a variety of dimensions.Fig. 4 is a kind of schematic diagram of tables of data provided by the embodiments of the present application, should referring to Fig. 4
The line number of tables of data is " 20 ", columns is " 13 ", data type is " numerical value ", document source is " local disk C ".
Step S102: the corresponding object handles condition of target data is obtained.
In the present embodiment, for different data, the corresponding object handles condition of the data can be preset.And at object
Manage bar part can lay particular emphasis on efficiency, can also lay particular emphasis on accuracy, can also lay particular emphasis on reliability etc..And then based on it is above-mentioned not
The corresponding object handles condition of target data is determined with data and the corresponding relationship of object handles condition.
For convenience of understanding, different types of object handles condition is introduced below:
1) efficiency term, the i.e. condition for treatment effeciency setting, the treatment effeciency indicate Data Migration in the unit time
Data volume.The data volume of Data Migration is bigger in unit time, and treatment effeciency is higher.
2) accuracy condition, the i.e. condition for accuracy setting, the accuracy indicate the number for not losing information after migrating
According to the ratio relative to data before migrating.The data for not losing information after migration are bigger relative to the ratio of data before migrating, quasi-
True property is higher.
3) reliability conditions, the i.e. condition for reliability setting, the reliability indicate to complete the probability of Data Migration.Generally
Rate is that 0 expression can not complete Data Migration, and probability is that Data Migration is surely completed in 1 expression one, and probability has been meant that closer to 1
A possibility that at Data Migration, is bigger, reliability is higher.
It should be noted that above description is only the citing of object handles condition, it is to be understood that other are not arranged
The other kinds of object handles condition lifted is also in the protection scope of the embodiment of the present application.
Step S103: being based on data characteristics and object handles condition, determines the target object for being directed to target data.
In the embodiment of the present application, for different object handles conditions, difference under the conditions of the object handles can be preset
The corresponding object of data characteristics, so that the object handles condition can be met when being handled with the objects on data by realizing.Into
And the corresponding relationship based on above-mentioned different data feature and object, determine mesh under the conditions of target data corresponding object handles
Mark the corresponding target object of data characteristics of data.
For convenience of understanding, continue to be illustrated the determination process of target object by taking efficiency term, tables of data as an example below:
To realize that the treatment effeciency of data meets efficiency term, the corresponding object of different data feature can be preset.
By taking the data characteristics of tables of data includes line number and columns as an example: if line number is in the first range, columns is in the second range,
Corresponding object is JDBC；If line number is in third range, columns is in the 4th range, corresponding object is Sqoop.
It should be noted that above-mentioned first range, the second range can be same or different, third range and the 4th range
Can be same or different, the present embodiment does not limit this, and can be configured according to actual needs.
Step S104: being handled target data with target object, so that the treatment effeciency satisfaction pair of target data
As treatment conditions.
In the embodiment of the present application, target data is migrated with the target object that step S103 is determined.Due to target pair
As if based on determined by data characteristics and object handles condition, therefore, when being migrated with target object to target data,
Target data can satisfy object handles condition.
For convenience of understanding, continue that different types of object handles condition is introduced below:
1) efficiency term includes the limitation requirement of the treatment effeciency of data, such as the processing effect of data in the efficiency term
Rate is higher than specified target treatment effeciency value.At this point, being migrated with the target object that step S103 is determined to target data, mesh
The treatment effeciency of data is marked higher than the target treatment effeciency value in efficiency term.
2) accuracy condition includes the limitation requirement of the accuracy of data in the accuracy condition, for example data is accurate
Property be higher than specified accuracy.At this point, target data is migrated with the target object that step S103 is determined, target data
Accuracy is higher than the target accuracy in efficiency term.
3) reliability conditions include the limitation requirement of the reliability of data in the reliability conditions, for example data is reliable
Property be higher than specified reliability.At this point, target data is migrated with the target object that step S103 is determined, target data
Target reliabilities of the high reliablity in efficiency term.
Data processing method provided by the embodiments of the present application can carry out the analysis of various dimensions to target data, be obtained with this
Treatment effeciency meets the target object of object handles condition and handles target data, this can be to choose for target data
Most suitable migration scheme.
As data characteristics and object handles condition is based on, a kind of realization side of the target object for target data is determined
Formula, the embodiment of the present application two discloses a kind of data processing method, as shown in figure 5, this method comprises the following steps:
Step S201: the data characteristics of target data to be processed at least one data dimension is obtained.
Step S202: the corresponding object handles condition of target data is obtained.
Step S203: the corresponding disaggregated model of object handles condition is obtained, wherein there is preset object mark using multiple
The sample training disaggregated model of label.
In the embodiment of the present application, for different object handles conditions, it is corresponding that the object handles condition can be preset
Disaggregated model.The disaggregated model is that training obtains by the way of supervised learning, specific:
Using the data with preset object tag as training sample, with the data to train classification models to training sample
The prediction result of feature levels off to object tag possessed by training sample as training objective, treats train classification models and is instructed
Practice and generates；Wherein, preset object tag characterize data most suitable migration scheme under the conditions of the object handles.
For convenience of understanding, continue to be illustrated the process of train classification models by taking efficiency term, tables of data as an example below:
To obtain the corresponding disaggregated model of efficiency term, the tables of data of a large amount of training is obtained ahead of time, and for each
Tables of data is performed both by following operation:
Obtain data characteristics of the tables of data at least one data dimension；There is number to this using different migration schemes
It is handled according to the tables of data of feature, meets efficiency term and the highest migration for the treatment of effeciency to choose from different migration schemes
Scheme is the object tag of tables of data calibration characterization process object as process object.
The data characteristics of a large amount of tables of data with object tag is inputted to train classification models, obtains classifying to training
The prediction result of the data characteristics for each tables of data of model output；According to object tag possessed by each tables of data and
The prediction result of the data characteristics of each tables of data calculates the loss function value to train classification models；Letter is lost to minimize
Numerical value is target, updates the parameter to train classification models, obtains final disaggregated model.
It should be noted that can be for any one machine learning algorithm to train classification models in the embodiment of the present application
Model, such as neural network algorithm, for another example logistic regression algorithm etc..
Step S204: data characteristics is inputted in disaggregated model, with output category result.
Step S205: at least one corresponding process object of classification results, determine what object handles condition matched
Target object.
For convenience of understanding, continue to be illustrated the process of determining target object by taking efficiency term as an example below:
It include that object tag used in multiple training is corresponding in the classification results of the corresponding disaggregated model output of efficiency term
The treatment effeciency of process object further determines that treatment effeciency meets the target object of efficiency term from multiple process objects,
For example determine that treatment effeciency is higher than the target object of target treatment effeciency in efficiency term.
Certainly, if it is determined that target object be it is multiple, can by randomly select or choose efficiency it is most high in a manner of from
The final target object processing target data of middle determination one.
Step S206: being handled target data with target object, so that the treatment effeciency satisfaction pair of target data
As treatment conditions.
Data processing method provided by the embodiments of the present application can use machine Learning Theory training for data characteristics
Disaggregated model, obtains treatment effeciency with this and meets the target object of object handles condition and handle target data, this can
To reduce interference from human factor, the accuracy of migration scheme selection is further increased.
As a kind of implementation using multiple sample training disaggregated models with preset object tag, the application
Embodiment three discloses a kind of data processing method, as shown in fig. 6, this method comprises the following steps:
Step S301: the data characteristics of target data to be processed at least one data dimension is obtained.
Step S302: the corresponding object handles condition of target data is obtained.
Step S303: the corresponding disaggregated model of object handles condition is obtained, wherein there is preset object mark using multiple
The process of the sample training disaggregated model of label includes:
At least one data sample is obtained, data sample has data characteristics, and number at least one data dimension
There is preset object tag according to sample, object tag characterization is dealt with objects to data sample so that object tag is corresponding
The efficiency of reason meets corresponding object handles condition；Data characteristics and its object tag based on data sample, to based on decision
The disaggregated model of tree algorithm is trained.
Fig. 7 is a kind of schematic diagram of Decision-Tree Classifier Model provided by the embodiments of the present application, referring to Fig. 7, first creation root
Data sample is placed on root node by node, selects an optimal characteristics and data sample is divided into multiple sons according to this feature
Data sample；If all subdata samples can correctly be classified, leaf node is further constructed, and by all subdatas
Sample is assigned on corresponding leaf node；If part of subdata sample can not correctly be classified, to this part subdata
The new optimal characteristics of samples selection continue to divide and construct corresponding leaf node, and such recurrence carries out, until all subdata samples
Originally can correctly be classified or without until suitable feature.
Each subdata sample standard deviation is assigned on leaf node at this time, that is, is had and clearly classified, this just generates a decision
Tree classification model.The Decision-Tree Classifier Model can be to one group of classifying rules be summarized, by the very big of regularization in data sample
Likelihood function minimizes.
Step S304: data characteristics is inputted in disaggregated model, with output category result.
Step S305: at least one corresponding process object of classification results, determine what object handles condition matched
Target object.
Step S306: being handled target data with target object, so that the treatment effeciency satisfaction pair of target data
As treatment conditions.
Data processing method provided by the embodiments of the present application can use decision Tree algorithms training for point of data characteristics
Class model, obtains treatment effeciency with this and meets the target object of object handles condition and handle target data, this
Interference from human factor is reduced, the accuracy of migration scheme selection is further increased.
Corresponding with above-mentioned data processing method, a kind of data processing equipment is also disclosed in the application, as shown in figure 8, the dress
It sets and includes:
Obtaining unit 101, for obtaining the data characteristics of target data to be processed at least one data dimension；And
Obtain the corresponding object handles condition of target data；
Determination unit 102 determines the target pair for being directed to target data for being based on data characteristics and object handles condition
As；
Processing unit 103, for being handled with target object target data, so that the treatment effeciency of target data
Meet object handles condition.
Optionally, data dimension, comprising:
The tables of data dimension of target data, wherein tables of data dimension includes: that line number, columns, data type and tables of data are come
One of source or a variety of dimensions.
Optionally, the treatment effeciency of target data meets object handles condition, comprising:
The treatment effeciency of target data is higher than the target treatment effeciency value in object handles condition.
Data processing equipment provided by the embodiments of the present application can carry out the analysis of various dimensions to target data, be obtained with this
Treatment effeciency meets the target object of object handles condition and handles target data, this can be to choose for target data
Most suitable migration scheme.
In another embodiment of data processing equipment disclosed in the present application, determination unit 102 be based on data characteristics and
Object handles condition determines the target object for being directed to target data, comprising:
Obtain the corresponding disaggregated model of object handles condition, wherein utilize multiple samples with preset object tag
Train classification models；Data characteristics is inputted in disaggregated model, with output category result；Classification results it is corresponding at least one
In process object, the target object that object handles condition matches is determined.
Data processing equipment provided by the embodiments of the present application can use machine Learning Theory training for data characteristics
Disaggregated model, obtains treatment effeciency with this and meets the target object of object handles condition and handle target data, this can
To reduce interference from human factor, the accuracy of migration scheme selection is further increased.
In another embodiment of data processing equipment disclosed in the present application, determination unit 102 has in advance using multiple
If object tag sample training disaggregated model, comprising:
At least one data sample is obtained, data sample has data characteristics, and number at least one data dimension
There is preset object tag according to sample, object tag characterization is dealt with objects to data sample so that object tag is corresponding
The efficiency of reason meets corresponding object handles condition；Data characteristics and its object tag based on data sample, to based on decision
The disaggregated model of tree algorithm is trained.
Data processing equipment provided by the embodiments of the present application can use decision Tree algorithms training for point of data characteristics
Class model, obtains treatment effeciency with this and meets the target object of object handles condition and handle target data, this
Interference from human factor is reduced, the accuracy of migration scheme selection is further increased.
For convenience of understanding, the application is described in detail by taking the selection of tables of data migration scheme as an example below:
With the arrival of big data era, the data volume for the tables of data that user generates is increasing, how to take out to it
It takes and migrates just into a urgent problem needed to be solved.Currently, being used as the technological means of data pick-up and migration has very much, it is most simple
Single such as JDBC, there are also Sqoop, Talend and Kettle etc..For tables of data selection migration scheme to be migrated
When, usually require it is artificial go to select suitable technology, the data dimension at this moment referred to is relatively simple, for example relies solely on
The size of data volume, and usually ignore the other information of tables of data, the technology chosen so is not often optimal in efficiency.
In order to solve the above problem of tables of data, the embodiment of the present application provides a kind of for intelligent selection Data Migration side
The data processing method of case:
Fig. 9 is the method flow diagram of data processing method provided by the embodiments of the present application, referring to Fig. 9:
Firstly, S401: choosing data characteristics (such as line number, columns, the data type of relevant to Data Migration tables of data
And data source)；
Further, S402: by the number with data characteristics (characteristic value of data characteristics can be different) of each training
Data Migration is carried out using different migration schemes respectively according to table, the highest migration side for the treatment of effeciency is chosen from different migration schemes
Process object of the case as the tables of data, and be the object tag of the tables of data calibration characterization process object；
Further, S403: using a large amount of there is object tag-data characteristics tables of data to divide decision Tree algorithms
Class model is trained to obtain disaggregated model；
Finally, S404: for target matrix to be processed, obtaining the data characteristics of the target matrix, and its is defeated
Enter to obtain the optimal migration scheme of the target matrix into disaggregated model, target matrix is carried out with the optimal migration scheme
Data Migration.
The embodiment of the present application advantage is as follows:
By carrying out the available optimal migration scheme of multi dimensional analysis to tables of data；Using machine Learning Theory from reality
It draws a conclusion in data, reduces the interference of human factor；Reduce cost of decision making.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
For, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is said referring to method part
It is bright.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond scope of the present application.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
The foregoing description of the disclosed embodiments makes professional and technical personnel in the field can be realized or use the application.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the application.Therefore, the application
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (7)
1. a kind of data processing method, comprising:
Obtain the data characteristics of target data to be processed at least one data dimension；
Obtain the corresponding object handles condition of the target data；
Feature and the object handles condition based on the data determine the target object for being directed to the target data；
The target data is handled with the target object, so that described in the treatment effeciency satisfaction of the target data
Object handles condition.
2. according to the method described in claim 1, feature and the object handles condition, determination are directed to described based on the data
The target object of target data, comprising:
Obtain the corresponding disaggregated model of the object handles condition, wherein utilize multiple samples with preset object tag
The training disaggregated model；
The data characteristics is inputted in the disaggregated model, with output category result；
In at least one corresponding process object of the classification results, the target pair that the object handles condition matches is determined
As.
3. according to the method described in claim 2, described utilize described in multiple sample trainings with preset object tag points
Class model, comprising:
At least one data sample is obtained, the data sample has data characteristics, and institute at least one data dimension
Data sample is stated with preset object tag, the object tag characterization is with the corresponding process object of the object tag to institute
It states the efficiency that data sample is handled and meets corresponding object handles condition；
The data characteristics and its object tag of sample based on the data, instructs the disaggregated model based on decision Tree algorithms
Practice.
4. method according to claim 1 or 2, the data dimension, comprising:
The tables of data dimension of the target data, wherein the tables of data dimension includes: line number, columns, data type and data
One of table source or a variety of dimensions.
5. method according to claim 1 or 2, the treatment effeciency of the target data meets the object handles condition,
Include:
The treatment effeciency of the target data is higher than the target treatment effeciency value in the object handles condition.
6. a kind of data processing equipment, comprising:
Obtaining unit, for obtaining the data characteristics of target data to be processed at least one data dimension；And obtain institute
State the corresponding object handles condition of target data；
Determination unit determines the mesh for being directed to the target data for feature and the object handles condition based on the data
Mark object；
Processing unit, for being handled with the target object the target data, so that the place of the target data
Reason efficiency meets the object handles condition.
7. a kind of electronic equipment, comprising:
Memory runs generated data for storing application program and the application program；
Processor, for executing the application program, to realize function: obtaining target data to be processed at least one data
Data characteristics in dimension；Obtain the corresponding object handles condition of the target data；Feature and described right based on the data
As treatment conditions, the target object for being directed to the target data is determined；With the target object to the target data at
Reason, so that the treatment effeciency of the target data meets the object handles condition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910596822.3A CN110309127B (en) | 2019-07-02 | 2019-07-02 | Data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910596822.3A CN110309127B (en) | 2019-07-02 | 2019-07-02 | Data processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110309127A true CN110309127A (en) | 2019-10-08 |
| CN110309127B CN110309127B (en) | 2021-07-16 |
Family
ID=68079006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910596822.3A Active CN110309127B (en) | 2019-07-02 | 2019-07-02 | Data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110309127B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081648A (en) * | 2010-12-20 | 2011-06-01 | 北京航空航天大学 | Case library system and method for supporting complex product advanced manufacture |
| CN106297280A (en) * | 2015-05-22 | 2017-01-04 | 高德软件有限公司 | A kind of information processing method and device |
| CN107887032A (en) * | 2016-09-27 | 2018-04-06 | 中国移动通信有限公司研究院 | A kind of data processing method and device |
| CN108363738A (en) * | 2018-01-19 | 2018-08-03 | 上海电气集团股份有限公司 | A kind of recommendation method of industrial equipment data analysis algorithm |
| CN108470071A (en) * | 2018-03-29 | 2018-08-31 | 联想(北京)有限公司 | A kind of data processing method and device |
| CN108960264A (en) * | 2017-05-19 | 2018-12-07 | 华为技术有限公司 | The training method and device of disaggregated model |
| CN109165249A (en) * | 2018-08-07 | 2019-01-08 | 阿里巴巴集团控股有限公司 | Data processing model construction method, device, server and user terminal |
| CN109508217A (en) * | 2018-10-22 | 2019-03-22 | 郑州云海信息技术有限公司 | A kind of data processing method, device, equipment and medium |
| CN109934249A (en) * | 2018-12-14 | 2019-06-25 | 网易（杭州）网络有限公司 | Data processing method, device, medium and calculating equipment |
-
2019
- 2019-07-02 CN CN201910596822.3A patent/CN110309127B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081648A (en) * | 2010-12-20 | 2011-06-01 | 北京航空航天大学 | Case library system and method for supporting complex product advanced manufacture |
| CN106297280A (en) * | 2015-05-22 | 2017-01-04 | 高德软件有限公司 | A kind of information processing method and device |
| CN107887032A (en) * | 2016-09-27 | 2018-04-06 | 中国移动通信有限公司研究院 | A kind of data processing method and device |
| CN108960264A (en) * | 2017-05-19 | 2018-12-07 | 华为技术有限公司 | The training method and device of disaggregated model |
| CN108363738A (en) * | 2018-01-19 | 2018-08-03 | 上海电气集团股份有限公司 | A kind of recommendation method of industrial equipment data analysis algorithm |
| CN108470071A (en) * | 2018-03-29 | 2018-08-31 | 联想(北京)有限公司 | A kind of data processing method and device |
| CN109165249A (en) * | 2018-08-07 | 2019-01-08 | 阿里巴巴集团控股有限公司 | Data processing model construction method, device, server and user terminal |
| CN109508217A (en) * | 2018-10-22 | 2019-03-22 | 郑州云海信息技术有限公司 | A kind of data processing method, device, equipment and medium |
| CN109934249A (en) * | 2018-12-14 | 2019-06-25 | 网易（杭州）网络有限公司 | Data processing method, device, medium and calculating equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110309127B (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108171280A (en) | A kind of grader construction method and the method for prediction classification | |
| WO2017216980A1 (en) | Machine learning device | |
| CN110196908A (en) | Data classification method, device, computer installation and storage medium | |
| CN109858476B (en) | Tag expansion method and electronic equipment | |
| CN108197668A (en) | The method for building up and cloud system of model data collection | |
| CN107958230A (en) | Facial expression recognizing method and device | |
| CN104615730B (en) | A kind of multi-tag sorting technique and device | |
| CN111723856B (en) | Image data processing method, device, equipment and readable storage medium | |
| CN108319888A (en) | The recognition methods of video type and device, terminal | |
| CN108229658A (en) | The implementation method and device of object detector based on finite sample | |
| CN109685805A (en) | A kind of image partition method and device | |
| CN109062947A (en) | User's portrait tag queries method, equipment and computer readable storage medium | |
| CN110489131B (en) | Gray level user selection method and device | |
| CN113268665A (en) | Information recommendation method, device and equipment based on random forest and storage medium | |
| CN107569851A (en) | On-hook method, apparatus of playing and system | |
| CN109711475B (en) | Method and device for identifying article, vending equipment and computer readable storage medium | |
| CN110309127A (en) | A kind of data processing method, device and electronic equipment | |
| CN110275895A (en) | It is a kind of to lack the filling equipment of traffic data, device and method | |
| CN110286587A (en) | Method, server and the computer readable storage medium of implementation model iteration control | |
| CN111984637B (en) | Missing value processing method and device in data modeling, equipment and storage medium | |
| CN113591881A (en) | Intention recognition method and device based on model fusion, electronic equipment and medium | |
| CN107729476A (en) | A kind of machine data on-line processing method and system | |
| CN107609171A (en) | Data source tracing method and device based on data warehouse | |
| CN107256252A (en) | Third-party multidimensional data migration method and device | |
| CN109308565B (en) | Crowd performance grade identification method and device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |