CN109669993A - Data processing method, data processing equipment and electronic equipment - Google Patents
Data processing method, data processing equipment and electronic equipment Download PDFInfo
- Publication number
- CN109669993A CN109669993A CN201811563206.XA CN201811563206A CN109669993A CN 109669993 A CN109669993 A CN 109669993A CN 201811563206 A CN201811563206 A CN 201811563206A CN 109669993 A CN109669993 A CN 109669993A
- Authority
- CN
- China
- Prior art keywords
- data
- level
- mapping relations
- adjusted
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Data processing method, data processing equipment and electronic equipment provided by the present application, are related to isomerization technical field of data storage.Wherein, data processing method includes: and carries out grade classification according to the attribute information of isomerization data to be processed to handle to obtain multiple data levels, wherein the isomerization data are made of multi-group data, and the corresponding data volume of each data level is of different sizes；A corresponding mapping relations are generated each described data level, and each group of data for belonging to identical data grade is based on the corresponding mapping relations of the data level and is stored；The data level for needing to be adjusted is judged whether there is, and the corresponding mapping relations of data level being adjusted to needs are adjusted.By the above method, it can improve and be unfavorable for searching after there is storage resource waste or storage when in the prior art storing isomerization data.
Description
Technical field
This application involves isomerization data processing technical field of memory, in particular to a kind of data processing method, number
According to processing unit and electronic equipment.
Background technique
In industrial IoT environment of internet of things, the IoT equipment for being dispersed in each region is usually to belong to some specific rent
Family, and the IoT equipment for belonging to each tenant then may be thousands of.Also, duty parameter (such as equipment machine between distinct device
Tool, electrical and GPS location attribute etc.) number change greatly, have from several to several hundred.Entrance is logged in as end user
SaaS system, need effectively to manage more equipment under these multi-tenant environments, using big data OLAP technology go various dimensions,
Monitor to multi-parameter, count and show these isomerization data.
When facing above-mentioned application demand, two kinds of technical solutions exist in the prior art.For example, a kind of scheme are as follows: if
If using common relational database system, an IoT device type is created that a database table (Table).Therefore, exist
On the basis of each tenant has plurality of devices type, each tenant just has multiple database tables under one's name.It is set according to such
Meter often increases a kind of device type newly and requires one new database table of creation, and the apparent consequence of one done so is exactly such as
Fruit is then developed by the inquiring for dimension with tenant, device type etc. to the operation associated of multiple tables, and be bound to extreme influence
System performance.Alternatively, if concluded to these device type tables, with a big table or several table demappings these
Equipment, then these tables are substantially exactly the intersection of these device attributes, and so, the numerical value of certain column of many rows is certainly
Sky, but Database Systems are had to occupy-place for it, and waste spatially is in turn resulted in.In addition it is known that in a table
If there is too many number of data will affect the performance of system data processing.
Another scheme are as follows: with the NoSQL system towards column storage, such as MongoDB goes to deposit these isomerization data.By
Then towards column design of memory systems, these systems are congenital just to have schema-free advantage, is relatively suitble to storage field elongated
Or unfixed data.But it is placed in the actual environment of this programme, this scheme will deposit is unfavorable for lookup after storing
Problem.
Therefore it provides a kind of can not only waste to avoid storage resource, but also data search can be effectively performed after storing
Technical solution, be a technical problem to be solved urgently.
Summary of the invention
In view of this, the application's is designed to provide a kind of data processing method, data processing equipment and electronic equipment,
There are be unfavorable for lookup after storage resource waste or storage to ask when being stored in the prior art to isomerization data with improving
Topic.
To achieve the above object, the embodiment of the present application adopts the following technical scheme that
A kind of data processing method, comprising:
Grade classification is carried out according to the attribute information of isomerization data to be processed to handle to obtain multiple data levels,
In, the isomerization data are made of multi-group data, and the corresponding data volume of each data level is of different sizes；
A corresponding mapping relations, and each group that will belong to identical data grade are generated each described data level
Data are based on the corresponding mapping relations of the data level and are stored；
The data level for needing to be adjusted is judged whether there is, and the data level being adjusted is corresponding to be reflected to needing
The relationship of penetrating is adjusted.
It is described to judge whether there is needs in above-mentioned data processing method in the embodiment of the present application preferably selects
The data level being adjusted, and the step of corresponding mapping relations of data level being adjusted to needs are adjusted, comprising:
Judge whether there is the data level for needing to be adjusted；
The data level for needing to be adjusted if it exists then judges whether to need to re-start grade classification processing；
If desired grade classification processing is re-started, then corresponding mapping is generated according to the data level repartitioned
Relationship.
In the embodiment of the present application preferably selects, in above-mentioned data processing method, the basis is repartitioned to obtain
Data level generate corresponding mapping relations the step of, specifically:
Corresponding mapping relations are generated according to the data level and consistency hash algorithm repartitioned.
It is described according to isomery to be processed in above-mentioned data processing method in the embodiment of the present application preferably selects
The attribute information for changing data carries out grade classification and handles the step of obtaining multiple data levels, comprising:
For each group of data in isomerization data to be processed, the data that each subdata has in this group of data are obtained
The quantity of type subdata corresponding with each data type, wherein each group of data include multiple subdatas；
Grade classification is carried out according to the quantity of data type subdata corresponding with each data type to handle
To multiple data levels.
It is described according to the data type in above-mentioned data processing method in the embodiment of the present application preferably selects
The quantity of subdata corresponding with each data type carries out grade classification and handles the step of obtaining multiple data levels, specifically
Are as follows:
It is divided according to the data type, the quantity of the corresponding subdata of each data type and a preset quantity
Processing obtains preset quantity data level.
The embodiment of the present application also provides a kind of data processing equipments, comprising:
Data level division module, for carrying out grade classification processing according to the attribute information of isomerization data to be processed
Obtain multiple data levels, wherein the isomerization data are made of multi-group data, the corresponding data volume size of each data level
It is different；
Data memory module for generating a corresponding mapping relations each described data level, and will belong to
The each group of data of identical data grade is based on the corresponding mapping relations of the data level and is stored；
Mapping relations adjust module, need the data level that is adjusted for judging whether there is, and to needing to carry out
The corresponding mapping relations of the data level of adjustment are adjusted.
In the embodiment of the present application preferably selects, in above-mentioned data processing equipment, the mapping relations adjust module
Include:
Level adjustment judging submodule, for judging whether there is the data level for needing to be adjusted；
Grade classification judging submodule needs weight for judging whether when in the presence of the data level for needing to be adjusted
It is new to carry out grade classification processing；
Mapping relations generate submodule, for when needing to re-start grade classification processing, according to repartitioning to obtain
Data level generate corresponding mapping relations.
In the embodiment of the present application preferably selects, in above-mentioned data processing equipment, the mapping relations generate submodule
Block is specifically used for:
Corresponding mapping relations are generated according to the data level and consistency hash algorithm repartitioned.
In the embodiment of the present application preferably selects, in above-mentioned data processing equipment, the data level division module
Include:
Attribute information acquisition submodule, for obtaining the group for each group of data in isomerization data to be processed
The quantity of the data type that each subdata has in data subdata corresponding with each data type, wherein each group of number
According to including multiple subdatas；
Data level divides submodule, for according to data type subdata corresponding with each data type
Quantity carries out grade classification and handles to obtain multiple data levels.
On the basis of the above, it the embodiment of the present application also provides a kind of electronic equipment, including memory, processor and deposits
The computer program that is stored in the memory and can run on the processor, the computer program realize above-mentioned data when running
Processing method.
Data processing method, data processing equipment and electronic equipment provided by the present application, by based on isomerization data
Attribute information obtains corresponding data amount multiple data levels of different sizes, and each group of data base that will belong to identical data grade
It is stored in the corresponding mapping relations of the data level, to realize the difference between each group of data with different data amount
Storage, on the basis of can effectively be searched after guaranteeing storage, moreover it is possible to avoid the problem that storage resource wastes, so as to improve
It is unfavorable for searching after there is storage resource waste or storage when in the prior art storing isomerization data, for
The storage of isomerization data has high practical value.Also, mapping relations can also be adjusted according to demand, to keep away
Exempt to lead to the problem of storage resource waste occur after isomerization data movement to be processed, further increases data processing
The practical value of method.
To enable the above objects, features, and advantages of the application to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
Fig. 1 is the block diagram of electronic equipment provided by the embodiments of the present application.
Fig. 2 is the flow diagram of data processing method provided by the embodiments of the present application.
Fig. 3 is the flow diagram of step S150 in Fig. 2.
Fig. 4 is a kind of effect diagram provided by the embodiments of the present application for re-starting data level division.
Fig. 5 is the block diagram for the functional module that data processing equipment provided by the embodiments of the present application includes.
Fig. 6 is the block diagram for the sub-function module that data level division module includes in Fig. 5.
Fig. 7 is the block diagram that mapping relations adjust the sub-function module that module includes in Fig. 5.
Icon: 10- electronic equipment；12- memory；14- processor；100- data processing equipment；110- data level is drawn
Sub-module；111- attribute information acquisition submodule；113- data level divides submodule；130- data memory module；150- reflects
Penetrate relationship adjustment module；151- level adjustment judging submodule；153- grade classification judging submodule；155- mapping relations generate
Submodule.
Specific embodiment
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment only
It is a part of the embodiment of the application, instead of all the embodiments.The application being usually described and illustrated herein in the accompanying drawings
The component of embodiment can be arranged and be designed with a variety of different configurations.
Therefore, the detailed description of the embodiments herein provided in the accompanying drawings is not intended to limit below claimed
Scope of the present application, but be merely representative of the selected embodiment of the application.Based on the embodiment in the application, this field is common
Technical staff's every other embodiment obtained without creative efforts belongs to the model of the application protection
It encloses.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.In the description of the present application
In, term " first ", " second ", " third ", " the 4th " etc. are only used for distinguishing description, and should not be understood as only or imply opposite
Importance.
In the description of the present application unless specifically defined or limited otherwise, term " setting ", " connected ", " connection " are answered
It is interpreted broadly, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can be mechanical connect
It connects, is also possible to be electrically connected；It can be directly connected, can also can be in two elements indirectly connected through an intermediary
The connection in portion.For the ordinary skill in the art, the tool of above-mentioned term in this application can be understood with concrete condition
Body meaning.
As shown in Figure 1, the embodiment of the present application provides a kind of electronic equipment 10, it may include memory 12, processor 14
With the computer program that is stored in the memory 12 and can be run on the processor 14.
It is directly or indirectly electrically connected between the memory 12 and processor 14, to realize the transmission or friendship of data
Mutually.It is electrically connected for example, can be realized between each other by one or more communication bus or signal wire.The processor 14 is used for
The executable computer program stored in the memory 12 is executed, to realize corresponding data processing method.
Wherein, the memory 12 may be, but not limited to, random access memory (Random Access Memory,
RAM), read-only memory (Read Only Memory, ROM), programmable read only memory (Programmable Read-Only
Memory, PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, EPROM),
Electricallyerasable ROM (EEROM) (Electric Erasable Programmable Read-Only Memory, EEPROM) etc..
Wherein, memory 12 is for storing program, and the processor 14 executes described program after receiving and executing instruction.
The processor 14 may be a kind of IC chip, the processing capacity with signal.Above-mentioned processor 14
It can be general processor, including central processing unit (Central Processing Unit, CPU), network processing unit
(Network Processor, NP), system on chip (System on Chip, SoC) etc.；It can also be digital signal processor
(DSP), specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate
Or transistor logic, discrete hardware components.May be implemented or execute disclosed each method in the embodiment of the present application,
Step and logic diagram.General processor can be microprocessor or the processor is also possible to any conventional processor
Deng.
It is appreciated that structure shown in FIG. 1 is only to illustrate, the electronic equipment 10 may also include more than shown in Fig. 1
Perhaps less component or with the configuration different from shown in Fig. 1.Each component shown in Fig. 1 can use hardware, software
Or combinations thereof realize.
Optionally, the type of the electronic equipment 10 is unrestricted, for example, may include, but is not limited to web (website) clothes
Servers and other equipment for having data-handling capacity such as business device, data server.
In conjunction with Fig. 2, the embodiment of the present application also provides a kind of data processing method that can be applied to above-mentioned electronic equipment 10.Its
In, method and step defined in the related process of the data processing method can be realized by the electronic equipment 10.Below will
Detailed process shown in Fig. 2 is described in detail.
Step S110 carries out grade classification according to the attribute information of isomerization data to be processed and handles to obtain multiple data
Grade.
In the present embodiment, the isomerization data are made of multi-group data, wherein each group of data are in practical applications
Refer to the multiple parameters of the corresponding equipment of a user.Also, each group of data may include multiple subdatas (parameter).
For example, the isomerization data can be made of 6 groups of data, respectively the equipment of user 1 in the example that can be substituted at one
The parameter (parameter a1, parameter a2 and parameter a3) that a has, parameter (the parameter b1, parameter b2, parameter that the equipment b of user 1 has
B3 and parameter b4), the parameter (parameter c1, parameter c2, parameter c3, parameter c4 and parameter c5) that the equipment c of user 2 has, user 2
The parameter (parameter d1, parameter d2, parameter d3, parameter d4 and parameter d5) that has of equipment d, the parameter that the equipment e of user 3 has
(parameter e1, parameter e2, parameter e3 and parameter e4), the parameter (parameter f1, parameter f2 and parameter f3) that the equipment f of user 3 has.
Wherein, the attribute information of the isomerization data refers to the attribute information of each group of data, for identifying this group of number
According to.Also, the particular content of the attribute information is unrestricted, can be selected according to practical application request, for example, upper
It states in example, the quantity for the parameter that each group of data include can be used as corresponding attribute information.
It should be noted that the attribute information for carrying out data level division can be having for the isomerization data
Whole attribute informations, be also possible to the attribute information of partial data.For example, when isomerization data include 10000 groups of data,
1000 groups of data can be selected as sample data wherein.Then, the attribute information based on each group of data in the sample data
Grade classification is carried out, to improve the efficiency of data processing.
After the attribute information for getting the isomerization data, grade classification can be carried out according to the attribute information,
Obtain multiple data levels.Wherein, the corresponding data volume of each data level is of different sizes.For example, in the examples described above, user 1
The parameter that has of the equipment f of parameter and user 3 that has of equipment a may belong to a data level, all include 3 parameters.
It should be noted that the identical two groups of data of the quantity for the parameter for including might not belong to a data level, it is also necessary to root
It is distinguished according to the type of each parameter.
For example, step S110 may comprise steps of: for isomery to be processed in the example that one kind can substitute
Change each group of data in data, it is corresponding with each data type to obtain the data type that each subdata has in this group of data
Subdata quantity；Grade classification is carried out according to the quantity of data type subdata corresponding with each data type
Processing obtains multiple data levels.
Wherein, data type can be field type, for example, may include, but be not limited to character string type, floating type, length
Integer, date type and Boolean type etc..
Also, the particular number for dividing obtained data level is also unrestricted, can be carried out according to practical application request
Selection.For example, can be according to the factors such as the processing capacity of processor 14 and the storage capacity of memory 12, predefining can be with
The quantity of the data level of division, for example, 3,5,7,8 etc..
That is, the step of carrying out grade classification, is specifically as follows: according to the data type, each data class
The quantity and a preset quantity of the corresponding subdata of type carry out division and handle to obtain preset quantity data level.
Step S130 generates a corresponding mapping relations each described data level, and will belong to identical data
The each group of data of grade is based on the corresponding mapping relations of the data level and is stored.
It in the present embodiment, can also be respectively to each data for convenient for searching after being stored data
Grade generates corresponding mapping relations, obtains multiple mapping relations.Then, each group of data is deposited based on each mapping relations
Storage.It should be noted that the specific manifestation form of above-mentioned mapping relations is unrestricted, can be selected according to practical application request
It selects.For example, generating different relation tables according to different data in the example that one kind can substitute.
In detail, it for different data grade, can generate respectively with different relation tables.For example, for larger data
Corresponding data level is measured, can be generated with the relation table compared with multiple row, it, can be with for data level corresponding compared with small data quantity
Generate the relation table with less column.Specifically, in the example that one kind can substitute, data level may include three, point
It Wei not the first estate, the second grade and the tertiary gradient.Wherein, in the corresponding relation table of the first estate character string type can for 1,
Floating type can for 1, long can be 1, it is as shown in the table:
| User | Equipment | String | Double | long |
In the corresponding relation table of second grade character string type can for 2, floating type can be 2, long can be 2
It is a, as shown in the table:
| User | Equipment | String1 | String2 | Double1 | Double2 | long1 | long2 |
In the corresponding relation table of the tertiary gradient character string type can for 3, floating type can be 3, long can be 3
It is a, as shown in the table:
Wherein, in above-mentioned 3 tables, String indicates that character string type, Double indicate that floating type, long indicate that length is whole
Type.
It, can be according to the data type of each parameter in each group of data according to above-mentioned after establishing above-mentioned mapping relations
Relation table is stored, for example, the parameter that the equipment f of the equipment a of user 1 has in above-mentioned example parameter and user 3 has
It can be stored based on the corresponding relation table of the first estate, as shown in the table:
| User | Equipment | String | Double | long |
| 1 | a | Parameter a1 | Parameter a2 | Parameter a3 |
| 3 | f | Parameter f1 | Parameter f2 | Parameter f3 |
The parameter that the equipment e of the equipment b of user 1 has in above-mentioned example parameter and user 3 has can be based on second
The corresponding relation table of grade is stored, as shown in the table:
| User | Equipment | String1 | String2 | Double1 | Double2 | long1 | long2 |
| 1 | b | Parameter b1 | Parameter b2 | Parameter b3 | Parameter b4 | ||
| 3 | e | Parameter e1 | Parameter e2 | Parameter 53 | Parameter e4 |
The parameter that the equipment d of the equipment c of user 2 has in above-mentioned example parameter and user 2 has can be based on third
The corresponding relation table of grade is stored, as shown in the table:
By the above method, the storage respectively between each group of data with different data amount may be implemented, guaranteeing to deposit
On the basis of capable of effectively being searched after storage, moreover it is possible to avoid the problem that storage resource wastes, so as to improve right in the prior art
Isomerization data are unfavorable for searching after there is storage resource waste or storage when being stored, for isomerization data
Storage has high practical value.
Step S150 judges whether there is the data level for needing to be adjusted, and to the data etc. that needs are adjusted
The corresponding mapping relations of grade are adjusted.
In the present embodiment, after by step S130 by the way that each group of data is stored according to generation mapping relations,
It can also determine whether the adjustment for needing to carry out data level.It is corresponding also, when judging to need to carry out data level adjustment
Ground is adjusted mapping relations so as to can further avoid when based on mapping relations adjusted storage each group of data
The problem of storage resource wastes.
For example, in the example of the corresponding relation table of the above-mentioned tertiary gradient, due to the corresponding position String3 and Double3
No storing data is set, thus, the problem of waste there is storage resource.It therefore, can be by the corresponding relation table of the tertiary gradient
Adjustment are as follows:
Optionally, it is unrestricted with the concrete mode for carrying out mapping relations adjustment to execute step S150, it can be according to reality
Application demand is selected, for example, in the example that can substitute of one kind, can repartitioning without data level, only
One or more mapping relations need to be adjusted processing (as shown above).In another example the example that can be substituted in another kind
In, in conjunction with Fig. 3, step S150 may include step S151, step S153 and step S155.
Step S151 judges whether there is the data level for needing to be adjusted.
In the present embodiment, after executing step S130 to complete the storage to isomerization data to be processed, consider
It can change to the isomerization data with the variation of time.It, can be with base to carry out storage effectively to the data after variation
The attribute information of data after variation judges whether the adjustment for needing to carry out data level.Also, judging to need to carry out
When the adjustment of data level, step S153 can be executed.
Step S153 judges whether to need to re-start grade classification processing.
In the present embodiment, can judge whether to need to re-start according to the specific situation of change of isomerization data
Grade division processing.Also, when not needing to re-start grade classification processing, directly corresponding mapping relations can be adjusted
Whole processing, for example, after parameter d5 is abandoned in the equipment d of user 2 in the examples described above, due to the corresponding relation table of the tertiary gradient
Corresponding data are not present in middle Long3, which can be deleted, obtain relation table as follows:
Wherein, when judging to need to re-start grade classification, step S155 can be executed.
Step S155 generates corresponding mapping relations according to the data level repartitioned.
In the present embodiment, the mode for repartitioning data level is unrestricted, can be carried out according to practical application request
Selection.For example, in a kind of example the attribute information of current isomerization data can be reacquired, then, based on current
Attribute information carries out corresponding data level division.In another example in another example, if the corresponding data of some grade have
Biggish increase can split the data level (as shown in Figure 4).For example, it is above-mentioned include 3 character string types, 3
Floating type, 3 longs relation table in, can obtain including 2 character string types, 2 floating-points on the basis of the relation table
The relation table of type, 3 longs, as shown in the table:
That is, before adjustment, one group of data and packet including 2 character string types, 2 floating types, 3 longs
Include 3 character string types, 3 floating types, 3 longs one group of data pass through a relation table storage；After the adjustment, including
2 character string types, 2 floating types, 3 longs one group of data and including 3 character string types, 3 floating types, 3 it is long whole
One group of data of type pass through different relation table storages respectively.
Wherein, to generate corresponding mapping relations based on the data level repartitioned unrestricted, can be according to actually answering
It is selected with demand, for example, in the example that one kind can substitute, to avoid stored data that biggish change occurs
It is dynamic, it can be handled based on the principle for entering bucket nearby in consistency hash algorithm.That is, can be according to repartitioning
Obtained data level and consistency hash algorithm generates corresponding mapping relations.
In conjunction with Fig. 5, the embodiment of the present application also provides a kind of data processing equipment that can be applied to above-mentioned electronic equipment 10
100.Wherein, the data processing equipment 100 may include data level division module 110, data memory module 130 and mapping
Relationship adjusts module 150.
The data level division module 110, for carrying out grade according to the attribute information of isomerization data to be processed
Division handles to obtain multiple data levels, wherein the isomerization data are made of multi-group data, the corresponding number of each data level
It is of different sizes according to measuring.In the present embodiment, the data level division module 110 can be used for executing step S110 shown in Fig. 2,
Related content about the data level division module 110 is referred to the description to step S110 above.
The data memory module 130, for generating a corresponding mapping relations each described data level, and
The each group of data for belonging to identical data grade is based on the corresponding mapping relations of the data level to store.In the present embodiment
In, the data memory module 130 can be used for executing step S130 shown in Fig. 2, the phase about the data memory module 130
Hold inside the Pass and is referred to the description to step S130 above.
The mapping relations adjust module 150, for judging whether there is the data level for needing to be adjusted, and to need
The corresponding mapping relations of the data level to be adjusted are adjusted.In the present embodiment, the mapping relations adjust module
150 can be used for executing step S150 shown in Fig. 2, and the related content about mapping relations adjustment module 150 is referred to
Above to the description of step S150.
In conjunction with Fig. 6, in the present embodiment, the data level division module 110 may include that attribute information obtains submodule
Block 111 and data grade classification submodule 113.
The attribute information acquisition submodule 111, for obtaining for each group of data in isomerization data to be processed
Take the quantity for the data type subdata corresponding with each data type that each subdata has in this group of data, wherein every
One group of data includes multiple subdatas.
The data level divides submodule 113, for corresponding with each data type according to the data type
The quantity of subdata carries out grade classification and handles to obtain multiple data levels.
In conjunction with Fig. 7, in the present embodiment, the mapping relations adjustment module 150 may include that level adjustment judges submodule
Block 151, grade classification judging submodule 153 and mapping relations generate submodule 155.
The level adjustment judging submodule 151, for judging whether there is the data level for needing to be adjusted.At this
In embodiment, the level adjustment judging submodule 151 can be used for executing step S151 shown in Fig. 2, about the grade tune
The related content of whole judging submodule 151 is referred to the description to step S151 above.
The grade classification judging submodule 153, for judging whether when in the presence of the data level for needing to be adjusted
Need to re-start grade classification processing.In the present embodiment, the grade classification judging submodule 153 can be used for executing Fig. 2
Shown step S153, the related content about the grade classification judging submodule 153 are referred to above to step S153
Description.
The mapping relations generate submodule 155, for when needing to re-start grade classification processing, according to drawing again
The data level got generates corresponding mapping relations.In the present embodiment, the mapping relations generate submodule 155 and can use
In executing step S155 shown in Fig. 2, it is right above that the related content about mapping relations generation submodule 155 is referred to
The description of step S155.
Wherein, the mapping relations generate submodule 155 and are specifically used for: according to the data level repartitioned and one
Cause property hash algorithm generates corresponding mapping relations.
In conclusion data processing method provided by the present application, data processing equipment 100 and electronic equipment 10, pass through base
Obtain corresponding data amount multiple data levels of different sizes in the attribute information of isomerization data, and identical data will be belonged to etc.
The each group of data of grade is based on the corresponding mapping relations of the data level and is stored, so that realizing has each group of different data amount
Storage respectively between data, on the basis of can effectively be searched after guaranteeing storage, moreover it is possible to storage resource be avoided to waste
The problem of, so as to improve when storing in the prior art to isomerization data, there are be unfavorable for after storage resource waste or storage
The problem of lookup, has high practical value for the storage of isomerization data.Also, mapping can also be closed according to demand
System is adjusted, to avoid leading to the problem of storage resource waste occur after isomerization data movement to be processed, further
Ground improves the practical value of data processing method.
In several embodiments provided by the embodiment of the present application, it should be understood that disclosed device and method, it can also
To realize by another way.Device and method embodiment described above is only schematical, for example, in attached drawing
Flow chart and block diagram show that the devices of multiple embodiments according to the application, method and computer program product are able to achieve
Architecture, function and operation.In this regard, each box in flowchart or block diagram can represent module, a program
A part of section or code, a part of the module, section or code include that one or more is patrolled for realizing defined
Collect the executable instruction of function.It should also be noted that in some implementations as replacement, function marked in the box
It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel
Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or
The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic
The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.
In addition, each functional module in each embodiment of the application can integrate one independent portion of formation together
Point, it is also possible to modules individualism, an independent part can also be integrated to form with two or more modules.
It, can be with if the function is realized and when sold or used as an independent product in the form of software function module
It is stored in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, electronic equipment or network equipment etc.) execute all or part of step of each embodiment the method for the application
Suddenly.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), deposits at random
The various media that can store program code such as access to memory (RAM, Random Access Memory), magnetic or disk.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to the packet of nonexcludability
Contain, so that the process, method, article or equipment for including a series of elements not only includes those elements, but also including
Other elements that are not explicitly listed, or further include for elements inherent to such a process, method, article, or device.
In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including the element
Process, method, article or equipment in there is also other identical elements.
The foregoing is merely preferred embodiment of the present application, are not intended to limit this application, for the skill of this field
For art personnel, various changes and changes are possible in this application.Within the spirit and principles of this application, made any to repair
Change, equivalent replacement, improvement etc., should be included within the scope of protection of this application.
Claims (10)
1. a kind of data processing method characterized by comprising
It carries out grade classification according to the attribute information of isomerization data to be processed to handle to obtain multiple data levels, wherein institute
It states isomerization data to be made of multi-group data, the corresponding data volume of each data level is of different sizes；
A corresponding mapping relations, and each group of data that will belong to identical data grade are generated each described data level
It is stored based on the corresponding mapping relations of the data level；
The data level for needing to be adjusted is judged whether there is, and the corresponding mapping of data level being adjusted to needs is closed
System is adjusted.
2. data processing method according to claim 1, which is characterized in that the needs that judge whether there is are adjusted
Data level, and the step of corresponding mapping relations of data level that are adjusted of needs are adjusted, comprising:
Judge whether there is the data level for needing to be adjusted；
The data level for needing to be adjusted if it exists then judges whether to need to re-start grade classification processing；
If desired grade classification processing is re-started, then corresponding mapping is generated according to the data level repartitioned and closed
System.
3. data processing method according to claim 2, which is characterized in that the data etc. that the basis is repartitioned
Grade generates the step of corresponding mapping relations, specifically:
Corresponding mapping relations are generated according to the data level and consistency hash algorithm repartitioned.
4. data processing method according to claim 1 to 3, which is characterized in that described according to be processed different
The attribute information of structure data carries out grade classification and handles the step of obtaining multiple data levels, comprising:
For each group of data in isomerization data to be processed, the data type that each subdata has in this group of data is obtained
The quantity of subdata corresponding with each data type, wherein each group of data include multiple subdatas；
It handles to obtain according to the quantity of data type subdata corresponding with each data type progress grade classification more
A data level.
5. data processing method according to claim 4, which is characterized in that described according to the data type and each
The quantity of the corresponding subdata of data type carries out grade classification and handles the step of obtaining multiple data levels, specifically:
Division processing is carried out according to the data type, the quantity of the corresponding subdata of each data type and a preset quantity
Obtain preset quantity data level.
6. a kind of data processing equipment characterized by comprising
Data level division module handles to obtain for carrying out grade classification according to the attribute information of isomerization data to be processed
Multiple data levels, wherein the isomerization data are made of multi-group data, and the corresponding data volume size of each data level is not
Together；
Data memory module for generating a corresponding mapping relations each described data level, and will belong to identical
The each group of data of data level is based on the corresponding mapping relations of the data level and is stored；
Mapping relations adjust module, are adjusted for judging whether there is the data level for needing to be adjusted, and to needs
The corresponding mapping relations of data level be adjusted.
7. data processing equipment according to claim 6, which is characterized in that the mapping relations adjust module and include:
Level adjustment judging submodule, for judging whether there is the data level for needing to be adjusted；
Grade classification judging submodule, in the presence of need be adjusted data level when, judge whether need again into
The processing of row grade classification；
Mapping relations generate submodule, for need to re-start grade classification processing when, according to the number repartitioned
Corresponding mapping relations are generated according to grade.
8. data processing equipment according to claim 7, which is characterized in that the mapping relations generate submodule and specifically use
In:
Corresponding mapping relations are generated according to the data level and consistency hash algorithm repartitioned.
9. according to data processing equipment described in claim 6-8 any one, which is characterized in that the data level divides mould
Block includes:
Attribute information acquisition submodule, for obtaining this group of data for each group of data in isomerization data to be processed
In the quantity of data type subdata corresponding with each data type that has of each subdata, wherein each group of data packet
Include multiple subdatas；
Data level divides submodule, for the quantity according to data type subdata corresponding with each data type
Grade classification is carried out to handle to obtain multiple data levels.
10. a kind of electronic equipment, which is characterized in that including memory, processor and be stored in the memory and can be at this
The computer program run on processor realizes data described in claim 1-5 any one when the computer program is run
Processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811563206.XA CN109669993A (en) | 2018-12-20 | 2018-12-20 | Data processing method, data processing equipment and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811563206.XA CN109669993A (en) | 2018-12-20 | 2018-12-20 | Data processing method, data processing equipment and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109669993A true CN109669993A (en) | 2019-04-23 |
Family
ID=66144009
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811563206.XA Pending CN109669993A (en) | 2018-12-20 | 2018-12-20 | Data processing method, data processing equipment and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109669993A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117390030B (en) * | 2023-12-12 | 2024-03-08 | 北京仁科互动网络技术有限公司 | Multidimensional parameter mapping configuration method and device and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102054388A (en) * | 2009-11-03 | 2011-05-11 | 厦门雅迅网络股份有限公司 | Method for receiving and displaying map |
| CN105631286A (en) * | 2015-09-29 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Methods and apparatuses for storing fingerprint template information and performing authentication by adopting fingerprint information |
| CN106156283A (en) * | 2016-06-27 | 2016-11-23 | 江苏迪纳数字科技股份有限公司 | Isomery Hadoop based on data temperature and joint behavior stores method |
| CN107463506A (en) * | 2017-07-04 | 2017-12-12 | 积成电子股份有限公司 | A kind of data storage and management method of electric energy meter |
-
2018
- 2018-12-20 CN CN201811563206.XA patent/CN109669993A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102054388A (en) * | 2009-11-03 | 2011-05-11 | 厦门雅迅网络股份有限公司 | Method for receiving and displaying map |
| CN105631286A (en) * | 2015-09-29 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Methods and apparatuses for storing fingerprint template information and performing authentication by adopting fingerprint information |
| CN106156283A (en) * | 2016-06-27 | 2016-11-23 | 江苏迪纳数字科技股份有限公司 | Isomery Hadoop based on data temperature and joint behavior stores method |
| CN107463506A (en) * | 2017-07-04 | 2017-12-12 | 积成电子股份有限公司 | A kind of data storage and management method of electric energy meter |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117390030B (en) * | 2023-12-12 | 2024-03-08 | 北京仁科互动网络技术有限公司 | Multidimensional parameter mapping configuration method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9015214B2 (en) | Process of generating a list of files added, changed, or deleted of a file server | |
| US20160253402A1 (en) | Adaptive data repartitioning and adaptive data replication | |
| CN109359115B (en) | Distributed storage method, device and system based on graph database | |
| CN104765749A (en) | Data storage method and device | |
| CN104281701A (en) | Method and system for querying distributed multi-scale spatial data | |
| CN104850649B (en) | A kind of method and system that point of interest sampling is carried out on map | |
| JP2013167990A (en) | File list generation method, system and program, and file list generation device | |
| CN110019298B (en) | Data processing method and device | |
| WO2015165545A1 (en) | Embedded processing of structured and unstructured data using a single application protocol interface (api) | |
| WO2013110357A1 (en) | Social network analysis | |
| US20160292233A1 (en) | Discarding data points in a time series | |
| US20220197950A1 (en) | Eliminating many-to-many joins between database tables | |
| CN109753504A (en) | Data query method and device | |
| CN110110172B (en) | Information display method and device | |
| CN105005567B (en) | Interest point query method and system | |
| CN109669993A (en) | Data processing method, data processing equipment and electronic equipment | |
| US9760836B2 (en) | Data typing with probabilistic maps having imbalanced error costs | |
| CN109669947A (en) | Data processing method, data processing equipment and electronic equipment | |
| CN110928902A (en) | Query method and system for acquiring cloud platform terminal data aiming at paging | |
| CN113779322B (en) | Method, apparatus, device and computer readable storage medium for graph retrieval | |
| CN110704481B (en) | Method and device for displaying data | |
| CN107070987B (en) | Data acquisition method and system for distributed object storage system | |
| CN109101595A (en) | A kind of information query method, device, equipment and computer readable storage medium | |
| CN104809146B (en) | System and method for determining index of the object in object sequence | |
| CN110019783A (en) | Attribute term clustering method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190423 |