CN105868218B - A kind of data processing method and electronic equipment - Google Patents
A kind of data processing method and electronic equipment Download PDFInfo
- Publication number
- CN105868218B CN105868218B CN201510035579.XA CN201510035579A CN105868218B CN 105868218 B CN105868218 B CN 105868218B CN 201510035579 A CN201510035579 A CN 201510035579A CN 105868218 B CN105868218 B CN 105868218B
- Authority
- CN
- China
- Prior art keywords
- data
- group
- block
- server
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The embodiment of the invention discloses a kind of data processing method and electronic equipments, obtain data to be stored；The data are grouped according to preset time period, obtain several group of data；Described group of data include the Query priority corresponding to the preset time period；Processing is ranked up to data in each group of data according to predetermined order strategy, the group data after being sorted；Group data after the sequence are subjected to block encapsulation process, obtain several data block；The Query priority for corresponding to the preset time period in conjunction with included by each group of data in the current storage status of server and the data block stores data block distribution to the server.
Description
Technical field
The present invention relates to field of cloud calculation more particularly to a kind of data processing methods and electronic equipment based on Hadoop.
Background technique
In the related technology, after the operation of Hadoop cluster, the increase and deletion of data can all cause data payload in system
It is unbalanced, and data provided by Hadoop distributed file system (Hadoop Distributed File System, DFS)
Load balancing program Balancer can carry out the balance of storage load to each node.It is deposited however, Balancer only considered
This factor of space is stored up, i.e., load balance is carried out according to the utilization rate of memory space, does not comprehensively consider number of servers, log
The influences of the factors to load balance such as data bulk, log query habit.Moreover, in HDFS due to the storage of block Block be with
Machine selection, therefore constantly adding or deleting with Hadoop cluster interior joint, Hadoop cluster is easy in load not
Balanced state.
In addition, due to, without Indexing Mechanism, causing Hadoop in the inquiry of processing structure mass data inside Hadoop,
Each Map Task is required to successively scan all data of fragment input, then filters out useless number further according to query context
According to the input fragment is generated initial data fragment by MapReduce frame.Therefore, this first to read the data filtered afterwards
Reading manner greatly wastes the performance of Hadoop, especially when handling the range query of mass data, a large amount of Hadoop
It can all be wasted in the reading of hash.
In conclusion not can avoid and load using traditional massive logs data processing method based on Hadoop
The case where unbalanced and query performance wastes.
Summary of the invention
In view of this, can be realized log an embodiment of the present invention is intended to provide a kind of data processing method and electronic equipment
Time equalization distribution of the data on each server, while the search efficiency of massive logs data being effectively provided.
In order to achieve the above objectives, the technical solution of the embodiment of the present invention is achieved in that
The embodiment of the present invention provides a kind of data processing method, is applied to electronic equipment, which comprises
Obtain data to be stored；
The data are grouped according to preset time period, obtain several group of data；Described group of data include corresponding to
In the Query priority of the preset time period；
Processing is ranked up to data in each group of data according to predetermined order strategy, the group data after being sorted；
Group data after the sequence are subjected to block encapsulation process, obtain several data block；
Correspond in conjunction with included by each group of data in the current storage status of server and the data block described default
The Query priority of period stores data block distribution to the server.
In above scheme, the predetermined order strategy includes default identity ordering rule and chronological order；
Correspondingly, described that processing is ranked up to data in each group of data according to predetermined order strategy, after obtaining sequence
Group data, comprising:
Processing is ranked up to data in each group of data according to default identity ordering rule and chronological order,
Group data after being sorted.
In above scheme, the basis presets identity ordering rule and chronological order to number in each group of data
It is handled according to being ranked up, the group data after being sorted, comprising:
The data for being under the jurisdiction of different terminals identity are carried out at the first sequence according to default identity ordering rule
Reason, the group data after obtaining the first sequence；
The second sequence processing is carried out to the data for being under the jurisdiction of same terminal identity mark according to chronological order, is arranged
Group data after sequence.
In above scheme, in the current storage status of the combination server and the data block included by each group of data
Corresponding to the Query priority of the preset time period, data block distribution is stored to the server, comprising:
The Query priority for corresponding to the preset time period according to included by each group of data in the data block, determines
Total Query priority corresponding to the data block；
Total Query priority in conjunction with corresponding to the current storage status of server and identified each data block, will be described
Data block, which is sequentially distributed, to be stored to the server.
In above scheme, the method also includes:
Identity ordering rule is preset in the basis and chronological order arranges data in each group of data
While sequence processing, the first data directory for corresponding to each group of data is established；
Correspondingly, the group data after the sequence are subjected to block encapsulation process, while obtaining several data block, root
The second data directory for corresponding to each data block is established according to first data directory；
While storing data block distribution to the server, establishes the server and store to the clothes
The corresponding relationship being engaged between the data block in device.
The embodiment of the present invention also provides a kind of electronic equipment, and the electronic equipment includes obtaining module, grouping module, sequence
Module, encapsulation process module and assignment storage module；
The acquisition module, for obtaining data to be stored；
The grouping module obtains several group of data for being grouped according to preset time period to the data；Institute
Stating group data includes the Query priority corresponding to the preset time period；
The sorting module is obtained for being ranked up processing to data in each group of data according to predetermined order strategy
Group data after sequence；
The encapsulation process module obtains several for the group data after the sequence to be carried out block encapsulation process
Data block；
The assignment storage module, in conjunction with server current storage status and the data block in each group of data institute
Including the Query priority corresponding to the preset time period, by the data block distribution store to the server.
In above scheme, the predetermined order strategy includes default identity ordering rule and chronological order；
Correspondingly, the sorting module presets identity ordering rule and chronological order to each for basis
Data are ranked up processing in group data, the group data after being sorted.
In above scheme, the sorting module includes the first sequencing unit and the second sequencing unit；Wherein,
First sequencing unit, for according to default identity ordering rule to being under the jurisdiction of different terminals identity
Data carry out the first sequence processing, the group data after obtaining the first sequence；
Second sequencing unit, for according to chronological order to be under the jurisdiction of the data of same terminal identity mark into
The second sequence of row processing, the group data after being sorted.
In above scheme, the assignment storage module includes determination unit and allocation unit；Wherein,
The determination unit, for corresponding to the preset time period according to included by each group of data in the data block
Query priority, determine total Query priority corresponding to the data block；
The allocation unit, for total corresponding to the current storage status of combination server and identified each data block
The data block is sequentially distributed and is stored to the server by Query priority.
In above scheme, the electronic equipment further includes establishing module；Wherein,
It is described to establish module, it is used in the sorting module according to default identity ordering rule and chronological order
While being ranked up processing to data in each group of data, the first data directory for corresponding to each group of data is established；Also use
In the group data after the sequence are carried out block encapsulation process in the encapsulation process module, the same of several data block is obtained
When, the second data directory for corresponding to each data block is established according to first data directory；It is also used to deposit in the distribution
While storage module stores data block distribution to the server, establishes the server and store to the server
In data block between corresponding relationship.
Data processing method and electronic equipment provided by the embodiment of the present invention obtain data to be stored；According to it is default when
Between section the data are grouped, obtain several group of data；Described group of data include corresponding to the preset time period
Query priority；Processing is ranked up to data in each group of data according to predetermined order strategy, the group data after being sorted；
Group data after the sequence are subjected to block encapsulation process, obtain several data block；In conjunction with the currently stored shape of server
The Query priority for corresponding to the preset time period in state and the data block included by each group of data, by the data block
Distribution is stored to the server.It so, it is possible to realize time equalization distribution of the daily record data on each server, have simultaneously
Effect provides the search efficiency of massive logs data.
Detailed description of the invention
Fig. 1 is the implementation process schematic diagram one of data processing method of the embodiment of the present invention；
Fig. 2 is the implementation process schematic diagram that described in the embodiment of the present invention data in each group of data are ranked up with processing；
Fig. 3 is to store data block distribution to the implementation process schematic diagram of the server described in the embodiment of the present invention；
Fig. 4 is the implementation process schematic diagram two of data processing method of the embodiment of the present invention；
Fig. 5 is the composed structure schematic diagram one of electronic equipment of the embodiment of the present invention；
Fig. 6 is the composed structure schematic diagram of sorting module of the embodiment of the present invention；
Fig. 7 is the composed structure schematic diagram of assignment storage module of the embodiment of the present invention；
Fig. 8 is the composed structure schematic diagram two of electronic equipment of the embodiment of the present invention.
Specific embodiment
In embodiments of the present invention, data to be stored is obtained；The data are grouped according to preset time period, are obtained
Several group of data；Described group of data include the Query priority corresponding to the preset time period；According to predetermined order strategy
Processing is ranked up to data in each group of data, the group data after being sorted；Group data after the sequence are subjected to block
Encapsulation process obtains several data block；In conjunction with each group of data institute in the current storage status of server and the data block
Including the Query priority corresponding to the preset time period, by the data block distribution store to the server.
With reference to the accompanying drawing and specific embodiment the present invention is further described in more detail.
Embodiment one
Fig. 1 is the implementation process schematic diagram one of data processing method of the embodiment of the present invention, is applied to electronic equipment, such as Fig. 1
Shown, data processing method of the embodiment of the present invention includes:
Step S101: data to be stored is obtained；
Here, the data to be stored can magnanimity history log data between client and company.Specifically, it supports
The data to be stored of the electronic equipment record such as magnanimity history log data of cloud computing platform, and it is flat to be temporarily stored in cloud computing
On platform.Wherein, the electronic equipment can be the mobile phone for being equipped with HDFS, and laptop, desktop PC, tablet computer are a
Personal digital assistant etc..
Step S102: the data are grouped according to preset time period, obtain several group of data；
Wherein, described group of data include the Query priority corresponding to the preset time period；
Here, the preset time period can be set according to the size and default packet count of acquired data to be stored
It sets.In practical applications, usually set one hour for the preset time period, i.e., according to hour be unit to the data into
Row grouping.
In addition, being accustomed to according to user the inquiry of the history log data, preset corresponding to the preset time
The Query priority TIMEARRY of section.It should be added that preferential when different inquiries is arranged to each preset time period
When grade, being grouped according to preset time period includes different Query priorities in rear obtained each group of data.Actually answering
In, the Query priority corresponding to preset time period is accustomed to configuring generally according to user query, generally away from current time
The frequency that remoter daily record data is queried is lower, thus by away from current time as closely to inquiry corresponding to remote preset time period
Priority is successively arranged from high to low.Specifically, the preset time period is set as one hour, then to go through up to one day
For history daily record data, maximum value 24 will be set as away from one hour nearest Query priority of current time, and so on, it will
Minimum value 1 is set as away from one hour farthest Query priority of current time.
Step S103: processing is ranked up to data in each group of data according to predetermined order strategy, after being sorted
Group data；
Wherein, the predetermined order strategy includes default identity ordering rule and chronological order.
Here, the electronic equipment is according to default identity ordering rule and chronological order in each group of data
Data are ranked up processing, the group data after being sorted.
Specifically, as shown in Fig. 2, the electronic equipment is according to default identity ordering rule and chronological order pair
Data are ranked up processing in each group of data, the group data after being sorted, comprising:
Step S1031: the data for being under the jurisdiction of different terminals identity are carried out according to default identity ordering rule
First sequence processing, the group data after obtaining the first sequence；
Here, the terminal identity mark is used for mark data source.The terminal identity mark can be for for identifying
The phone number of data source.The default identity ordering rule includes being ranked up from big to small according to phone number,
Or it is ranked up from small to large.Correspondingly, by taking the terminal identity is identified as phone number as an example, electronic equipment is according to cell-phone number
The rule that code is ranked up from big to small carries out the first sequence processing to the data for being under the jurisdiction of different phone numbers.
Step S1032: the second sequence is carried out to the data for being under the jurisdiction of same terminal identity mark according to chronological order
Processing, the group data after being sorted.
Here, the chronological order includes the sequencing of the record time of the data.Generally, morning time is handled
Data corresponding to the record time it is also corresponding relatively early.Correspondingly, by taking the terminal identity is identified as phone number as an example, electronics
Equipment carries out the data for being under the jurisdiction of same phone number at the second sequence according to the sequencing of the record time of the data
Reason, the group data after being sorted.
Step S104: the group data after the sequence are subjected to block encapsulation process, obtain several data block；
Specifically, the group data after the sequence are packaged processing according to Hadoop conventional method.Usual Hadoop
The size of a block Block be defaulted as 64M, therefore by the group data after the sequence by size for the block of 64M as unit of carry out
It is packaged encapsulation, obtains several data block.
Step S105: correspond in conjunction with included by each group of data in the current storage status of server and the data block
The Query priority of the preset time period stores data block distribution to the server.
Specifically, as shown in figure 3, each group number in the current storage status of electronic equipment combination server and the data block
According to the included Query priority corresponding to the preset time period, data block distribution is stored to the server,
Include:
Step S1051: correspond to the inquiry of the preset time period according to included by each group of data in the data block
Priority determines total Query priority corresponding to the data block；
Specifically, electronic equipment will correspond to looking into for the preset time period included by each group of data in the data block
It askes priority to sum up, or the mode of arithmetic mean of instantaneous value is taken to obtain total Query priority corresponding to the data block.
Step S1052: the always inquiry in conjunction with corresponding to the current storage status of server and identified each data block is excellent
The data block is sequentially distributed and is stored to the server by first grade.
Wherein, the current storage status of the server includes currently stored load value.The currently stored load can be with
According to currently available memory space L, cpu load ability C and memory M are determined.
Specifically, electronic equipment will combine corresponding to currently stored load value and the identified each data block of server
Total Query priority, will be described according to the small sequence for preferentially storing the big data block of total Query priority of currently stored load value
Data block, which is sequentially distributed, to be stored to the server.
In this way, data processing method through the embodiment of the present invention, can be realized daily record data on each server when
Between equiblibrium mass distribution.
Embodiment two
Fig. 4 is the implementation process schematic diagram two of data processing method of the embodiment of the present invention, is applied to electronic equipment, such as Fig. 4
Shown, data processing method of the embodiment of the present invention includes:
Step S401: data to be stored is obtained；
Here, the data to be stored can magnanimity history log data between client and company.Specifically, it supports
The data to be stored of the electronic equipment record such as magnanimity history log data of cloud computing platform, and it is flat to be temporarily stored in cloud computing
On platform.Wherein, the electronic equipment can be the mobile phone for being equipped with HDFS, and laptop, desktop PC, tablet computer are a
Personal digital assistant etc..
Step S402: the data are grouped according to preset time period, obtain several group of data；
Wherein, described group of data include the Query priority corresponding to the preset time period；
Here, the preset time period can be set according to the size and default packet count of acquired data to be stored
It sets.In practical applications, usually set one hour for the preset time period, i.e., according to hour be unit to the data into
Row grouping.
In addition, being accustomed to according to user the inquiry of the history log data, preset corresponding to the preset time
The Query priority TIMEARRY of section.It should be added that preferential when different inquiries is arranged to each preset time period
When grade, being grouped according to preset time period includes different Query priorities in rear obtained each group of data.Actually answering
In, the Query priority corresponding to preset time period is accustomed to configuring generally according to user query, generally away from current time
The frequency that remoter daily record data is queried is lower, thus by away from current time as closely to inquiry corresponding to remote preset time period
Priority is successively arranged from high to low.Specifically, the preset time period is set as one hour, then to go through up to one day
For history daily record data, maximum value 24 will be set as away from one hour nearest Query priority of current time, and so on, it will
Minimum value 1 is set as away from one hour farthest Query priority of current time.
Step S403: processing is ranked up to data in each group of data according to predetermined order strategy, after being sorted
Group data, and establish the first data directory for corresponding to each group of data；
Wherein, the predetermined order strategy includes default identity ordering rule and chronological order；Described first
Data directory includes each group of data and the mark of terminal identity corresponding to data in described group of data and corresponding between the time
Relationship.
Specifically, the electronic equipment is ranked up processing to data in each group of data according to predetermined order strategy, obtains
Group data after to sequence；Meanwhile establish include each group of data and corresponding terminal identity mark (such as phone number) and
First data directory of the corresponding relationship between the time.
Step S404: carrying out block encapsulation process for the group data after the sequence, obtain several data block, and according to
First data directory establishes the second data directory for corresponding to each data block；
Wherein, second data directory includes terminal identity corresponding to data in the data block and the data block
Corresponding relationship between mark and time.
Specifically, the electronic equipment is ranked up processing to data in each group of data according to predetermined order strategy, obtains
Group data after to sequence；Meanwhile being established according to established first data directory includes the data block Block and the number
According to terminal identity corresponding to data in block mark the time between corresponding relationship the second data directory, and with data block
It is stored on the corresponding NAME NODE node of Block.
Step S405: correspond in conjunction with included by each group of data in the current storage status of server and the data block
Data block distribution is stored to the server, and establishes the server by the Query priority of the preset time period
With the corresponding relationship between the data block that stores into the server.
Specifically, the electronic equipment in conjunction with server current storage status and the data block in each group of data institute
Including the Query priority corresponding to the preset time period, data block distribution is stored same to the server
When, the corresponding relationship for establishing the server between the data block Block that stores into the server, and with data block
It is stored on the corresponding NAME NODE node of Block.
It, can be according to inquiry when user is when executing data query in this way, data processing method through the embodiment of the present invention
Condition setting search index, the search index include terminal identity mark (such as phone number) and time；Further according to inquiry rope
Draw by NAME NODE node stored include data block Block with terminal identity mark and between the time corresponding close
Second data directory of system obtains the data block Block for corresponding to search index；Finally, being stored according in NAMENODE node
Server and data block Block between corresponding relationship, the position of server where obtaining acquired data block Block,
And the Block data are read, obtain query result.
It so, it is possible to realize time equalization distribution of the daily record data on each server, while magnanimity day being effectively provided
The search efficiency of will data.
Embodiment three
Fig. 5 is the composed structure schematic diagram one of electronic equipment of the embodiment of the present invention, as shown in figure 5, the electronic equipment packet
It includes and obtains module 501, grouping module 502, sorting module 503, encapsulation process module 504 and assignment storage module 505；
The acquisition module 501, for obtaining data to be stored；
The grouping module 502 obtains several group of number for being grouped according to preset time period to the data
According to；Described group of data include the Query priority corresponding to the preset time period；
The sorting module 503 is obtained for being ranked up processing to data in each group of data according to predetermined order strategy
Group data after to sequence；
Wherein, the predetermined order strategy includes default identity ordering rule and chronological order.
Specifically, the sorting module 503 is according to default identity ordering rule and chronological order to each group
Data are ranked up processing in data, the group data after being sorted.
The encapsulation process module 504 obtains several for the group data after the sequence to be carried out block encapsulation process
Data block；
The assignment storage module 505, in conjunction with server current storage status and the data block in each group number
According to the included Query priority corresponding to the preset time period, data block distribution is stored to the server.
In one embodiment, as shown in fig. 6, the sorting module 503 includes the first sequencing unit 5031 and the second sequence
Unit 5032；Wherein,
First sequencing unit 5031, for according to default identity ordering rule to being under the jurisdiction of different terminals identity
The data of mark carry out the first sequence processing, the group data after obtaining the first sequence；
Second sequencing unit 5032, for according to chronological order to the number for being under the jurisdiction of same terminal identity mark
Group data according to carry out the second sequence processing, after being sorted.
In one embodiment, as shown in fig. 7, the assignment storage module 505 includes determination unit 5051 and allocation unit
5052；Wherein,
The determination unit 5051, when for corresponding to described default according to included by each group of data in the data block
Between section Query priority, determine total Query priority corresponding to the data block；
The allocation unit 5052, for combining corresponding to current storage status and the identified each data block of server
Total Query priority, the data block is sequentially distributed and is stored to the server.
In one embodiment, as shown in figure 8, the electronic equipment further includes establishing module 506；Wherein,
It is described to establish module 506, it is used in the sorting module according to default identity ordering rule and time order and function
While sequence is ranked up processing to data in each group of data, the first data directory for corresponding to each group of data is established；
Wherein, first data directory includes each group of data and terminal identity corresponding to data in described group of data
Corresponding relationship between mark and time.
It is described to establish module 506, it is also used to that the group data after the sequence are carried out block envelope in the encapsulation process module
Dress processing while obtaining several data block, establishes according to first data directory and corresponds to the of each data block
Two data directories；
Wherein, second data directory includes terminal identity corresponding to data in the data block and the data block
Corresponding relationship between mark and time.
It is described to establish module 506, it is also used to store data block distribution to the clothes in the assignment storage module
While business device, the corresponding relationship between the server and the data block stored into the server is established.
In practical applications, included by each module and each module in the electronic equipment provided in the embodiment of the present invention
Each unit can be realized by the processor in the electronic equipment, can also be realized by specific logic circuit；Than
It such as, in practical applications, can be by central processing unit (CPU), the microprocessor (MPU), digital signal positioned at the electronic equipment
Processor (DSP) or field programmable gate array (FPGA) are realized.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (10)
1. a kind of data processing method is applied to electronic equipment, which is characterized in that the described method includes:
Obtain data to be stored；
The data are grouped according to preset time period, obtain several group of data；Described group of data include corresponding to institute
State the Query priority of preset time period；
Processing is ranked up to data in each group of data according to predetermined order strategy, the group data after being sorted；
Group data after the sequence are subjected to block encapsulation process, obtain several data block；
Correspond to the preset time in conjunction with included by each group of data in the current storage status of server and the data block
The Query priority of section stores data block distribution to the server.
2. the method according to claim 1, wherein the predetermined order strategy includes default identity sequence
Rule and chronological order；
Correspondingly, described that processing is ranked up to data in each group of data according to predetermined order strategy, the group after being sorted
Data, comprising:
Processing is ranked up to data in each group of data according to default identity ordering rule and chronological order, is obtained
Group data after sequence.
3. according to the method described in claim 2, it is characterized in that, the basis presets identity ordering rule and the time is first
Sequence is ranked up processing to data in each group of data afterwards, the group data after being sorted, comprising:
The first sequence processing is carried out to the data for being under the jurisdiction of different terminals identity according to default identity ordering rule, is obtained
Group data to after the first sequence；
The second sequence processing is carried out to the data for being under the jurisdiction of same terminal identity mark according to chronological order, after obtaining sequence
Group data.
4. the method according to claim 1, wherein the current storage status and the number of the combination server
According to included by each group of data in block correspond to the preset time period Query priority, by the data block distribution store to
The server, comprising:
According to included by each group of data in the data block correspond to the preset time period Query priority, determine described in
Total Query priority corresponding to data block；
Total Query priority in conjunction with corresponding to the current storage status of server and identified each data block, by the data
Block, which sequentially distributes, to be stored to the server.
5. according to the method in claim 2 or 3, which is characterized in that the method also includes:
Identity ordering rule is preset in the basis and chronological order is ranked up place to data in each group of data
While reason, the first data directory for corresponding to each group of data is established；
Correspondingly, the group data after the sequence are subjected to block encapsulation process, while obtaining several data block, according to institute
It states the first data directory and establishes the second data directory for corresponding to each data block；
While storing data block distribution to the server, establishes the server and store to the server
In data block between corresponding relationship.
6. a kind of electronic equipment, which is characterized in that the electronic equipment includes obtaining module, grouping module, sorting module, encapsulation
Processing module and assignment storage module；
The acquisition module, for obtaining data to be stored；
The grouping module obtains several group of data for being grouped according to preset time period to the data；Described group
Data include the Query priority corresponding to the preset time period；
The sorting module is sorted for being ranked up processing to data in each group of data according to predetermined order strategy
Group data afterwards；
The encapsulation process module obtains several data for the group data after the sequence to be carried out block encapsulation process
Block；
The assignment storage module, in conjunction with server current storage status and the data block in included by each group of data
The Query priority corresponding to the preset time period, by the data block distribution store to the server.
7. electronic equipment according to claim 6, which is characterized in that the predetermined order strategy includes default identity
Ordering rule and chronological order；
Correspondingly, the sorting module presets identity ordering rule and chronological order to each group of number for basis
It is ranked up processing according to middle data, the group data after being sorted.
8. electronic equipment according to claim 7, which is characterized in that the sorting module includes the first sequencing unit and the
Two sequencing units；Wherein,
First sequencing unit, for according to default identity ordering rule to the number for being under the jurisdiction of different terminals identity
Group data according to carry out the first sequence processing, after obtaining the first sequence；
Second sequencing unit, for carrying out the to the data for being under the jurisdiction of same terminal identity mark according to chronological order
Two sequences processing, the group data after being sorted.
9. electronic equipment according to claim 6, which is characterized in that the assignment storage module includes determination unit and divides
With unit；Wherein,
The determination unit, for corresponding to looking into for the preset time period according to included by each group of data in the data block
Priority is ask, determines total Query priority corresponding to the data block；
The allocation unit, for total inquiry corresponding to the current storage status of combination server and identified each data block
The data block is sequentially distributed and is stored to the server by priority.
10. electronic equipment according to claim 7 or 8, which is characterized in that the electronic equipment further includes establishing module；
Wherein,
It is described to establish module, it is used in the sorting module according to default identity ordering rule and chronological order to every
While data are ranked up processing in a group of data, the first data directory for corresponding to each group of data is established；It is also used to
Group data after the sequence are carried out block encapsulation process by the encapsulation process module, while obtaining several data block,
The second data directory for corresponding to each data block is established according to first data directory；It is also used to store mould in the distribution
While block stores data block distribution to the server, establishes the server and store into the server
Corresponding relationship between data block.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510035579.XA CN105868218B (en) | 2015-01-23 | 2015-01-23 | A kind of data processing method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510035579.XA CN105868218B (en) | 2015-01-23 | 2015-01-23 | A kind of data processing method and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105868218A CN105868218A (en) | 2016-08-17 |
| CN105868218B true CN105868218B (en) | 2019-08-30 |
Family
ID=56624081
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510035579.XA Active CN105868218B (en) | 2015-01-23 | 2015-01-23 | A kind of data processing method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105868218B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107766380B (en) * | 2016-08-22 | 2020-06-30 | 中国移动通信集团河北有限公司 | Method, device and system for balanced distribution and search of service data |
| CN108073356B (en) * | 2016-11-10 | 2021-07-20 | 杭州海康威视系统技术有限公司 | Data storage and search method, device and data processing system |
| CN107357659B (en) * | 2017-07-04 | 2020-09-29 | 东北大学 | Grouping method and query method for query of global load balance in Storm continuous range |
| CN108398641B (en) * | 2017-11-30 | 2021-03-09 | 深圳市科列技术股份有限公司 | Battery data processing method and battery data server |
| CN108111607A (en) * | 2017-12-25 | 2018-06-01 | 广东星鼎通信科技有限公司 | A kind of aggregation of data processing system based on internet |
| TWI821105B (en) * | 2022-12-21 | 2023-11-01 | 艾普勒科技股份有限公司 | Sorting method of communication data and communication apparatus for sorting communication data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1972462A (en) * | 2005-11-23 | 2007-05-30 | 大唐移动通信设备有限公司 | Packet service scheduling method in mobile communication system |
| CN101310483A (en) * | 2005-11-18 | 2008-11-19 | 艾利森电话股份有限公司 | Method and base station for scheduling HSDPA |
| CN102841931A (en) * | 2012-08-03 | 2012-12-26 | 中兴通讯股份有限公司 | Storage method and storage device of distributive-type file system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9235636B2 (en) * | 2012-12-20 | 2016-01-12 | Dropbox, Inc. | Presenting data in response to an incomplete query |
-
2015
- 2015-01-23 CN CN201510035579.XA patent/CN105868218B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101310483A (en) * | 2005-11-18 | 2008-11-19 | 艾利森电话股份有限公司 | Method and base station for scheduling HSDPA |
| CN1972462A (en) * | 2005-11-23 | 2007-05-30 | 大唐移动通信设备有限公司 | Packet service scheduling method in mobile communication system |
| CN102841931A (en) * | 2012-08-03 | 2012-12-26 | 中兴通讯股份有限公司 | Storage method and storage device of distributive-type file system |
Non-Patent Citations (1)
| Title |
|---|
| 面向移动环境的数据存储管理方法关键技术研究;曾文英;《中国博士学位论文全文数据库》;20120615;第1-208页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105868218A (en) | 2016-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105868218B (en) | A kind of data processing method and electronic equipment | |
| TWI743458B (en) | Method, device and system for parallel execution of blockchain transactions | |
| CN102968498B (en) | Data processing method and device | |
| US10467245B2 (en) | System and methods for mapping and searching objects in multidimensional space | |
| US10810168B2 (en) | Allocating file system metadata to storage nodes of distributed file system | |
| CN106407207B (en) | Real-time newly-added data updating method and device | |
| CN107169083A (en) | Public security bayonet socket magnanimity vehicle data storage and retrieval method and device, electronic equipment | |
| CN106161633B (en) | Transmission method and system for packed files based on cloud computing environment | |
| CN108683720A (en) | A kind of container cluster service configuration method and device | |
| CN111913955A (en) | Data sorting processing device, method and storage medium | |
| CN104111936B (en) | Data query method and system | |
| US10417064B2 (en) | Method of randomly distributing data in distributed multi-core processor systems | |
| US9210219B2 (en) | Systems and methods for consistent hashing using multiple hash rings | |
| TWI694700B (en) | Data processing method and device, user terminal | |
| CN104407879A (en) | A power grid timing sequence large data parallel loading method | |
| CN111723073B (en) | Data storage processing method, device, processing system and storage medium | |
| WO2016169237A1 (en) | Data processing method and device | |
| CN111447265A (en) | File storage method, file downloading method, file processing method and related components | |
| US10049122B1 (en) | Cluster file system comprising multiple burst buffers each including virtual file system and metadata server components | |
| CN107391303A (en) | Data processing method, device, system, server and computer-readable storage medium | |
| CN110471947A (en) | Querying method, server and storage medium based on distributed search engine | |
| CN107408239A (en) | Carry out the framework of Mass Data Management in communication applications by multiple mailboxes | |
| CN109101531A (en) | Document handling method, apparatus and system | |
| CN110765082A (en) | Hadoop file processing method and device, storage medium and server | |
| US20190303021A1 (en) | Rebalancing of user accounts among partitions of a storage service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |