CN108804711A - A kind of method, apparatus and computer readable storage medium of data processing - Google Patents
A kind of method, apparatus and computer readable storage medium of data processing Download PDFInfo
- Publication number
- CN108804711A CN108804711A CN201810678182.6A CN201810678182A CN108804711A CN 108804711 A CN108804711 A CN 108804711A CN 201810678182 A CN201810678182 A CN 201810678182A CN 108804711 A CN108804711 A CN 108804711A
- Authority
- CN
- China
- Prior art keywords
- data
- instruction
- client
- server
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Abstract
The embodiment of the invention discloses a kind of method, apparatus of data processing and computer readable storage mediums, corresponding client-side program and serve end program are provided with for each node, client-side program can provide a user service interface, in order to which user inputs corresponding instruction according to demand.The inquiry and reading of data may be implemented in serve end program, and client-side program can be interacted with the serve end program of different nodes.Client-side program can send order corresponding with the instruction to corresponding server-side, to obtain the response results of server-side feedback, and according to the data carried in the response results, corresponding shell script be called to handle data according to the instruction received.It is dependent in compared with the prior art and data is handled completely manually, in this process, user, which need to only input instruction, can realize the automatic business processing of data, improve the efficiency of data processing in kvstore, and be effectively guaranteed the accuracy of data processing.
Description
Technical field
The present invention relates to distributed file system technology field, more particularly to a kind of method, apparatus of data processing and
Computer readable storage medium.
Background technology
The role of management module, that is, Mon module performer managers in distributed file system, maintains entire cluster
State, ensure that the associated component of cluster can reach an agreement in synchronization, the leadership being equivalent in cluster.Mon modules
When managing each node, the data information of each node can be stored in the form of chart, that is, map tables, the wherein type of map can be with
Including osdmap, monmap, mdsmap, authmap, logmap etc., these data informations are all stored in key value database (Key-
Value Store, kvstore) in.
In distributed file system, each node has its corresponding kvstore, when administrative staff need to obtain
When data in kvstore, need first to take out data by digital independent tool (kvstore-tool).Due in kvstore
Data be by serializing treated data, therefore administrative staff need by another decoder work after taking out data
Have and unserializing processing is carried out to the data of serializing, can just obtain the data convenient for administrative staff's identification in this way.
Data in distributed file system in each kvstore should be consistent.In order to verify in each kvstore
Whether data are consistent, need the mode according to above-mentioned inquiry data, obtain the data of respective nodes, and the side by manually comparing
Formula determines whether the data on each node are consistent.In the prior art, either data are inquired, or to different nodes
On data carry out consistency inspection, be required for manually being operated, treatment effeciency is relatively low.
It is those skilled in the art's urgent problem to be solved as it can be seen that how to promote the efficiency of data processing in kvstore.
Invention content
The purpose of the embodiment of the present invention is to provide a kind of method, apparatus and computer readable storage medium of data processing,
The efficiency of data processing in kvstore can be promoted.
In order to solve the above technical problems, the embodiment of the present invention provides a kind of method of data processing, to distributed field system
Corresponding client-side program and serve end program is arranged in each node in system, the method includes：
Client sends order corresponding with described instruction according to the instruction received, to corresponding server-side；
The client receives the response results of the server-side feedback；
The client according to the data carried in the response results, call corresponding shell script to the data into
Row processing, to realize the automatic business processing of data.
Optionally, the client is sent corresponding with described instruction according to the instruction received to corresponding server-side
Order includes：
Client receives described instruction；
The client inquires the correspondence of host name and IP address according to the host name carried in described instruction, with
Obtain the corresponding IP address of host；
The client is established with corresponding server-side and is connected according to the IP address, and to the server-side send with
The corresponding order of described instruction.
Optionally, described instruction includes information comparison instruction and data inquiry instruction；Wherein, in described information comparison instruction
Carry comparison mark and the host name of multiple nodes；The host of at least one node is carried in the data query instruction
Name.
Optionally, when described instruction is information comparison instruction, order corresponding with described instruction is to obtain the clothes
The order of all data informations in business end；
Correspondingly, the client calls corresponding shell script to institute according to the data carried in the response results
It states data to be handled, to realize that the automatic business processing of data includes：
The client receives the mapping table of each server-side feedback；Wherein, it is stored with data class in the mapping table
The correspondence of type and versions of data；
The client compares all mapping tables, to obtain each mapping table respectively exclusive data class
Type and versions of data；
The client receives the key assignments table of each server-side feedback, and calls unserializing script to the key assignments table
In target data carry out unserializing processing, and preserve unserializing treated key assignments table；Wherein, it is deposited in the key assignments table
Contain the correspondence of each versions of data and its corresponding target data.
The embodiment of the present invention additionally provides a kind of device of data processing, is set to each node in distributed file system
Corresponding client-side program and serve end program are set, described device includes transmission unit, receiving unit and processing unit；
The transmission unit, for according to the instruction received, being sent to corresponding server-side corresponding with described instruction
Order；
The receiving unit, the response results for receiving the server-side feedback；
The processing unit, for according to the data carried in the response results, calling corresponding shell script to institute
It states data to be handled, to realize the automatic business processing of data.
Optionally, the transmission unit includes receiving subelement, inquires subelement and establish subelement；
The receiving subelement, for receiving described instruction；
The inquiry subelement, for according to the host name carried in described instruction, inquiring pair of host name and IP address
It should be related to, to obtain the corresponding IP address of host；
It is described to establish subelement, for according to the IP address, establishing and connecting with corresponding server-side, and to the service
End sends order corresponding with described instruction.
Optionally, described instruction includes information comparison instruction and data inquiry instruction；Wherein, in described information comparison instruction
Carry comparison mark and the host name of multiple nodes；The host of at least one node is carried in the data query instruction
Name.
Optionally, when described instruction is information comparison instruction,
Correspondingly, the processing unit includes receiving subelement, contrast subunit and saving subunit；
The receiving subelement, the mapping table for receiving each server-side feedback；Wherein, it is stored in the mapping table
There is the correspondence of data type and versions of data；
The contrast subunit, it is respectively only to obtain each mapping table for comparing all mapping tables
Some data types and versions of data；
The saving subunit, the key assignments table for receiving each server-side feedback, and call unserializing script pair
Data in the key assignments table carry out unserializing processing, and preserve unserializing treated key assignments table.
The embodiment of the present invention additionally provides a kind of device of data processing, including：
Memory, for storing computer program；
Processor, for the step of executing the computer program to realize the method such as above-mentioned data processing.
The embodiment of the present invention additionally provides a kind of computer readable storage medium, is deposited on the computer readable storage medium
The step of containing computer program, the method such as above-mentioned data processing realized when the computer program is executed by processor.
Corresponding client-side program and server-side journey are provided with for each node it can be seen from above-mentioned technical proposal
Sequence, client-side program can provide a user service interface, in order to which user inputs corresponding instruction according to demand.Server-side journey
The inquiry and reading of data may be implemented in sequence, and client-side program can be interacted with the serve end program of different nodes.Client
It holds program according to the instruction received, order corresponding with the instruction can be sent to corresponding server-side, to obtain service
Hold the response results of feedback, and according to the data carried in the response results, call corresponding shell script to data at
Reason, to realize the automatic business processing of data.It is dependent in compared with the prior art and data is handled completely manually, in the mistake
Cheng Zhong, user need to only input instruction, can realize the automatic business processing of data, improve the effect of data processing in kvstore
Rate, and it is effectively guaranteed the accuracy of data processing.
Description of the drawings
In order to illustrate the embodiments of the present invention more clearly, attached drawing needed in the embodiment will be done simply below
It introduces, it should be apparent that, drawings in the following description are only some embodiments of the invention, for ordinary skill people
For member, without creative efforts, other drawings may also be obtained based on these drawings.
Fig. 1 is a kind of flow chart of the method for data processing provided in an embodiment of the present invention；
Fig. 2 is a kind of structural schematic diagram of the device of data processing provided in an embodiment of the present invention；
Fig. 3 is a kind of hardware architecture diagram of the device of data processing provided in an embodiment of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, rather than whole embodiments.Based on this
Embodiment in invention, without making creative work, what is obtained is every other by those of ordinary skill in the art
Embodiment belongs to the scope of the present invention.
In order to enable those skilled in the art to better understand the solution of the present invention, with reference to the accompanying drawings and detailed description
The present invention is described in further detail.
Next, the method that a kind of data processing that the embodiment of the present invention is provided is discussed in detail.Fig. 1 is that the present invention is implemented
Corresponding client is arranged to each node in distributed file system in a kind of flow chart of the method for data processing that example provides
End program and serve end program, this method include：
S101：Client sends order corresponding with instruction according to the instruction received, to corresponding server-side.
In embodiments of the present invention, in order to realize automation, the facilitation processing of data, being directed to each node can be arranged
Corresponding client-side program and serve end program.Wherein, Python may be used in client-side program and serve end program.
In practical applications, the serve end program of each node is equivalent to background program, needs always in running background.And it is objective
Family end program is just activated when it needs operating instruction namely client-side program only can just be opened upon receipt of the instructions
It is dynamic.For the ease of introducing, client-side program can be referred to as to client, serve end program is referred to as server-side.
Client is used to receive instruction, establishes the connection with server-side, data progress analyzing processing etc. to receiving.Clothes
Business end is used to read the data in the kvstore of own node and by data feedback to client.
Data in each node kvstore can be deposited in the form of prefix (prefix)-key (key)-key assignments (value)
Storage, wherein prefix can specifically include the data types such as osdmap, monmap, mdsmap, authmap, logmap, and key can
To be the versions of data corresponding to each data type, value can be the specifying information corresponding to corresponding key.
For example, osdmap:128, osdmap prefix, 128 be key, i.e., the version number of osdmap is 128.And it corresponds to
Value be exactly 128 versions osdmap specifying information.
In practical applications, user can input accordingly in the related interfaces on any one node such as node 1
Instruction, the client on the node 1 is activated at this time, and client can inquire host name according to the host name carried in instruction
It, thus according to the IP address, is established with corresponding server-side with the correspondence of IP address with obtaining the corresponding IP address of host
Connection, and send order corresponding with instruction to server-side.
Host name is used to indicate the title of node.In embodiments of the present invention, each node has its unique host
Name, can distinguish different nodes according to host name.
Each server-side has its corresponding IP address.Client can be established and the IP address institute according to IP address
The communication connection of the server-side of direction.
S102：Client receives the response results of server-side feedback.
Server-side receive client transmission order after, can read in the kvstore of own node with the order phase
Corresponding data, and result feeds back to client in response by these data.
S103：Client calls corresponding shell script to handle data according to the data carried in response results,
To realize the automatic business processing of data.
Instruction input by user is different, and server-side that client is connected and the order sent to server-side also can be
Difference, correspondingly, the data carried in the response results of server-side feedback would also vary from, for different data, client
The processing mode at end also can difference.
Next will in the illustrated manner the flow chart of data processing of concrete form be unfolded to introduce.
Different operational requirements are directed to, user can input various forms of instructions on node.Implement in the present invention
In example, instruction is segmented into two types, and a type is information comparison instruction, and another type instructs for data query.
Wherein, consistency of the information comparison instruction for the data in the kvstore to different nodes is verified.Information
Comparison mark and the host name of multiple nodes can be carried in comparison instruction.
Data query is instructed for inquiring the data stored in the kvstore of some or certain nodes, to obtain
Take required data.The host name of at least one node can be carried in data query instruction.
In order to ensure the logic of inquiry, Query Information be avoided to cause confusion, generally using a node as the place of inquiry
Unit is managed, namely carries the host name of a node in data query instruction.
It is corresponding with instruction to order to obtain all data informations in server-side when instruction is information comparison instruction
Order.Specifically, the data carried in the response results of server-side feedback may include mapping table and key assignments table.Wherein, it maps
The correspondence of data type and versions of data is stored in table；Each versions of data and its corresponding target are stored in key assignments table
The correspondence of data.Target data is the specific data information corresponding to versions of data.
Correspondingly, client can receive the mapping table of each server-side feedback, and all mapping tables are compared, with
To the respective exclusive data type and versions of data of each mapping table.Client receives the key assignments table of each server-side feedback, and calls
Unserializing script carries out unserializing processing to the target data in key assignments table, and preserves unserializing treated key assignments
Table.
It illustrates, it is assumed that user needs the data in the kvstore of node 1 and node 2 being compared, specific to locate
It is as follows to manage flow,
(1) user can on any one node such as node 1 operating instruction " store-tool-client
AutoContrast host1 host2 " carry out the client on starter node 1.
Wherein, " store-tool-client " indicates client-side program, and autoContrast is command parameter, and host1 is
The host name of node 1, host2 are the host name of node 2.
Since it is desired that the data in obtaining the kvstore of two nodes, in the concrete realization, client can first and wherein
A node server-side establish connection, obtain the data in node kvstore.Then again with the service of another node
Connection is established at end, obtains the data in node kvstore.By taking the server-side of elder generation and node 1 establishes connection as an example, client can
To enter (2).
(2) correspondence of the client according to host name and IP address, gets the IP address of node 1.According to the IP
Location can be established with the server-side on node 1 and connect, at this point, client can be sent to the server-side inquires all prefix
Order.
(3) after the server-side of node 1 receives the order, first enumerate the prefix in local kvstore, then by
A key inquired corresponding to different prefix.
May include in the kvstore of one node have multiple prefix, each prefix have its corresponding key with
And value corresponding with the key.
Server-side can first send the mapping table of prefix-key to client, then by the query result of different prefix
It is packaged into internet message, is sent to client one by one.
Wherein, query result can be the key assignments table i.e. key-value tables that server-side obtains.Each prefix has its right
The key-value tables answered.
(4) after client receives the mapping table of the prefix-key that the server-side of node 1 is beamed back, first the mapping table is preserved
Get off.Then all key-value tables of the different prefix after the packing that processing server-side is beamed back.
Since the key-value tables of server-side feedback are the data after serializing.Client can call unserializing foot
This program carries out unserializing processing to data, and then by unserializing, treated that data store.Specifically, data are deposited
The format of hostname+prefix may be used in filename when storage.
(5) client can execute corresponding processing according to the process flow of above-mentioned steps (2) (3) (4) to node 2.
(6) mapping table of client comparison node 1 and the prefix-key to node 2, it is exclusive to export the two nodes
Prefix and key.
In the concrete realization, the output result after being compared is preserved or is shown that user is defeated according to this
Go out as a result, can directly know whether node 1 and the data in the kvstore of node 2 are consistent.
When node 1 and prefix and key not exclusive to node 2, then in the kvstore for illustrating the two nodes
Data it is consistent, realize the synchronization of data.
When there are when exclusive prefix and key, then illustrating the kvstore of the two nodes in node 1 or to node 2
In data it is inconsistent, be not implemented data synchronization.
In the prior art user's manual queries, artificial contrast and etc. compare, in embodiments of the present invention, user only needs
One instruction of input can get corresponding output result.The operating process for simplifying user improves the processing of data
Efficiency.
When it is data query instruction to instruct, according in practical operation, the difference of the data information obtained needed for user counts
May include the instruction for having 4 types according to inquiry instruction：List, dump, dumpAll and get.
Wherein, all prefix and key in the kvstore of list expressions query node.
By taking the client on starter node 1 as an example, the embodiment of list instructions is as follows,
(1) user operating instruction " tore-tool-client list host1 " can start client on host1
End.
(2) correspondence of the client according to host name and IP address, gets the IP address of node 1.According to the IP
Location can be established with the server-side on node 1 and connect, at this point, client can send list orders to the server-side.
(3) after the server-side of host1 receives list orders, all prefix of local kvstore are inquired, it will
Prefix-key mapping tables are sent to client.
(4) the prefix-key mapping tables received can be output on screen by client, in order to which user checks.
All key-value of some prefix in the kvstore of dump expression query nodes.
By taking the client on starter node 1 as an example, the embodiment of dump instructions is as follows,
(1) operating instruction " store-tool-client dump host1 prefix " starts client on host1
End.
Wherein, prefix can be a specific data type, for example, the specific instruction of the osdmap of inquiry host1
For：store-tool-client dump host1 osdmap.
(2) correspondence of the client according to host name and IP address, gets the IP address of node 1.According to the IP
Location can be established with the server-side on node 1 and connect, at this point, client can send dump orders to the server-side.
(3) after server-side receives dump orders, inquire local kvstore specified prefix all key and
Query result is packaged into internet message and is sent to client by value.
(4) after client receives the message that server-side is fed back, unserializing shell script can be called to the number in message
According to unserializing processing is carried out, then by unserializing, treated that data store.
All prefix and its corresponding all key- in the kvstore of dumpALL expression query nodes
value。
By taking the client on starter node 1 as an example, the embodiment of dumpALL instructions is as follows,
(1) operating instruction " store-tool-client dumpALL host1 " starts client on host1.
(2) correspondence of the client according to host name and IP address, gets the IP address of node 1.According to the IP
Location can be established with the server-side on node 1 and connect, at this point, client can send dumpALL orders to the server-side.
(3) after server-side receives dumpALL orders, inquire local kvstore in all prefix and each
The key-value of different prefix is packaged into internet message, is sent to client one by one by the corresponding key-value of prefix.
(4) after client receives the message that server-side is fed back, unserializing shell script can be called to the number in message
According to unserializing processing is carried out, then by unserializing, treated that data store.
The specified key in prefix is specified in the kvstore of get expression query nodes.
By taking the client on starter node 1 as an example, the embodiment of get instructions is as follows,
(1) operating instruction " the store-tool-client get host1 prefix key out " on host1 " is opened
Dynamic client.
(2) correspondence of the client according to host name and IP address, gets the IP address of node 1.According to the IP
Location can be established with the server-side on node 1 and connect, at this point, client can send get orders to the server-side.
(3) after server-side receives get orders, the correspondence key of the correspondence prefix in the kvstore of local is inquired, by the key
Value be sent to client.
(4) client call unserializing shell script carries out unserializing processing to the value received, then will be anti-
Treated that value is output on screen for serializing, in order to which user checks.
It should be noted that interaction of the client on a node in addition to realizing information with the server-side of own node
Outside, the interaction that information can also be realized with the server-side on other nodes, so as to realize the function of cross-node inquiry.
For example, after user inputs corresponding instruction in the related interfaces of node 1, the client on node 1 is activated, false
If what is carried in instruction is the host name of node 2, client can be obtained according to the correspondence of host name and IP address at this time
To the IP address corresponding to the host name, pointed by the IP address is the server-side on node 2, at this time the client on node 1
End can be established with the server-side on node 2 and communicate to connect.Since the server-side on node 2 can read the kvstore of node 2
In data, and by the data feedback of reading to client.Therefore, user inputs instruction on node 1, can get section
Data in point 2 realize cross-node inquiry.
Corresponding client-side program and server-side journey are provided with for each node it can be seen from above-mentioned technical proposal
Sequence, client-side program can provide a user service interface, in order to which user inputs corresponding instruction according to demand.Server-side journey
The inquiry and reading of data may be implemented in sequence, and client-side program can be interacted with the serve end program of different nodes.Client
It holds program according to the instruction received, order corresponding with the instruction can be sent to corresponding server-side, to obtain service
Hold the response results of feedback, and according to the data carried in the response results, call corresponding shell script to data at
Reason, to realize the automatic business processing of data.It is dependent in compared with the prior art and data is handled completely manually, in the mistake
Cheng Zhong, user need to only input instruction, can realize the automatic business processing of data, improve the effect of data processing in kvstore
Rate, and it is effectively guaranteed the accuracy of data processing.
Fig. 2 is a kind of structural schematic diagram of the device of data processing provided in an embodiment of the present invention, to distributed field system
Corresponding client-side program and serve end program is arranged in each node in system, and described device includes transmission unit 21, receives list
Member 22 and processing unit 23；
Transmission unit 21, for according to the instruction received, order corresponding with instruction to be sent to corresponding server-side；
Receiving unit 22, the response results for receiving server-side feedback；
Processing unit 23, for according to the data carried in response results, corresponding shell script being called to carry out data
Processing, to realize the automatic business processing of data.
Optionally, transmission unit includes receiving subelement, inquires subelement and establish subelement；
Receiving subelement, for receiving instruction；
Subelement is inquired, for according to the host name carried in instructing, inquiring the correspondence of host name and IP address, with
Obtain the corresponding IP address of host；
Subelement is established, sends and instructs for according to IP address, establishing and connecting with corresponding server-side, and to server-side
Corresponding order.
Optionally, instruction includes information comparison instruction and data inquiry instruction；Wherein, it is carried pair in information comparison instruction
Than mark and the host name of multiple nodes；The host name of at least one node is carried in data query instruction.
Optionally, when instruction is information comparison instruction,
Correspondingly, processing unit includes receiving subelement, contrast subunit and saving subunit；
Receiving subelement, the mapping table for receiving each server-side feedback；Wherein, be stored in mapping table data type and
The correspondence of versions of data；
Contrast subunit, for comparing all mapping tables, to obtain each mapping table respectively exclusive data type
And versions of data；
Saving subunit, the key assignments table for receiving each server-side feedback, and call unserializing script in key assignments table
Data carry out unserializing processing, and preserve unserializing treated key assignments table.Feature says in embodiment corresponding to Fig. 2
The bright related description that may refer to embodiment corresponding to Fig. 1, no longer repeats one by one here.
Corresponding client-side program and server-side journey are provided with for each node it can be seen from above-mentioned technical proposal
Sequence, client-side program can provide a user service interface, in order to which user inputs corresponding instruction according to demand.Server-side journey
The inquiry and reading of data may be implemented in sequence, and client-side program can be interacted with the serve end program of different nodes.Client
It holds program according to the instruction received, order corresponding with the instruction can be sent to corresponding server-side, to obtain service
Hold the response results of feedback, and according to the data carried in the response results, call corresponding shell script to data at
Reason, to realize the automatic business processing of data.It is dependent in compared with the prior art and data is handled completely manually, in the mistake
Cheng Zhong, user need to only input instruction, can realize the automatic business processing of data, improve the effect of data processing in kvstore
Rate, and it is effectively guaranteed the accuracy of data processing.
Fig. 3 is a kind of hardware architecture diagram of the device 30 of data processing provided in an embodiment of the present invention, and device 30 wraps
It includes：
Memory 31, for storing computer program；
Processor 32, for the step of executing computer program to realize the method such as above-mentioned data processing.
The embodiment of the present invention additionally provides a kind of computer readable storage medium, is stored on computer readable storage medium
Computer program, when computer program is executed by processor the step of the realization such as method of above-mentioned data processing.
It is provided for the embodiments of the invention a kind of method, apparatus and computer readable storage medium of data processing above
It is described in detail.Each embodiment is described by the way of progressive in specification, the highlights of each of the examples are
Difference from other examples, just to refer each other for identical similar portion between each embodiment.Embodiment is disclosed
Device for, since it is corresponded to the methods disclosed in the examples, so description is fairly simple, related place is referring to method
Part illustrates.It should be pointed out that for those skilled in the art, before not departing from the principle of the invention
It puts, can be with several improvements and modifications are made to the present invention, these improvement and modification also fall into the guarantor of the claims in the present invention
It protects in range.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, depends on the specific application and design constraint of technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Claims (10)
1. a kind of method of data processing, which is characterized in that the corresponding visitor of each node setting in distributed file system
Family end program and serve end program, the method includes：
Client sends order corresponding with described instruction according to the instruction received, to corresponding server-side；
The client receives the response results of the server-side feedback；
The client according to the data carried in the response results, call corresponding shell script to the data at
Reason, to realize the automatic business processing of data.
2. according to the method described in claim 1, it is characterized in that, the client takes according to the instruction received to corresponding
Business end sends order corresponding with described instruction：
Client receives described instruction；
The client inquires the correspondence of host name and IP address according to the host name carried in described instruction, to obtain
The corresponding IP address of host；
The client is established with corresponding server-side and is connected according to the IP address, and to the server-side send with it is described
Instruct corresponding order.
3. method according to claim 1 or 2, which is characterized in that described instruction includes that information comparison instruction and data is looked into
Ask instruction；Wherein, comparison mark and the host name of multiple nodes are carried in described information comparison instruction；The data query
The host name of at least one node is carried in instruction.
4. according to the method described in claim 3, it is characterized in that, when described instruction be information comparison instruction when, with the finger
Corresponding order is enabled to obtain the order of all data informations in the server-side；
Correspondingly, the client calls corresponding shell script to the number according to the data carried in the response results
According to being handled, to realize that the automatic business processing of data includes：
The client receives the mapping table of each server-side feedback；Wherein, be stored in the mapping table data type and
The correspondence of versions of data；
The client compares all mapping tables, with obtain each mapping table respectively exclusive data type and
Versions of data；
The client receives the key assignments table of each server-side feedback, and calls unserializing script in the key assignments table
Target data carries out unserializing processing, and preserves unserializing treated key assignments table；Wherein, it is stored in the key assignments table
The correspondence of each versions of data and its corresponding target data.
5. a kind of device of data processing, which is characterized in that the corresponding visitor of each node setting in distributed file system
Family end program and serve end program, described device include transmission unit, receiving unit and processing unit；
The transmission unit, for according to the instruction received, order corresponding with described instruction to be sent to corresponding server-side；
The receiving unit, the response results for receiving the server-side feedback；
The processing unit, for according to the data carried in the response results, calling corresponding shell script to the number
According to being handled, to realize the automatic business processing of data.
6. device according to claim 5, which is characterized in that the transmission unit includes receiving subelement, inquiry son list
Member and establish subelement；
The receiving subelement, for receiving described instruction；
The inquiry subelement, for according to the host name carried in described instruction, inquiring host name and the corresponding of IP address being closed
System, to obtain the corresponding IP address of host；
It is described to establish subelement, for according to the IP address, establishing and connecting with corresponding server-side, and sent out to the server-side
Send order corresponding with described instruction.
7. device according to claim 5 or 6, which is characterized in that described instruction includes that information comparison instruction and data is looked into
Ask instruction；Wherein, comparison mark and the host name of multiple nodes are carried in described information comparison instruction；The data query
The host name of at least one node is carried in instruction.
8. device according to claim 7, which is characterized in that when described instruction is information comparison instruction,
Correspondingly, the processing unit includes receiving subelement, contrast subunit and saving subunit；
The receiving subelement, the mapping table for receiving each server-side feedback；Wherein, it is stored with number in the mapping table
According to the correspondence of type and versions of data；
The contrast subunit, it is respectively exclusive to obtain each mapping table for comparing all mapping tables
Data type and versions of data；
The saving subunit, the key assignments table for receiving each server-side feedback, and call unserializing script to described
Data in key assignments table carry out unserializing processing, and preserve unserializing treated key assignments table.
9. a kind of device of data processing, which is characterized in that including：
Memory, for storing computer program；
Processor, for executing the computer program to realize the side of the data processing as described in Claims 1-4 any one
The step of method.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program, realizing the method for data processing as described in any one of Claims 1-4 when the computer program is executed by processor
Step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810678182.6A CN108804711B (en) | 2018-06-27 | 2018-06-27 | Data processing method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810678182.6A CN108804711B (en) | 2018-06-27 | 2018-06-27 | Data processing method and device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108804711A true CN108804711A (en) | 2018-11-13 |
| CN108804711B CN108804711B (en) | 2022-12-06 |
Family
ID=64071206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810678182.6A Active CN108804711B (en) | 2018-06-27 | 2018-06-27 | Data processing method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108804711B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739828A (en) * | 2018-12-29 | 2019-05-10 | 咪咕文化科技有限公司 | A kind of data processing method, equipment and computer readable storage medium |
| WO2021047193A1 (en) * | 2019-09-12 | 2021-03-18 | 百度在线网络技术（北京）有限公司 | Voice processing method and apparatus, device, and computer storage medium |
| WO2021047197A1 (en) * | 2019-09-12 | 2021-03-18 | 百度在线网络技术（北京）有限公司 | Speech processing method, apparatus and device, and computer storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104333578A (en) * | 2014-10-23 | 2015-02-04 | 秦科 | Distributed data exchange system and method |
| US20150169623A1 (en) * | 2012-07-26 | 2015-06-18 | Tencent Technology (Shenzhen) Company Limited | Distributed File System, File Access Method and Client Device |
| CN106302607A (en) * | 2015-06-05 | 2017-01-04 | 腾讯科技（深圳）有限公司 | It is applied to block storage system and the method for cloud computing |
| CN107203594A (en) * | 2017-04-28 | 2017-09-26 | 努比亚技术有限公司 | A kind of data processing equipment, method and computer-readable recording medium |
| CN107256269A (en) * | 2013-05-03 | 2017-10-17 | 脸谱公司 | Method, computer-readable non-transitory storage medium and system |
| CN108171473A (en) * | 2017-12-26 | 2018-06-15 | 北京九章云极科技有限公司 | A kind of Data Analysis Services system and data analysis processing method |
-
2018
- 2018-06-27 CN CN201810678182.6A patent/CN108804711B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150169623A1 (en) * | 2012-07-26 | 2015-06-18 | Tencent Technology (Shenzhen) Company Limited | Distributed File System, File Access Method and Client Device |
| CN107256269A (en) * | 2013-05-03 | 2017-10-17 | 脸谱公司 | Method, computer-readable non-transitory storage medium and system |
| CN104333578A (en) * | 2014-10-23 | 2015-02-04 | 秦科 | Distributed data exchange system and method |
| CN106302607A (en) * | 2015-06-05 | 2017-01-04 | 腾讯科技（深圳）有限公司 | It is applied to block storage system and the method for cloud computing |
| CN107203594A (en) * | 2017-04-28 | 2017-09-26 | 努比亚技术有限公司 | A kind of data processing equipment, method and computer-readable recording medium |
| CN108171473A (en) * | 2017-12-26 | 2018-06-15 | 北京九章云极科技有限公司 | A kind of Data Analysis Services system and data analysis processing method |
Non-Patent Citations (2)
| Title |
|---|
| 瞧见风: "Ceph网络模块介绍", 《CSDN博客》 * |
| 瞧见风: "Ceph高级工具介绍之ceph-kvstore-tool的使用", 《CSDN博客》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739828A (en) * | 2018-12-29 | 2019-05-10 | 咪咕文化科技有限公司 | A kind of data processing method, equipment and computer readable storage medium |
| WO2021047193A1 (en) * | 2019-09-12 | 2021-03-18 | 百度在线网络技术（北京）有限公司 | Voice processing method and apparatus, device, and computer storage medium |
| WO2021047197A1 (en) * | 2019-09-12 | 2021-03-18 | 百度在线网络技术（北京）有限公司 | Speech processing method, apparatus and device, and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108804711B (en) | 2022-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108510389B (en) | Intelligent contract calling method and device based on block chain and readable storage medium | |
| US6493751B1 (en) | Network configuration method and system for a window-based operating system environment | |
| DE60124440T2 (en) | Naming convention for different types of devices, and apparatus and method for applying this naming convention | |
| US5864669A (en) | Method and system for accessing a particular instantiation of a server process | |
| JP4647096B2 (en) | Method and system for configuring a computer to connect to a network using a network connection object | |
| US20030069948A1 (en) | Automated online subscription | |
| EP0613274A2 (en) | Socket structure for concurrent multiple protocol access | |
| CN107613037B (en) | Domain name redirection method and system | |
| CN108804711A (en) | A kind of method, apparatus and computer readable storage medium of data processing | |
| CN109040207B (en) | Method, device and equipment for accessing user mode network file system | |
| CN112104757A (en) | Method, equipment and system for configuring IP address | |
| CN107707656A (en) | The remote loading method and system of operating system | |
| CN106603593A (en) | HTTP calling method and device based on adaption | |
| EP1246059B1 (en) | Dynamic interface aggregation on demand | |
| CN106411742A (en) | Message transmission method and device | |
| CN110417876A (en) | Node server and main control device in session method, distributed system | |
| US7907619B2 (en) | Method, system and program product for adapting to protocol changes | |
| CN116566830A (en) | Network configuration method, device, system, edge equipment and storage medium | |
| CN115242882B (en) | Method and device for accessing k8s container environment based on transport layer route | |
| CN110022394A (en) | Method, apparatus, equipment and the storage medium of address book contact intelligent updating | |
| CN115225606A (en) | Domain name access method and system of cross-network protocol of container cloud platform | |
| CN108279924A (en) | Program dissemination method and device | |
| Cisco | Configuring IBM Channel Attach | |
| Cisco | Configuring IBM Channel Attach | |
| KR100397468B1 (en) | Network management device and method using mib name server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |