CN108965362A - A kind of data processing method, server and storage medium - Google Patents
A kind of data processing method, server and storage medium Download PDFInfo
- Publication number
- CN108965362A CN108965362A CN201710358969.XA CN201710358969A CN108965362A CN 108965362 A CN108965362 A CN 108965362A CN 201710358969 A CN201710358969 A CN 201710358969A CN 108965362 A CN108965362 A CN 108965362A
- Authority
- CN
- China
- Prior art keywords
- server
- destination server
- information
- task data
- destination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Computer And Data Communications (AREA)
Abstract
The present invention provides a kind of data processing method, which comprises first server receives mission bit stream, parses the mission bit stream, obtains the parsing result including task data information and destination server information；The mission bit stream is used to indicate distributed tasks data to destination server；When the parsing result also shows that the mission bit stream is sent by source server, task data is obtained from the source server based on the task data information；The task data information is sent to the destination server, so that the destination server is based on task data described in the task data acquisition of information based on the destination server information.The present invention also provides a kind of server and storage mediums.
Description
Technical field
The present invention relates to data processing technique more particularly to a kind of data processing methods, server and storage medium.
Background technique
Initial stage is applied for the website of browser/server (Browser/Server, B/S) structure, due to user volume and
Amount of access is few, and therefore, a server is just able to satisfy the demand of user；At this point, need to will only be adjusted when carrying out procedure site deployment
It tries successful program data and is uploaded to server, just complete the distribution of program data；The distribution of configuration data is also used
Process same as the distribution of above procedure data.But with the increase of user volume and amount of access, need to accordingly increase clothes
The quantity of business device, and identical procedure site is disposed on every server, so that every server can independent process user
Service request, the handling capacity of lifting system.
With the increase of number of servers, centralization scheme is generallyd use to distribute data；Made with an electronic equipment
For center source device, by the equipment file or data all destination servers are distributed to by network one by one.As can be seen that
The data dissemination process be it is linear, serial, therefore, data distribution speed slows down also with increasing for number of servers；
When multiple data distribution tasks carry out simultaneously, the load of center source device also can increase at double, further decrease data point
Send out efficiency；Also, since center source device is solely responsible for data distribution task, when center source device breaks down, number
It will be unable to be normally carried out according to distributed tasks.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of data processing side to solve problems of the prior art
Method, server and storage medium.
The embodiment of the present invention provides a kind of data processing method, which comprises
First server receives mission bit stream, parses the mission bit stream, and obtaining includes task data information and mesh
Mark the parsing result of server info；The mission bit stream is used to indicate distributed tasks data to destination server；
When the parsing result also shows that the mission bit stream is sent by source server, it is based on the task data information
Task data is obtained from the source server；
The task data information is sent to the destination server, so that the mesh based on the destination server information
It marks server and is based on task data described in the task data acquisition of information.
It is described to be taken based on the destination server information transmission task data information to the target in above scheme
Business device, comprising:
The destination server is divided at least one destination server group based on the destination server information, often
A destination server group includes at least a destination server
At least one first subtask information is generated based at least one described destination server group, each first son is appointed
Task data information and destination server sub-information associated with destination server group are carried in business information；
Either objective server of the first subtask information into corresponding destination server group is sent respectively, so that institute
It states either objective server and is based on task data described in the acquisition of information of first subtask.
It is described when the parsing result, which also shows the mission bit stream not, to be sent by source server in above scheme
Method further include:
The first server obtains the task data that user equipment is sent.
It is described to be taken based on the destination server information transmission task data information to the target in above scheme
Business device, comprising:
Multiple destination servers are divided at least two destination server groups based on the destination server information, often
A destination server group includes at least one destination server；
Corresponding at least two second subtask information, Mei Ge are generated based at least two destination server group
Task data information and destination server sub-information associated with destination server group are carried in two subtask information；
Either objective server of the second subtask information into corresponding destination server group is sent respectively, so that institute
It states either objective server and is based on task data described in the acquisition of information of second subtask.
It is described to be based on when the task data information is used to characterize the store path of the task data in above scheme
The task data information obtains task data from the source server, comprising:
The first server establishes the communication connection between the source server；
Based on the communication connection, number of tasks is obtained from the source server according to the store path of the task data
According to.
The embodiment of the present invention also provides a kind of server, and the server includes:
Processing unit parses the mission bit stream for receiving mission bit stream, obtain include task data information and
The parsing result of destination server information；The mission bit stream is used to indicate distributed tasks data to destination server；
Judging unit, for judging whether the parsing result shows that the mission bit stream is sent by source server；
First acquisition unit, for judging that the parsing result also shows the mission bit stream by source in the judging unit
When server is sent, task data is obtained from the source server based on the task data information；
Transmission unit, for sending the task data information to the destination service based on the destination server information
Device, so that the destination server is based on task data described in the task data acquisition of information.
In above scheme, the transmission unit is specifically used for being based on the destination server information for the destination service
Device is divided at least one destination server group, and each destination server group includes at least a destination server
At least one first subtask information is generated based at least one described destination server group, each first son is appointed
Task data information and destination server sub-information associated with destination server group are carried in business information；
Either objective server of the first subtask information into corresponding destination server group is sent respectively, so that institute
It states either objective server and is based on task data described in the acquisition of information of first subtask.
In above scheme, the judging unit judges that the parsing result also shows that the mission bit stream is serviced by source
When device is sent, the server further includes second acquisition unit, for obtaining the task data of user equipment transmission.
In above scheme, the transmission unit is specifically used for being based on the destination server information for multiple destination services
Device is divided at least two destination server groups, and each destination server group includes at least one destination server；
Corresponding at least two second subtask information, Mei Ge are generated based at least two destination server group
Task data information and destination server sub-information associated with destination server group are carried in two subtask information；
Either objective server of the second subtask information into corresponding destination server group is sent respectively, so that institute
It states either objective server and is based on task data described in the acquisition of information of second subtask.
In above scheme, when the task data information is used to characterize the store path of the task data, the transmission
Unit, specifically for the communication connection established between the server and the source server；
Based on the communication connection, number of tasks is obtained from the source server according to the store path of the task data
According to.
The embodiment of the present invention also provides a kind of computer readable storage medium, is stored thereon with computer program, the program
The step of above method is realized when being executed by processor.
The embodiment of the present invention also provides a kind of server, including memory, processor and storage are on a memory and can be
The step of computer program run on processor, the processor realizes the above method when executing described program.
Data processing method provided in the embodiment of the present invention, server and storage medium, first server receive
Mission bit stream parses the mission bit stream, obtains the parsing result including task data information and destination server information；Institute
It states mission bit stream and is used to indicate distributed tasks data to destination server；When the parsing result also show the mission bit stream by
When source server is sent, task data is obtained from the source server based on the task data information；Based on the target
Server info sends the task data information to the destination server, so that the destination server is based on the task
Data information obtains the task data.In this way, after first server gets task data based on the mission bit stream, as
Source server executes data distribution task, and the task data information carried in mission bit stream is sent to destination server；Not only
When improving data distribution efficiency, and avoiding an independent source dispensing machines and be responsible for data distribution, due to source dispensing machines
The problem of data distribution task caused by failure can not be normally carried out.
Detailed description of the invention
Fig. 1 is the network topology structure schematic diagram that the present invention distributes data using centralization scheme；
Fig. 2 is the processing flow schematic diagram that the present invention distributes data using centralization scheme；
Fig. 3 is a kind of processing flow schematic diagram one of data processing method of the embodiment of the present invention；
Fig. 4 is a kind of processing flow schematic diagram two of data processing method of the embodiment of the present invention；
Fig. 5 is a kind of processing flow schematic diagram three of data processing method of the embodiment of the present invention；
Fig. 6 is a kind of processing flow schematic diagram four of data processing method of the embodiment of the present invention；
Fig. 7 to realize the present invention embodiment first object server send subtask network topology structure schematic diagram；
Fig. 8 is the network topology that the second destination server of the embodiment of the present invention and the 6th destination server pull task data
Structural schematic diagram；
Fig. 9 is the network topology knot that the second destination server of the embodiment of the present invention and the second destination server distribute subtask
Structure schematic diagram；
The network topology structure schematic diagram of Figure 10 embodiment data processing method to realize the present invention；
Figure 11 is the composed structure schematic diagram of server of the embodiment of the present invention；
Figure 12 is the hardware composed structure schematic diagram of server of the embodiment of the present invention.
Specific embodiment
Invention is further described in detail in the following with reference to the drawings and specific embodiments.
Data processing method described in embodiment in order to better understand the present invention, below in the prior art using center
Change scheme distribution data are briefly introduced.
Using the network topology structure schematic diagram of centralization scheme distribution data, as shown in Figure 1, utilizing an electronic equipment
Centered on source device, file or data be uploaded to center source device by terminal device by user, and center source device again will be literary
Part or data pass through network and are transmitted to all destination servers one by one.Specifically, using the processing of centralization scheme distribution data
Flow diagram, as shown in Fig. 2, file or data are sent to center source device by terminal device by user, and in described
Heart source device sends the control command for carrying destination server list so that the center source device by this document or data one by one
All destination servers being distributed in the destination server list.Center source device is after receiving control command, judgement
When the destination server list carried in the control command is not sky, a destination server sends file or number thereto
According to until the file or data are sent to all destination servers.
The processing flow schematic diagram one of a kind of data processing method of the embodiment of the present invention, as shown in figure 3, including following step
It is rapid:
Step S101, first server receive mission bit stream, parse the mission bit stream；
Specifically, for first server by parsing the mission bit stream, obtaining includes that task data information and target take
The parsing result for device information of being engaged in.
Here, the task data information is the information that first server can get task data according to it；Citing
For, the task data information can be the store path of task data, or store the file of the task data
Press from both sides title.The destination server information is the information for the destination server that characterization receives task data, for example, the mesh
Mark the title that server info can be destination server, or the network protocol (Internet of destination server
Protocol, IP) address.Correspondingly, the mission bit stream is used to indicate first server distributed tasks data to destination service
Device.
Step S102 is appointed when the parsing result also shows that the mission bit stream is sent by source server based on described
Business data information obtains task data from the source server；
Specifically, when it is described also show that the mission bit stream is sent by source server when, the first server establish with
Communication connection between the source server, and the communication connection and the task data information are based on from the source server
Obtain task data.
Here, when the task data information is used to characterize the store path of the task data, the first service
Device pulls task data from the source server according to the store path of the task data.
Wherein, the task data stored in the source server is sent by user by terminal device.
Step S103 sends the task data information to the destination server based on the destination server information；
Specifically, the destination server is divided by least one destination server based on the destination server information
Group, each destination server group include at least a destination server；Based at least one described destination server group
At least one first subtask information is generated, task data information is carried in the information of each first subtask and is taken with target
The associated destination server sub-information of Wu Qi group；The first subtask information is sent respectively to corresponding destination server group
In either objective server so that the either objective server be based on first subtask acquisition of information described in number of tasks
According to.
Here, N number of destination server is divided by M destination server group with first server, and in each target
A destination server is selected in server farm, and the second subtask information is sent to selected M destination server；
After M destination server receives the second subtask information, it is based on the second subtask acquisition of information task data；With M
For a destination server in destination server, the second subtask information is being received, and be based on second subtask
After acquisition of information task data, it is found that in the information of second subtask further include multiple destination servers, therefore will be the multiple
Destination server is divided into X destination server group, in each destination server group in X destination server group
A destination server is selected, and sends the destination server of third subtask information to selection, so that the selected target of institute
Server is based on the third subtask acquisition of information task data；And so on, until the acquisition of target complete server is taken office
Business data.
The processing flow schematic diagram two of a kind of data processing method of the embodiment of the present invention, as shown in figure 4, including following step
It is rapid:
Step S201, first server receive mission bit stream, parse the mission bit stream；
Specifically, for first server by parsing the mission bit stream, obtaining includes that task data information and target take
The parsing result for device information of being engaged in.
Here, the task data information is the information that first server can get task data according to it；Citing
For, the task data information can be the store path of task data, or store the file of the task data
Press from both sides title.The destination server information is the information for the destination server that characterization receives task data, for example, the mesh
Mark the title that server info can be destination server, or the IP address of destination server.Correspondingly, the task
Information is used to indicate first server distributed tasks data to destination server.
Step S202, when the parsing result, which also shows the mission bit stream not, to be sent by source server, described
One server obtains the task data that user equipment is sent；
Specifically, when the parsing result, which also shows the mission bit stream not, to be sent by source server, described first
Server is as source server, the directly task data of reception user equipment transmission.
Step S203 sends the task data information to the destination server based on the destination server information；
Specifically, the destination server information is based on by multiple destination servers as the first server of source server
At least two destination server groups are divided into, each destination server group includes at least one destination server.
Corresponding at least two second subtask information, Mei Ge are generated based at least two destination server group
Task data information and destination server sub-information associated with destination server group are carried in two subtask information.
Either objective server of the second subtask information into corresponding destination server group is sent respectively, so that institute
It states either objective server and is based on task data described in the acquisition of information of second subtask.
The processing flow schematic diagram three of a kind of data processing method of the embodiment of the present invention, as shown in figure 5, including following step
It is rapid:
Step S301, first server receive mission bit stream, parse the mission bit stream, obtain parsing result；
Specifically, for the first server by parsing the mission bit stream, obtaining includes task data information and mesh
Mark the parsing result of server info；The mission bit stream is used to indicate distributed tasks data to destination server.
Whether step S302, first server judge in the parsing result to include source server；It is yes in judging result
When, execute step S303；When the judgment result is No, step S304 is executed；
Here, include source server in the judging result, show that the mission bit stream is sent by source server；Accordingly
, the task data also is stored in the source server；Therefore, the first server can be based on the number of tasks it is believed that
Breath obtains task data from the source server.
When not including source server in the judging result, show the mission bit stream be not sent by source server, and
It is to be sent by user equipment, i.e., user is sent by terminal device；Therefore, first server can directly receive user equipment transmission
Task data.
Step S303, first server obtain task data from source server；
Specifically, the first server actively establishes the communication connection with source server, is based on the communication connection, presses
Task data is pulled from the source server according to the data task information.
Step S304 judges in the parsing result with the presence or absence of destination server list；When the judgment result is yes, it holds
Row step S305；When the judgment result is No, terminate process.
Specifically, the first server shows institute there are when destination server list in judging the parsing result
State the destination server that acquired task data need to be sent in destination server list by first server.
Destination server in the destination server list is divided at least two destination server groups by step S305
Group；
Here, each destination server group includes at least one destination server.
Step S306 generates corresponding at least two first subtasks letter based at least two destination server group
Breath；
Here, task data information and associated with destination server group is carried in the information of each first subtask
Destination server sub-information.
Wherein, destination server sub-information is for characterizing the destination server information for receiving the task data information.
Step S307, first server send the first subtask information appointing into corresponding destination server group respectively
One destination server, so that the either objective server is based on task data described in the acquisition of information of first subtask.
Step S308, the destination server for receiving the first subtask information are based on the first subtask acquisition of information task
Data.
Step S309, the destination server for receiving the first subtask information judge whether take in the information of first subtask
Band destination server list；Judging result is to execute step 310 when being, when judging result is no, terminates process.
Step S310 receives the mesh that the destination server of the first subtask information is carried into first subtask information
It marks server and sends one or more subtasks information, so that the destination server is believed based on one or more of subtasks
Breath obtains task data.
Specifically, receiving the destination server of the first subtask information can will carry in the information of first subtask
Destination server is divided into multiple destination server groups, and a destination service is arbitrarily selected in each destination server group
Device, for receiving the second subtask information, according to the second subtask acquisition of information task data；Receive the second subtask letter
The destination server of breath sends third subtask information to other destination servers where itself in destination server group again,
Until target complete server gets task data.Receive the first subtask information destination server can also directly by
Second subtask information is sent to each destination server in the first subtask information, so that each destination server is based on institute
State the second subtask acquisition of information task data.
Here, first subtask information, the second subtask information and third subtask information carry task data
Information.
Step S311, the destination server carried in the information of first subtask subtask information based on one or more
Obtain task data.
By taking 8 destination servers need to update 2 files as an example, a kind of process of data processing method of the embodiment of the present invention
Schematic diagram four, as shown in Figure 6, comprising the following steps:
Step S401, terminal device send mission bit stream to first object server；
Specifically, the first object server is terminal device optional mesh from 8 destination servers
Mark server；Therefore, in this step, terminal device can also be sent to other destination servers in addition to first object server
Mission bit stream.
First object server can parse the mission bit stream by the shell tool of itself, and obtaining includes text to be updated
The store path of part and the parsing result of destination server list.
Here, the parsing result is as shown in table 1, the store path including destination server list and file to be updated；
Wherein, the store path of file to be updated is respectively/data/example/file1 and/data/example/file1；Target
Server list includes eight destination server IP address corresponding with eight destination servers；First object server pair
The IP address answered is 192.168.1.1, and the corresponding IP address of the second destination server is 192.168.1.2, third destination service
The corresponding IP address of device is 192.168.1.3, and the corresponding IP address of the 4th destination server is 192.168.1.4, the 5th target
The corresponding IP address of server be 192.168.1.5, the corresponding IP address of the 6th destination server be 192.168.1.6, the 7th
The corresponding IP address of destination server is 192.168.1.7, and the corresponding IP address of the 8th destination server is 192.168.1.8.
Table 1
Step S402, first object server are based on destination server list creating subtask information；
Specifically, the subtask information that the first object server that IP address is 192.168.1.1 creates is as shown in table 2,
Including subtask 1 and subtask 2；Wherein, the source server of subtask 1 is first object server, sends server for IP
Location is the second destination server of 192.168.1.2, and destination server list includes the third mesh that IP address is 192.168.1.3
Mark server, the 4th destination server that IP address is 192.168.1.4 and the 5th target that IP address is 192.168.1.5 clothes
Business device, listed files are /data/example/file1 and/data/example/file1；That is, IP address is
192.168.1.1 first object server sends subtask 1 to the second destination server that IP address is 192.168.1.2,
It indicates that the second destination server obtains task data based on the path of file shown in listed files, and indicates the second destination service
Third destination server, fourth destination server and fiveth destination server of the device into destination server list send file column
Table, so that third destination server, the 4th destination server and the 5th destination server are based on the listed files and obtain task
Data.The source server of subtask 2 is first object server, and sending server is the 6th that IP address is 192.168.1.6
Destination server, destination server list is the 7th destination server that IP address is 192.168.1.7 and IP address is
192.168.1.8 the 8th destination server, listed files are /data/example/file1 and/data/example/
file1；That is, the first object server that IP address is 192.168.1.1 is to the 6th that IP address is 192.168.1.6
Destination server sends subtask 2, is used to indicate the 6th destination server and is obtained based on the path of file shown in listed files
Task data, and indicate seventh destination server and eightth destination service of the 6th destination server into destination server list
Device sends listed files, so that the 7th destination server and the 8th destination server, which are based on the listed files, obtains number of tasks
According to.
Table 2
In the embodiment of the present invention, first object server sends the network topology structure schematic diagram of subtask, such as Fig. 7 institute
Show, the subtask 1 and subtask 2 can be sent to by the ssh tool in first object server the second destination server and
6th destination server.
Step S403, the second destination server and the 6th destination server pull task data；
Specifically, after the second destination server receives subtask 1, the subtask 1 is parsed, the first solution is obtained
Analyse result；After 6th destination server receives subtask 2, the subtask 2 is parsed, the second parsing result is obtained；
First parsing result and second parsing result include the data such as destination server list, listed files and source service
Device；After second destination server and the 6th destination server judge the source server for first object server,
It actively establishes a connection with the first object server, and from the first object server pull task data.
In the embodiment of the present invention, the second destination server and the 6th destination server pull the network topology knot of task data
Structure schematic diagram, as shown in figure 8, the second destination server and the 6th destination server pass through itself SFTP tool from first respectively
Destination server pulls task data.
Step S404, the second destination server are based on destination server list creating subtask information；
Specifically, the second destination server determines to include destination server list, and destination server list in subtask 1
Including IP address be 192.168.1.3 third destination server, the 4th destination server that IP address is 192.168.1.4 and
IP address is the 5th destination server of 192.168.1.5；Therefore, the second destination server can create three subtasks, described
Three subtasks respectively correspond third destination server, the 4th destination server and the 5th destination server；Second destination service
Device can also create two subtasks, and described two subtasks respectively correspond third destination server, the 4th destination server and
Any two in five destination servers；Second destination server can also create a subtask, and one subtask is corresponding
Any one in third destination server, the 4th destination server and the 5th destination server.
Here, by taking the second destination server creates two subtasks as an example, the son of the second destination server creation is appointed
Information of being engaged in is as shown in table 3, including subtask 3 and subtask 4；Wherein, the source server of subtask 3 is the second destination server,
The IP address for sending server is the third destination server of 192.168.1.3, and destination server list is 192.168.1.4's
4th destination server；Listed files is /data/example/file1 and/data/example/file1；That is, IP
The second destination server that address is 192.168.1.2 sends son to the third destination server that IP address is 192.168.1.3
Task 3, instruction third destination server obtains task data based on the path of file shown in listed files, and indicates third mesh
It marks fourth destination server of the server into destination server list and sends listed files, so that the 4th destination server is based on
The listed files obtains task data.The source server of subtask 4 is the second destination server, sends the IP address of server
For the 5th destination server of 192.168.1.5, list is empty for destination server；Listed files is /data/example/
File1 and/data/example/file1；That is, IP address be 192.168.1.2 the second destination server to IP
The 5th destination server that location is 192.168.1.5 sends subtask 4, and the 5th destination server of instruction is based on shown in listed files
File path obtain task data.
Table 3
Step S405, the 6th destination server are based on destination server list creating subtask information；
Specifically, the 6th destination server determines to include destination server list, and destination server list in subtask 2
Including the 7th destination server that IP address is 192.168.1.7, the 8th destination server that IP address is 192.168.1.8；
Therefore, the 6th destination server can create two subtasks, and two subtasks respectively correspond the 7th destination server and
Eight destination servers；6th destination server can also create a subtask, corresponding 7th destination service in one subtask
Any one in device and the 8th destination server.
Here, by taking the 6th destination server creates two subtasks as an example, the son of the second destination server creation is appointed
Information of being engaged in is as shown in table 4, including subtask 5 and subtask 6；Wherein, the source server of subtask 5 is the 6th destination server,
The IP address for sending server is the 7th destination server of 192.168.1.7, and list is empty for destination server；Listed files
For/data/example/file1 and/data/example/file1；That is, IP address is the 6 of 192.168.1.6
Destination server sends subtask 5 to the 7th destination server that IP address is 192.168.1.7, indicates the 7th destination server
Path based on file shown in listed files obtains task data.The source server of subtask 6 is the 6th destination server, hair
The 8th destination server that the IP address of server is 192.168.1.8 is sent, list is empty for destination server；Listed files be/
Data/example/file1 and/data/example/file1；That is, IP address is the 6th mesh of 192.168.1.6
It marks server and sends subtask 6 to the 8th destination server that IP address is 192.168.1.8, indicate the 8th destination server base
The path of the file shown in listed files obtains task data.
Table 4
It should be noted that may be performed simultaneously step S404 and step S405 in the embodiment of the present invention；It can also first hold
Row step S404, then execute step S405；Step S405 can also be first carried out, then executes step S404.
The network topology structure schematic diagram of second destination server and the second destination server distribution subtask, such as Fig. 9 institute
Show, the ssh tool that the second destination server and the 6th destination server pass through itself respectively sends subtask.
Step S406, third destination server and the 5th destination server pull task data；
Specifically, third destination server and the 5th destination server pull the treatment process and step S403 of task data
Described in the second destination server and the 6th destination server pull task data treatment process it is identical, which is not described herein again.
Step S407, third destination server are based on destination server list creating subtask information；
Specifically, third destination server determines to include destination server list, and destination server list in subtask 3
The 4th destination server for being 192.168.1.4 including IP address；The third destination server creates subtask information such as table 5
It is shown, including subtask 7；Wherein, the source server of subtask 7 is third destination server, and the IP address for sending server is
192.168.1.4 the 4th destination server, list is empty for destination server；Listed files is /data/example/file1
With/data/example/file1；That is, the third destination server that IP address is 192.168.1.3 is to IP address
192.168.1.4 the 4th destination server sends subtask 7, and the 4th destination server of instruction is based on text shown in listed files
The path of part obtains task data.
Table 5
Step S408, the 4th destination server pull task data；
Specifically, the process of the 4th destination server pulling data and third destination server and the 5th described in step S406
The process of destination server pulling data is identical, and which is not described herein again.
So far, it completes to be respectively/the text of data/example/file1 and/data/example/file1 to store path
The distribution of part.Realize the network topology structure schematic diagram of data processing method of the embodiment of the present invention, as shown in Figure 10；It can see
Out, the 6th destination server that the second destination server and IP address that IP address is 192.168.1.2 are 192.168.1.6 is held
The process of row data distribution can be parallel；Therefore, the distribution of data distribution is carried out using scheme described in the embodiment of the present invention
Time complexity is 0 (logN), wherein N is the quantity of destination server；Centralization scheme in compared with the existing technology carries out
For the distribution time complexity 0 (N) of data distribution, the quantity of destination server N is bigger, the distribution effect of scheme of the present invention
Rate is higher.Meanwhile in the embodiment of the present invention, any one destination server all can serve as the source machine of data distribution for task
Information is forwarded to another destination server, not only assumes responsibility for the pressure of source server data distribution, improves distribution efficiency, and
And caused by can be avoided because of source server failure the problem of data distribution termination.
Above-mentioned data processing method embodiment to realize the present invention, the embodiment of the present invention also provides a kind of server, described
The composed structure schematic diagram of server, as shown in figure 11, comprising: processing unit 11, judging unit 12,13 and of first acquisition unit
Transmission unit 14；Wherein,
The processing unit 11 parses the mission bit stream for receiving mission bit stream, obtain include number of tasks it is believed that
The parsing result of breath and destination server information；The mission bit stream is used to indicate distributed tasks data to destination server；
The judging unit 12, for judging whether the parsing result shows that the mission bit stream is sent out by source server
It send；
The first acquisition unit 13, for judging that the parsing result also shows the task in the judging unit 12
When information is sent by source server, task data is obtained from the source server based on the task data information；
The transmission unit 14, for sending the task data information to the mesh based on the destination server information
Server is marked, so that the destination server is based on task data described in the task data acquisition of information.
In a specific embodiment, the transmission unit 14 is specifically used for being based on the destination server information for institute
It states destination server and is divided at least one destination server group, each destination server group includes at least a target clothes
Business device
At least one first subtask information is generated based at least one described destination server group, each first son is appointed
Task data information and destination server sub-information associated with destination server group are carried in business information；
Either objective server of the first subtask information into corresponding destination server group is sent respectively, so that institute
It states either objective server and is based on task data described in the acquisition of information of first subtask.
In a specific embodiment, when the task data information is used to characterize the store path of the task data,
The transmission unit 14, specifically for the communication connection established between the server and the source server；Based on described logical
Letter connection obtains task data from the source server according to the store path of the task data.
In a specific embodiment, the judging unit 12 judges that the parsing result also shows the mission bit stream not
It is when being sent by source server, the server further includes second acquisition unit 15, for obtaining the task of user equipment transmission
Data.
Correspondingly, the transmission unit 14, is specifically used for being based on the destination server information for multiple destination servers
At least two destination server groups are divided into, each destination server group includes at least one destination server；
Corresponding at least two second subtask information, Mei Ge are generated based at least two destination server group
Task data information and destination server sub-information associated with destination server group are carried in two subtask information；
Either objective server of the second subtask information into corresponding destination server group is sent respectively, so that institute
It states either objective server and is based on task data described in the acquisition of information of second subtask.
It should be noted that in the embodiment of the present invention, when the first acquisition unit 13 executes corresponding operation, the clothes
Business device receives the mission bit stream that source server is sent, and the source server is any one service in multiple destination servers
Device.When the second acquisition unit 15 executes corresponding operation, the server receives the mission bit stream that user equipment is sent.
In the exemplary embodiment, the server can by one or more application specific integrated circuit (ASIC,
Application Specific Integrated Circuit), digital signal processor (DSP, Digital Signal
Processor), programmable logic device (PLD, Programmable Logic Device), Complex Programmable Logic Devices
(CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-
Programmable Gate Array), general processor, controller, microcontroller (MCU, Micro Controller
Unit), microprocessor (Microprocessor) or other electronic components are realized, for executing preceding method.
The embodiment of the present invention also provides a kind of computer readable storage medium, is stored thereon with computer program, the program
The step of above method is realized when being executed by processor.The computer readable storage medium include: movable storage device, only
Read memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk
Or the various media that can store program code such as CD.
If alternatively, the above-mentioned integrated unit of the present invention is realized in the form of software function module and as independent product
When selling or using, it also can store in a computer readable storage medium.Based on this understanding, the present invention is implemented
Substantially the part that contributes to existing technology can be embodied in the form of software products the technical solution of example in other words,
The computer software product is stored in a storage medium, including some instructions are used so that computer equipment (can be with
It is personal computer, server or network equipment etc.) execute all or part of each embodiment the method for the present invention.
And storage medium above-mentioned includes: that movable storage device, ROM, RAM, magnetic or disk etc. are various can store program code
Medium.
The embodiment of the present invention also provides a kind of server, the hardware composed structure schematic diagram of the server, such as Figure 12 institute
Show, including processor 701 and memory 702, is stored with application program 7022 on the memory 702.
The method that the embodiments of the present invention disclose can be applied in processor 701, or be realized by processor 701.
Processor 701 may be a kind of IC chip, the processing capacity with signal.During realization, the above method it is each
Step can be completed by the integrated logic circuit of the hardware in processor 701 or the instruction of software form.Above-mentioned processing
Device 701 can be general processor, digital signal processor (DSP, Digital Signal Processor) or other can
Programmed logic device, discrete gate or transistor logic, discrete hardware components etc..Processor 701 may be implemented or hold
Disclosed each method, step and logic diagram in the row embodiment of the present invention.General processor can be microprocessor or appoint
What conventional processor etc..The step of method in conjunction with disclosed in the embodiment of the present invention, can be embodied directly at hardware decoding
Reason device executes completion, or in decoding processor hardware and software module combine and execute completion.Software module can be located at
In storage medium, which is located at memory 702, and processor 701 reads the information in memory 702, in conjunction with its hardware
The step of completing preceding method.
In several embodiments provided herein, it should be understood that disclosed device and method can pass through it
Its mode is realized.Apparatus embodiments described above are merely indicative, for example, the division of the unit, only
A kind of logical function partition, there may be another division manner in actual implementation, such as: multiple units or components can combine, or
It is desirably integrated into another system, or some features can be ignored or not executed.In addition, shown or discussed each composition portion
Mutual coupling or direct-coupling or communication connection is divided to can be through some interfaces, the INDIRECT COUPLING of equipment or unit
Or communication connection, it can be electrical, mechanical or other forms.
Above-mentioned unit as illustrated by the separation member, which can be or may not be, to be physically separated, aobvious as unit
The component shown can be or may not be physical unit, it can and it is in one place, it may be distributed over multiple network lists
In member；Some or all of units can be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
In addition, each functional unit in various embodiments of the present invention can be fully integrated in one processing unit, it can also
To be each unit individually as a unit, can also be integrated in one unit with two or more units；It is above-mentioned
Integrated unit both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.
Claims (12)
1. a kind of data processing method, which is characterized in that the described method includes:
First server receives mission bit stream, parses the mission bit stream, and obtaining includes that task data information and target take
The parsing result for device information of being engaged in；The mission bit stream is used to indicate distributed tasks data to destination server；
When the parsing result also shows that the mission bit stream is sent by source server, the task data information is based on from institute
It states and obtains task data in source server；
The task data information is sent to the destination server, so that the target takes based on the destination server information
Device be engaged in based on task data described in the task data acquisition of information.
2. the method according to claim 1, wherein described send described appoint based on the destination server information
Data information be engaged in the destination server, comprising:
The destination server is divided at least one destination server group, each mesh based on the destination server information
It marks server farm and includes at least a destination server；
At least one first subtask information, each first subtask letter are generated based at least one described destination server group
Task data information and destination server sub-information associated with destination server group are carried in breath；
Either objective server of the first subtask information into corresponding destination server group is sent respectively, so that described
One destination server is based on task data described in the acquisition of information of first subtask.
3. the method according to claim 1, wherein when the parsing result also shows that the mission bit stream is not
When being sent by source server, the method also includes:
The first server obtains the task data that user equipment is sent.
4. according to the method described in claim 3, it is characterized in that, described send described appoint based on the destination server information
Data information be engaged in the destination server, comprising:
Multiple destination servers are divided at least two destination server groups, each mesh based on the destination server information
Marking server farm includes at least one destination server；
Corresponding at least two second subtask information, each second son are generated based at least two destination server group
Task data information and destination server sub-information associated with destination server group are carried in mission bit stream；
Either objective server of the second subtask information into corresponding destination server group is sent respectively, so that described
One destination server is based on task data described in the acquisition of information of second subtask.
5. method according to claim 1 or 2, which is characterized in that the task data information is for characterizing the task
It is described that task data is obtained from the source server based on the task data information when store path of data, comprising:
The first server establishes the communication connection between the source server；
Based on the communication connection, task data is obtained from the source server according to the store path of the task data.
6. a kind of server, which is characterized in that the server includes:
Processing unit parses the mission bit stream, obtaining includes task data information and target for receiving mission bit stream
The parsing result of server info；The mission bit stream is used to indicate distributed tasks data to destination server；
Judging unit, for judging whether the parsing result shows that the mission bit stream is sent by source server；
First acquisition unit, for judging that the parsing result also shows that the mission bit stream is serviced by source in the judging unit
When device is sent, task data is obtained from the source server based on the task data information；
Transmission unit is used for based on the destination server information transmission task data information to the destination server,
So that the destination server is based on task data described in the task data acquisition of information.
7. server according to claim 6, which is characterized in that the transmission unit is specifically used for being based on the target
The destination server is divided at least one destination server group by server info, and each destination server group is at least
Including a destination server
At least one first subtask information, each first subtask letter are generated based at least one described destination server group
Task data information and destination server sub-information associated with destination server group are carried in breath；
Either objective server of the first subtask information into corresponding destination server group is sent respectively, so that described
One destination server is based on task data described in the acquisition of information of first subtask.
8. server according to claim 6, which is characterized in that the judging unit judges that the parsing result also shows
The mission bit stream is not when being sent by source server, and the server further includes second acquisition unit, is set for obtaining user
The task data that preparation is sent.
9. server according to claim 8, which is characterized in that the transmission unit is specifically used for being based on the target
Multiple destination servers are divided at least two destination server groups by server info, and each destination server group includes
At least one destination server；
Corresponding at least two second subtask information, each second son are generated based at least two destination server group
Task data information and destination server sub-information associated with destination server group are carried in mission bit stream；
Either objective server of the second subtask information into corresponding destination server group is sent respectively, so that described
One destination server is based on task data described in the acquisition of information of second subtask.
10. server according to claim 6 or 7, which is characterized in that the task data information is for characterizing described appoint
When the store path for data of being engaged in, the transmission unit is logical between the server and the source server specifically for establishing
Letter connection；
Based on the communication connection, task data is obtained from the source server according to the store path of the task data.
11. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The step of claim 1 to 5 the method is realized when execution.
12. a kind of server including memory, processor and stores the computer that can be run on a memory and on a processor
Program, which is characterized in that the step of processor realizes claim 1 to 5 the method when executing described program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710358969.XA CN108965362B (en) | 2017-05-19 | 2017-05-19 | Data processing method, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710358969.XA CN108965362B (en) | 2017-05-19 | 2017-05-19 | Data processing method, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108965362A true CN108965362A (en) | 2018-12-07 |
| CN108965362B CN108965362B (en) | 2023-03-24 |
Family
ID=64461740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710358969.XA Active CN108965362B (en) | 2017-05-19 | 2017-05-19 | Data processing method, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108965362B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6748447B1 (en) * | 2000-04-07 | 2004-06-08 | Network Appliance, Inc. | Method and apparatus for scalable distribution of information in a distributed network |
| CN103856460A (en) * | 2012-12-04 | 2014-06-11 | 华为技术有限公司 | Access control method, device and system |
| CN103905526A (en) * | 2014-03-05 | 2014-07-02 | 深圳市同洲电子股份有限公司 | Dispatching method and server |
| US20150254102A1 (en) * | 2014-03-05 | 2015-09-10 | Fujitsu Limited | Computer-readable recording medium, task assignment device, task execution device, and task assignment method |
| CN105162878A (en) * | 2015-09-24 | 2015-12-16 | 网宿科技股份有限公司 | Distributed storage based file distribution system and method |
| CN105516334A (en) * | 2015-12-25 | 2016-04-20 | 上海帝联信息科技股份有限公司 | CDN (content delivery network) file delivery system |
| CN105681370A (en) * | 2014-11-17 | 2016-06-15 | 腾讯科技(成都)有限公司 | File synchronizing method, server and client |
| CN106302780A (en) * | 2016-08-25 | 2017-01-04 | 北京京东尚科信息技术有限公司 | The method of cluster device bulk transfer data, Apparatus and system, server |
| CN106325988A (en) * | 2016-08-03 | 2017-01-11 | 天津西瑞尔信息工程有限公司 | Task scheduling method and device |
| CN106453460A (en) * | 2015-08-12 | 2017-02-22 | 腾讯科技（深圳）有限公司 | File distributing method, apparatus and system |
| CN106557366A (en) * | 2015-09-28 | 2017-04-05 | 阿里巴巴集团控股有限公司 | Task distribution method, apparatus and system |
| CN106571968A (en) * | 2016-11-10 | 2017-04-19 | 华为技术有限公司 | Business switching method and system |
-
2017
- 2017-05-19 CN CN201710358969.XA patent/CN108965362B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6748447B1 (en) * | 2000-04-07 | 2004-06-08 | Network Appliance, Inc. | Method and apparatus for scalable distribution of information in a distributed network |
| CN103856460A (en) * | 2012-12-04 | 2014-06-11 | 华为技术有限公司 | Access control method, device and system |
| CN103905526A (en) * | 2014-03-05 | 2014-07-02 | 深圳市同洲电子股份有限公司 | Dispatching method and server |
| US20150254102A1 (en) * | 2014-03-05 | 2015-09-10 | Fujitsu Limited | Computer-readable recording medium, task assignment device, task execution device, and task assignment method |
| CN105681370A (en) * | 2014-11-17 | 2016-06-15 | 腾讯科技(成都)有限公司 | File synchronizing method, server and client |
| CN106453460A (en) * | 2015-08-12 | 2017-02-22 | 腾讯科技（深圳）有限公司 | File distributing method, apparatus and system |
| CN105162878A (en) * | 2015-09-24 | 2015-12-16 | 网宿科技股份有限公司 | Distributed storage based file distribution system and method |
| CN106557366A (en) * | 2015-09-28 | 2017-04-05 | 阿里巴巴集团控股有限公司 | Task distribution method, apparatus and system |
| CN105516334A (en) * | 2015-12-25 | 2016-04-20 | 上海帝联信息科技股份有限公司 | CDN (content delivery network) file delivery system |
| CN106325988A (en) * | 2016-08-03 | 2017-01-11 | 天津西瑞尔信息工程有限公司 | Task scheduling method and device |
| CN106302780A (en) * | 2016-08-25 | 2017-01-04 | 北京京东尚科信息技术有限公司 | The method of cluster device bulk transfer data, Apparatus and system, server |
| CN106571968A (en) * | 2016-11-10 | 2017-04-19 | 华为技术有限公司 | Business switching method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108965362B (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103946800B (en) | Lossless uninterrupted message processing method during system software upgrading | |
| CN105721535A (en) | Parallel processing of service functions in service function chains | |
| CN104737519A (en) | System and apparatus of generalized network controller for a software defined network (SDN) | |
| CN102577258B (en) | Configuring a three-stage CLOS-network packet switch | |
| CN105847139A (en) | Data transmission method, apparatus and system | |
| CN102273179B (en) | Method and device for terminal batch upgrading | |
| CN115065637B (en) | Method and device for transmitting computing power resource information and electronic equipment | |
| CN103067295A (en) | Method and device and system for service transmission | |
| CN105785905A (en) | Configurable field bus system and configuration method thereof | |
| CN104811403B (en) | Group list processing method, apparatus and group table configuration unit based on open flows | |
| CN105190530A (en) | Transmitting hardware-rendered graphical data | |
| CN110069288A (en) | USB device sharing method, apparatus and system | |
| CN106059964B (en) | Message forwarding method and device | |
| CN104158683A (en) | Cross-device aggregation group rapid convergence method, and cross-device aggregation group rapid convergence device | |
| CN110417687A (en) | A kind of message sends and receives method and device | |
| CN109413227A (en) | IPv4 user based on more IPv6 networks accesses IPv6 network interworking device and equipment | |
| CN111885564B (en) | Data transmission method, equipment upgrading method and computer readable storage medium | |
| CN108011802A (en) | A kind of file transmitting method and two three-layer network appliances | |
| CN106063169B (en) | A kind of data processing method and device | |
| CN104978162B (en) | Splice displaying system and its tiled display control method | |
| CN108965362A (en) | A kind of data processing method, server and storage medium | |
| CN110417573A (en) | A kind of method and system of data transmission | |
| CN205594417U (en) | Field bus system that can dispose | |
| CN105591902A (en) | Main-standby switching method and device | |
| CN105471726A (en) | Forwarding parameter passing method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |