WO2020199724A1 - 一种数据处理方法及服务器、设备 - Google Patents

一种数据处理方法及服务器、设备 Download PDF

Info

Publication number
WO2020199724A1
WO2020199724A1 PCT/CN2020/072336 CN2020072336W WO2020199724A1 WO 2020199724 A1 WO2020199724 A1 WO 2020199724A1 CN 2020072336 W CN2020072336 W CN 2020072336W WO 2020199724 A1 WO2020199724 A1 WO 2020199724A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
group
check value
server
data group
Prior art date
Application number
PCT/CN2020/072336
Other languages
English (en)
French (fr)
Inventor
赵勇
沈玺
孔丽
Original Assignee
中国银联股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国银联股份有限公司 filed Critical 中国银联股份有限公司
Publication of WO2020199724A1 publication Critical patent/WO2020199724A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/275Synchronous replication

Definitions

  • the present invention relates to the field of computer technology, in particular to a data processing method, server and equipment.
  • Data synchronization is a commonly used method of data processing. By synchronizing data on different devices (such as client devices and servers, client devices and storage devices, or servers and storage devices), the integrity of data on different devices can be maintained And unity.
  • the server detects that the resource data package installed on the server is upgraded from resource data package 1 to resource data package 2, then The resource data package 2 can be sent to the client device, so that the client device can update the resource data package installed on the client device from resource data package 1 to resource data package 2, thereby maintaining the resource data on the server and the client device The consistency of the package.
  • resource data package 2 there may be only some data differences between resource data package 2 and resource data package 1, and most of the data are the same; for example, the total data volume of resource data package 1 and resource data package 2 are both It is 50M. Only 5M data in resource data package 2 is different from resource data package 1, while other 45M data in resource data package 2 is the same as resource data package 1. If the above method is adopted, the client device will send the 50M data included in the resource data package 2 to the server. The 50M data may include the same 45M data as the resource data package 1 and different from the resource data package 1. 5M data. It can be seen that the server may transmit repeated data in the above manner, thereby losing more network traffic, resulting in low efficiency of data synchronization.
  • the embodiments of the present invention provide a data processing method, server, and equipment to improve the efficiency of data synchronization.
  • an embodiment of the present invention provides a data processing method, the method including:
  • the server obtains the first information sent by the client device, where the first information includes the check value of the N first data blocks corresponding to the first data packet, and the server performs the first test on the N first data blocks. Divide to obtain M groups of data, and generate the check value corresponding to each group of data according to the check value corresponding to at least one first data block included in each group of data and the second data packet corresponding to the first data packet Information; further, the server determines the difference information corresponding to the first data packet according to the verification information corresponding to the M sets of data and the second data packet, and sends the difference information to the client Terminal device, so that the client device updates the first data packet according to the difference information.
  • the server can transmit the difference information to the client device without sending the entire first data packet to the client device, thereby avoiding transmission Repeated data information reduces the loss of network traffic and improves the efficiency of data synchronization.
  • the check value of the data block is used to determine the check value of the data group, so that the client device and the server The synchronization process can be completed by performing only one communication interaction between them, without performing multiple communication interactions, thereby reducing the pressure of network communication and reducing network transmission traffic.
  • the data processing method in the embodiment of the present invention can be adapted to a low bandwidth and traffic-sensitive network environment.
  • the server determining that the maximum amount of data in the M sets of data is greater than or equal to the minimum amount of data in the data amounts of the N first data blocks.
  • the first information further includes the data amount of the N first data blocks, and the check value corresponding to the N first data blocks included in the first information is the N
  • the first check value corresponding to each first data block; the check value corresponding to at least one first data block included in each group of data and the second data packet corresponding to the first data packet are generated to generate
  • the verification information corresponding to each set of data includes: the M sets of data include a Yth data set, the Yth data set includes T first data blocks, and is specific to the Yth data block in the M sets of data.
  • the server determines the second check value corresponding to the Yth data group according to the first check values corresponding to the T first data blocks included in the Yth data group; the server Starting from the i-th bit data in the second data packet corresponding to the Y-1th data group, obtain a first target data group with the same amount of data as the Y-th data group, and perform a calculation on the first target data group Divided to obtain T second data blocks, the data amount of the T second data blocks is the same as the data amount of the T first data blocks; according to the first check value of the T second data blocks Determine the second check value corresponding to the first target data group; generate the second check value corresponding to the Yth data group and the second check value corresponding to the first target data group The verification information of the Y data group; the server updates the second data packet corresponding to the Y-1 group data to obtain the second data packet corresponding to the Y data group.
  • the first check value may be a strong check value
  • the second check value may be a weak check value.
  • the weak check value filters out the data information that is obviously the same as the Yth data group in the destination data group, thereby reducing the workload and improving the efficiency of data processing; on the other hand, the strong check value and the weak check value are used to determine the first data group.
  • the verification information corresponding to the Y data group can improve the accuracy of the verification information corresponding to the Yth data group.
  • the generating the check information of the Yth data group according to the second check value corresponding to the Yth data group and the second check value corresponding to the first target data group includes: If the server determines that the second check value corresponding to the Y-th data group is the same as the second check value corresponding to the first target data group, then according to the T first data included in the Y-th data group The first check value corresponding to each block is determined, the first check value corresponding to the Y-th data group is determined, and the corresponding first target data group is determined according to the first check value of the T second data blocks If it is determined that the first check value corresponding to the Yth data group is different from the first check value corresponding to the first target data group, then the first check value of the Yth data group is generated Check information; if the server determines that the second check value corresponding to the Y-th data group is different from the second check value corresponding to the first target data group, it will correspond to the Y-1th data group Start with the i+1
  • the method further includes: when the server determines that the first check value corresponding to the Yth data group is the same as the first check value corresponding to the first target data group, if it is determined that the first check value If the data volume of the Y data group is equal to the minimum data volume in the N first data blocks, the second check information of the Y-th data group is generated; if it is determined that the data volume of the Y-th data group is greater than the The smallest amount of data in the N first data blocks starts from the i+1-th bit data in the second data packet corresponding to the Y-1th data group, and obtains the same amount of data as the Yth data group The second target data group.
  • the verification information may include first verification information and second verification information.
  • the first verification information is verification information corresponding to the same data information as the Yth data group in the target data group
  • the second verification information The verification information may be verification information corresponding to data information different from the Yth data group in the target data group.
  • the method further includes: if the server determines that the data amount of the second data packet corresponding to the M-th group of data is greater than or equal to a preset data amount threshold, performing processing on the N first data blocks The second division.
  • the verification information obtained by multiple rounds of verification can be used to improve the accuracy of the verification information obtained by the server, thereby making the data synchronization process between the client device and the server more accurate;
  • the data volume of each group of data corresponding to the second round of verification can be less than the data volume of each group of data corresponding to the first round of verification. That is to say, the above technical solution can avoid using only one round of verification through multiple rounds of verification.
  • the technical problem of repeated transmission of data information caused by the verification can save network traffic and improve the efficiency of data synchronization.
  • the server divides the N first data blocks a second time to obtain P groups of data; if the server determines that the maximum amount of data in the P groups of data is less than the N first data blocks Is the smallest amount of data in the data amount, then the verification information of the first to Mth groups of data is sent to the client device.
  • an embodiment of the present invention provides a server, where the server includes:
  • An obtaining module configured to obtain first information sent by a client device, where the first information includes check values of N first data blocks corresponding to the first data packet;
  • the processing module is configured to divide the N first data blocks for the first time to obtain M groups of data, and to obtain M groups of data according to the sum of the check values corresponding to at least one first data block included in each group of data and the first data A second data packet corresponding to the packet, generating verification information corresponding to each group of data;
  • a transceiver module configured to determine the difference information corresponding to the first data packet according to the verification information corresponding to the M groups of data and the second data packet, and send the difference information to the client device, So that the client device updates the first data packet according to the difference information.
  • the processing module generates a check value corresponding to each group of data according to a check value corresponding to at least one first data block included in each group of data and a second data packet corresponding to the first data packet. Before the information, it is also used to determine that the maximum amount of data in the M groups of data is greater than or equal to the minimum amount of data in the data amounts of the N first data blocks.
  • the first information further includes the data amount of the N first data blocks, and the check value corresponding to the N first data blocks included in the first information is the N The first check value corresponding to each of the first data blocks;
  • the processing module is configured to: the M groups of data include a Y-th data group, and the Y-th data group includes T first data blocks, for The Yth data group in the M groups of data is executed: according to the first check value of the T first data blocks included in the Yth data group, the second check value corresponding to the Yth data group is determined ; Starting from the i-th bit data in the second data packet corresponding to the Y-1th data group, obtain a first target data group with the same amount of data as the Yth data group, and compare the first target data group Divide to obtain T second data blocks.
  • the data amount of the T second data blocks is the same as the data amount of the T first data blocks; according to the first check of the T second data blocks Value determines the second check value corresponding to the first target data group; according to the first check value corresponding to the Yth data group and the second check value corresponding to the first target data group, the The verification information of the Yth data group; updating the second data packet corresponding to the Y-1th data group to obtain the second data packet corresponding to the Yth data group;
  • the processing module is configured to: if it is determined that the second check value corresponding to the Yth data group is the same as the second check value corresponding to the first target data group, then according to the Yth data
  • the first check value corresponding to the T first data blocks included in the group is determined, and the first check value corresponding to the Y-th data group is determined, and the first check value is determined according to the first check value of the T second data blocks
  • the first check value corresponding to the first target data group if it is determined that the first check value corresponding to the Yth data group is different from the first check value corresponding to the first target data group, then all
  • the first check information of the Yth data group if it is determined that the second check value corresponding to the Yth data group is different from the second check value corresponding to the first target data group, then start from the Yth data group. Start with the i+1-th data in the second data packet corresponding to the -1 data group, and acquire a second target data group with the same amount of
  • the processing module is further configured to: when it is determined that the first check value corresponding to the Yth data group is the same as the first check value corresponding to the first target data group, if the Yth data group is determined If the data amount of the data group is equal to the minimum data amount in the N first data blocks, then the second check information of the Yth data group is generated; if it is determined that the data amount of the Yth data group is greater than the N The smallest amount of data in the first data block, starting from the i+1-th bit data in the second data packet corresponding to the Y-1th data group, obtain the same amount of data as the Yth data group The third target data group.
  • the processing module is further configured to: if it is determined that the data amount of the second data packet corresponding to the M-th group of data is greater than or equal to a preset data amount threshold, perform the first data block on the N first data blocks. Secondary division.
  • the processing module is configured to: perform a second division on the N first data blocks to obtain P groups of data; if it is determined that the maximum amount of data in the P groups of data is less than the N first data If the data amount of the block is the smallest, the verification information of the first to Mth groups of data is sent to the client device.
  • a computer-readable storage medium provided by an embodiment of the present invention includes instructions that, when run on a computer, cause the computer to execute the data processing method as described in the first aspect or any of the first aspects.
  • a computer program product provided by an embodiment of the present invention, when it runs on a computer, causes the computer to execute the data processing method described in the first aspect or any of the first aspects.
  • a data processing device provided by an embodiment of the present invention includes:
  • At least one processor and,
  • a memory communicatively connected with the at least one processor; wherein,
  • the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute any of the above-mentioned first aspect or any of the first aspect.
  • FIG. 1 is a schematic flowchart corresponding to a data processing method provided by an embodiment of the present invention
  • FIG. 2 is a schematic diagram of interaction between a client device and a server according to an embodiment of the present invention
  • Figure 3a is a schematic diagram of an embodiment provided in an embodiment of the present invention.
  • Figure 3b is a schematic diagram of an embodiment provided in an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a server provided by an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram corresponding to a data processing device provided by an embodiment of the present invention.
  • one possible implementation is: to perform the resource data package 2 installed in the server with the resource data package 1 installed in the client device
  • the verification information between the resource data package 2 and the resource data package 1 is determined.
  • the verification information can be used to identify the difference information between the file a2 and the file a1, so that the server can send the difference information to the client device, So that the client device updates the resource data package 1.
  • the executor of determining the verification information may be a client device or a server, which is not specifically limited.
  • the server Take the server as an example to determine the execution subject of the verification information.
  • the server can obtain accurate difference information based on the accurate verification information, so that the server can compare the accurate
  • the difference information is sent to the client device to improve the efficiency of data synchronization.
  • the accuracy of the verification information will affect the accuracy of the difference information. If you want to ensure the accuracy of the difference information, the server needs to obtain accurate verification information.
  • FIG. 1 is a schematic flowchart corresponding to a data processing method provided by an embodiment of the present invention, and the method includes:
  • Step 101 The server obtains the first information sent by the client device.
  • the first data packet may be stored in the client device, and the second data packet may be stored in the server.
  • the second data packet in the server may be a data packet obtained by updating a third data packet stored in the server, and the third data packet may be the same as the first data packet or may be different from the first data packet.
  • the same resource data package can be installed on both the server and the client device.
  • the resource data package installed on the client device is the resource data package corresponding to version V1.1 (that is, the first data package), and the resource data package installed on the server
  • the resource data package is the resource data package corresponding to version V1.3 (that is, the second data package); in an example, if the resource data package corresponding to the version V1.3 installed on the server is the version V1.1 originally installed on the server
  • the corresponding resource data package (that is, the third data package) is obtained by upgrading, that is, the version V1.3 is obtained by upgrading the version V1.1, then in this example, the first data package and the third data package can be the same (All resource data packages corresponding to version V1.1); in another example, if the resource data package corresponding to version V1.3 installed on the server is the resource data package corresponding to the version V1.2 originally installed on the server ( That is, the third data package) is obtained by upgrading, that is, version V1.3 is obtained by upgrading version V1.2, and version V1.2 is obtained by upgrading version V1.1.
  • the first The first data packet and the third data packet may
  • the following embodiments of the present invention take the process of using the second data packet in the server to synchronize the data of the first data packet in the client device as an example to illustrate the solution in the embodiment of the present invention. It is understandable that other devices perform data synchronization.
  • the data synchronization process can also be implemented with reference to the data synchronization process in the embodiment of the present invention, which is not specifically limited.
  • the first information may include the data amount of the N first data blocks corresponding to the first data packet and the check values respectively corresponding to the N first data blocks.
  • the client device may divide the first data packet into N first data blocks according to the data volume of the first data packet. There are many ways to divide the first data packet. In a possible implementation, the client device can divide the first data packet into N first data blocks according to a preset data amount, each The data amount of the first data block can be the same.
  • the data volume of the first data packet is 905M
  • the first data packet can be divided to obtain the first data block b1 to the first data block b10; among them, the first data block b1 to the ninth first data block b9
  • the data volume of each first data block in the data block may be 100M
  • the data volume of the first data block b10 may be 5M.
  • the check value corresponding to each first data block may be determined.
  • the check value corresponding to each first data block may be the first check value (strong check value) of each first data block, or it may be the second check value (strong check value) of each data block. value).
  • the check value of the first data block b1 to the first data block b10 included in the first information may be the first check value of the first data block b1 to the first data block b10, or may also be the first check value.
  • the check value is not specifically limited.
  • the first check value may be a strong check value, such as an MD5 check value.
  • the MD5 check value can be a key obtained through the MD5 hash method with salting or the MD5 hash method without salt.
  • the acquisition process is an irreversible encryption process, that is, the MD5 check value corresponding to a certain data Unable to obtain information about the data.
  • the same encryption method such as the MD5 hash method with salt
  • the key corresponding to the first data is the same as the key corresponding to the second data.
  • the first data block b1 can be divided into 512 first groups in advance, and each first group is divided into 32 first sub-groups. Grouping. Furthermore, by using MD5 hash to encrypt the 512 first packets, four 32-bit second sub-packets can be obtained. After concatenating these four 32-bit second sub-packets, one can be obtained. 128-bit hash value. Wherein, the hash value may be the first check value corresponding to the first data block b1. The reliability of the first check value is strong. If the first check values corresponding to the N first data blocks are used to obtain the check information, the accuracy of the check information can be submitted.
  • the second check value may be a weak check value, such as a check value determined based on a checksum algorithm (that is, the Asler32 algorithm).
  • the weak check value of the first data block may be obtained by adding up each field included in the first data group, and the obtained sum value may be used as the first checksum value; accordingly, The sum of the phase values of each step in the process of determining the first checksum value is used as the second checksum value.
  • the first checksum value and the second checksum value can both be 16-bit values, and further, the 16-bit first checksum value and the 16-bit second checksum value can be spliced into one A 32-bit integer, that is, the second check value of the first data block.
  • Step 102 The server divides the N first data blocks to obtain M groups of data.
  • the server may divide the N first data blocks into M groups of data according to the preset number of data blocks. , Wherein the number of data blocks included in each group of data can be the same.
  • the server may divide the first data block b1 to the first data block b10 into the first data group to the fourth data group, wherein each of the first data group to the third data group includes data
  • the number of blocks may be three, and the number of data blocks included in the fourth data group may be one.
  • the first data group includes first data block b1 to first data block b3, the second data group includes first data block b4 to first data block b6, and the third data group includes first data block b7 to first data.
  • Block b9, the fourth data group includes the first data block b10.
  • the data volume of each data group in the first data group to the third data group may be 300M, and the data volume of the fourth data group may be 5M.
  • Step 103 The server determines the verification information of each group of data in the M groups of data.
  • the server determines that the maximum amount of data in the M groups of data is greater than or equal to the minimum amount of data in the N first data blocks, it indicates that the server verifies the second data packet The process is not over, you can continue to perform the subsequent steps.
  • the smallest data amount among the data amounts of the N first data blocks may refer to the data amount of the first data block with the smallest data amount among the N first data blocks.
  • the smallest data amount among the data amounts of the first data block b1 to the first data block b10 may refer to the data amount of the first data block b10, that is, 5M.
  • the above implementation is only an exemplary and simple description, and the conditions listed for determining that the process of verifying the second data packet by the server is not completed are only for the convenience of explaining the solution, and do not constitute a solution to the solution.
  • the condition for determining that the process of verifying the second data packet by the server has not ended can be set by those skilled in the art according to the actual situation. For example, a preset data volume threshold can be set.
  • the preset data amount threshold may be a fixed value, or it may be based on the data in the first data block.
  • the value obtained by determining the minimum data amount of, for example, may be 3/4 of the minimum data amount in the first data block, which is not specifically limited.
  • the second data group may include the first data block b4 to the first data block b6. In specific implementation, it may be based on the first data block b4 to the first data block b6 included in the second data group. Respectively corresponding check values, determine the check value corresponding to the second data group. Several possible scenarios for determining the check value corresponding to the second data group are described below.
  • the check value of the first data block included in the first information may be the first check value of the first data block, that is, the first data block b4 to the first data sent in step 101 Block b6 respectively corresponds to the first check value.
  • the first check value corresponding to the second data group may be determined according to the first check value corresponding to the first data block b4 to the first data block b6, or the first check value corresponding to the first data block b4 ⁇
  • the first check value corresponding to the first data block b6 to determine the second check value corresponding to the second data group, or the first check value corresponding to the first data block b4 to the first data block b6 respectively
  • the determination of the first check value and the second check value corresponding to the second data group is not specifically limited. The following describes these three sub-situations respectively.
  • the first check value corresponding to the second data group is determined according to the first check value corresponding to the first data block b4 to the first data block b6, then the first data block b4 to the first data block b6 can be respectively corresponding to
  • the first check value of is divided into 3 groups, and each group is divided into 32 sub-groups. Further, by using MD5 hashing to encrypt three packets, multiple 32-bit sub-packets can be obtained, and after multiple 32-bit sub-packets are concatenated, a 128-bit hash value can be obtained.
  • the hash value can be the first check value corresponding to the second data group.
  • the first checksum value corresponding to the second data group is determined according to the first check value corresponding to the first data block b4 to first data block b6, then the first data block b4 to first data block b6 can be compared A checksum value is added, and the obtained sum can be used as the first checksum value; accordingly, the sum of the phase values of each step in the process of determining the first checksum value can be used as the second checksum And value. Further, the first checksum value and the second checksum value may be spliced into an integer, and the integer may be the second check value corresponding to the second data group.
  • the first checksum value corresponding to the first data block b4 to the first data block b6 are R1, R2, and R3, respectively, each byte in R1, R2, and R3 can be added and summed to The obtained sum value is used as the first checksum value; accordingly, the sum of the phase values of each step in the process of determining the first checksum value can be used as the second checksum value.
  • the first checksum value and the second checksum value can both be 16-bit values, and further, the 16-bit first checksum value and the 16-bit second checksum value can be spliced into one A 32-bit integer, that is, the second check value corresponding to the second data group.
  • first check value and the second check value corresponding to the second data group are determined according to the first check values respectively corresponding to the first data block b4 to the first data block b6, the second data can be determined separately in the above manner The first check value and the second check value corresponding to the group will not be repeated here.
  • the check value of the first data block included in the first information may be the second check value of the first data block, that is, the first data block b4 to the first data sent in step 101 Block b6 respectively corresponds to the second check value.
  • the first check value corresponding to the second data group may be determined according to the second check value corresponding to the first data block b4 to the first data block b6, or the first check value corresponding to the first data block b4 ⁇
  • the second check value corresponding to the first data block b6 to determine the second check value corresponding to the second data group, or the second check value corresponding to the first data block b4 to the first data block b6 respectively
  • the determination of the first check value and the second check value corresponding to the second data group is not specifically limited. The following describes these three sub-situations respectively.
  • first check value corresponding to the second data group is determined according to the second check value corresponding to the first data block b4 to the first data block b6, then the first data block b4 to the first data block b6 can be respectively corresponding to
  • the second check value of is divided into 3 groups, and each group is divided into 32 sub-groups. Further, by using MD5 hashing to encrypt three packets, multiple 32-bit sub-packets can be obtained, and after multiple 32-bit sub-packets are concatenated, a 128-bit hash value can be obtained.
  • the hash value may be the second check value corresponding to the second data group.
  • the first data block b4 to first data block b6 can be compared
  • the sum of the two checksum values can be used as the first checksum value; accordingly, the sum of the phase values of each step in the process of determining the first checksum value can be used as the second checksum value And value.
  • the first checksum value and the second checksum value may be spliced into an integer, and the integer may be the second check value corresponding to the second data group.
  • first check value and the second check value corresponding to the second data group are determined according to the second check values respectively corresponding to the first data block b4 to the first data block b6, the second data can be determined respectively in the above manner The first check value and the second check value corresponding to the group will not be repeated here.
  • the server may verify the second data packet corresponding to the first data group according to the check value of the second data group.
  • the second data packet corresponding to the first data group may be a data packet obtained after performing step 103 for the first data group. Specifically, after verifying the second data packet according to the check value of the first data group, the second data packet can be updated to obtain the second data packet corresponding to the first data group; accordingly, according to After the check value of the second data group is checked against the second data packet corresponding to the first data group, the second data packet corresponding to the first data group can be updated to obtain the second data packet corresponding to the second data group . It should be noted that the process of obtaining the second data packet corresponding to the third data group and the second data packet corresponding to the fourth data group can be performed with reference to the first data group and the second data group, and will not be repeated.
  • the server may, after obtaining the second data packet corresponding to the first data group stored in the server, start with the i-th bit data in the second data packet corresponding to the first data group, obtain the data corresponding to the first data group Target data groups with the same data volume can be divided to obtain T second data blocks; wherein the data volume of the T second data blocks can be the same as the data volume of the T first data blocks. Further, the server may determine the check value corresponding to the target data group according to the check value of the T second data blocks. For example, in the first sub-situation, the server may determine the check value corresponding to the first check value of the T second data blocks.
  • the first check value corresponding to the target data group is determined, and the second data packet can be checked according to the first check value of the second data group and the first check value corresponding to the target data group to generate second data
  • the check information corresponding to the group in the second sub-case, the server can determine the second check value corresponding to the target data group according to the first check value of the T second data blocks, and can according to the first check value of the second data group
  • the second check value and the second check value corresponding to the target data group are checked for the second data packet to generate the check information corresponding to the second data group
  • the server in the third sub-case, the server can check the second data group according to T
  • the first check value of the data block determines the first check value and the second check value corresponding to the target data group, and can be based on the first check value of the second data group and the first check value corresponding to the target data group
  • the second check value of the second data group and the second check value corresponding to the target data group are checked for the second data packet to generate
  • the server may determine the first check value corresponding to the target data group according to the second check value of the T second data blocks, and may determine the first check value corresponding to the second data group The first check value corresponding to the target data group is checked for the second data packet to generate the check information corresponding to the second data group; in the fifth sub-scenario, the server can be based on the second data block of T The second check value determines the second check value corresponding to the target data group, and can check the second data packet according to the second check value of the second data group and the second check value corresponding to the target data group, Generate the check information corresponding to the second data group; in the sixth sub-case, the server may determine the first check value and the second check value corresponding to the target data group according to the second check values of the T second data blocks , And according to the first check value of the second data group and the first check value corresponding to the target data group, the second check value of the second data group and the second check value corresponding to the target
  • the second sub-scenario is taken as an example to specifically describe the implementation process of verifying the second data packet and generating the verification information corresponding to the second data group.
  • the server can download the second data group corresponding to the first data block.
  • the server can download the second data group corresponding to the first data block.
  • Starting with the first bit of data in the data packet sequentially obtain 300M data as the target data group, and divide the target data group into second data block c4 to second data block c6; among them, the second data block c4 to second data
  • the data amount of each second data block in block c6 may be 100M.
  • the server can determine the first check value of the second data block c4 to the second data block c6, and can determine the first check value of the target data group according to the first check value of the second data block c4 to the second data block c6 Second check value (this can be performed with reference to the process of determining the second check value of the second data group, and will not be repeated here).
  • the server may compare the second check value corresponding to the second data group with the second check value corresponding to the target data group, and if it is determined that the second check value corresponding to the second data group corresponds to the target data group If the second check value is the same, the first check value corresponding to the second data group can be determined according to the first check value corresponding to the T first data blocks included in the second data group, and the first check value corresponding to the second data group can be determined according to the T first data blocks. The first check value of the two data blocks determines the first check value corresponding to the target data group. Take the process of determining the first check value corresponding to the second data group as an example.
  • the first check value corresponding to the first data block b4 to the first data block b6 included in the second data group is R1 , R2, and R3, you can combine R1, R2, and R3 into one value, and then calculate the HD5 hash value of the value, that is, the first check value corresponding to the second data group.
  • the server may generate the first check information of the second data group.
  • the first check information of the second data group may include bit information (that is, identification information) corresponding to the same target data group in the second data packet as the first data packet.
  • the first check information of the second data group may be: the 1M to 300M data in the second data packet and The second data group (the 301Mth data to the 601Mth data) in the first data packet is the same.
  • the second data packet corresponding to the first data group may also be updated.
  • the target data group in the second data packet corresponding to the first data group that is, data from 1M to 300M
  • the target data group in the second data packet corresponding to the first data group can be replaced with holes, or the data from 1M to 300M can be modified to special characters, or
  • the data from the 1M to the 300M can also be labeled, which is not specifically limited.
  • the server may start with the next bit of data in the target data group and execute the above steps. It should be noted that if the target data group is the same as the second data group, in the second execution process, the server can start from the first data after the target data group is removed from the second data packet and execute the above process. After the second data group is used to execute each data in the second data packet corresponding to the first data group, the server can sequentially update the second data packet corresponding to the first data group through the above execution steps to obtain the first data group The second data packet corresponding to the second data group.
  • the server can determine whether the division process is over (all the bit data in the second data packet has become holes and/or all have been marked, or the data amount of the second data group is equal to the minimum data amount in the N first data blocks, then Make sure the division process is over). If the server determines that the division process has not ended, it can use the third data group to start from the first bit of data corresponding to the second data group to perform the above verification process.
  • the server can generate the second check information of the second data group, and can use the third The data group starts from the first bit of data corresponding to the second data group, and executes the above-mentioned verification process.
  • the second check information may include bit information (that is, identification information) corresponding to a target data group different from the first data packet in the second data packet.
  • the second check information of the second data group may be: the 1M to 300M data in the second data packet and The first data packet is different.
  • the first check value may be a strong check value
  • the second check value may be a weak check value.
  • the first verification information is verification information corresponding to the same data information as the Yth data group in the target data group
  • the second verification information It can be the check information corresponding to the data information different from the Yth data group in the target data group.
  • Step 104 If the server determines that the data volume of the second data packet corresponding to the M-th data group is less than the preset data volume threshold, step 105 is executed; if it is determined that the data volume of the second data packet corresponding to the M-th data group is greater than or equal to the preset data volume. If the data volume threshold is set, step 102 is executed.
  • the server after the server uses the third data group to verify the second data packet corresponding to the second data group, it can update the second data packet corresponding to the second data group to generate the third data group corresponding The second data packet. Further, if the server determines that the data volume of the second data packet corresponding to the third data group is less than the preset data volume threshold, it means that all the data in the second data packet is the same as the data in the first data packet (ie, the second data All bits of data in the packet become holes and/or all have been marked, and the verification process ends); at this time, the server can execute step 105.
  • the preset data amount threshold can be set by those skilled in the art based on experience, and is not specifically limited.
  • the server may divide the N first data blocks for the second time. If the N first data is divided into P data groups, the data blocks included in each of the P data groups are The number may be less than or equal to the number of data blocks included in each of the M data groups obtained by the first division. That is, P can be greater than or equal to M.
  • the server determines that the data volume of each group of data in the P group of data is less than the minimum data volume among the data volumes of the N first data blocks, it indicates that the process of verifying the second data packet by the server has ended. At this time, the server can send the verification information corresponding to the M groups of data to the client device. Correspondingly, if the server determines that the largest amount of data in the P group of data is greater than or equal to the smallest amount of data in the N first data blocks, it indicates that the process of verifying the second data packet by the server has not ended. At this time, The server may perform step 103 to step 105 (or 102) for each of the P data groups.
  • the verification information obtained by the multi-round verification method can improve the accuracy of the verification information obtained by the server, thereby making the data synchronization process between the client device and the server more accurate;
  • the data volume of each group of data corresponding to the second round of verification may be less than the data volume of each group of data corresponding to the first round of verification. That is to say, the embodiment of the present invention adopts multiple rounds of verification to avoid only using The technical problem of repeated transmission of data information caused by a round of verification can save network traffic and improve the efficiency of data synchronization.
  • Step 105 The server determines the difference information according to the verification information of the M groups of data and the second data packet, and sends the difference information to the client device.
  • the verification information of the M groups of data may include only the first verification information, or may also only include the second verification information. Information, or may also include first verification information and second verification information. If the verification information of the M sets of data only includes the first verification information, it means that the data in the second data packet is the same as the data in the first data packet; if the verification information of the M sets of data only includes the second verification information, It shows that the data in the second data packet is completely different from the data in the first data packet; if the verification information of the M groups of data includes the first verification information and the second verification information, it means that part of the data in the second data packet is The data in the first data packet is the same, and another part of the data is different from the data in the first data packet.
  • the server can obtain the 1M data in the second data packet.
  • To the 300Mth data that is, difference information
  • the client device can parse the second information to obtain the difference information, and update the second data packet according to the difference information. For example, if the difference information parsed by the client device is data from the 1M to 300M in the second data packet, the client device may update the data from the 1M to 300M in the first data packet to the second data Data from 1M to 300M in the package.
  • step number is only an example of the execution process, and does not constitute a limitation on the order of execution of each step.
  • the server can transmit the difference information to the client device without sending the entire first data packet to the client device, thereby The transmission of repeated data information can be avoided, the loss of network traffic is reduced, and the efficiency of data synchronization is improved; moreover, in the embodiment of the present invention, the check value of the data group is determined by the check value of the data block, so that the client The synchronization process can be completed by performing only one communication interaction between the device and the server without performing multiple communication interactions, thereby reducing the pressure of network communication and reducing network transmission traffic.
  • the data processing method in the embodiment of the present invention can be adapted to a low-bandwidth and traffic-sensitive network environment.
  • FIG. 2 is a schematic diagram of a flow of interaction between a client device and a server according to an embodiment of the present invention. The method includes:
  • Step 201 The client device sends the first information to the server.
  • the first information may include the data amount of the N data blocks and the first check value corresponding to the N data blocks, where the N data blocks may be obtained by dividing the first data packet by the client device .
  • the first information may also include other information related to the first data packet, such as the identification information of the first data packet, the bit information of the N data blocks, and the update time of the N data blocks. limited.
  • Step 202 The server determines the verification information based on the first information, and determines the difference information based on the verification information and the second data packet.
  • the server may parse the first information to obtain the identification information of the first data packet included in the first information. Further, the server may obtain the second data packet corresponding to the first data packet stored in the server according to the identification information of the first data packet, and may determine the verification information according to the first information and the second data packet.
  • the specific implementation process can refer to the process illustrated in step 102 to step 105 (or step 102), which is not specifically limited.
  • bit refers to bit data in serial communication, which may include 6 bits or 9 bits, which is not specifically limited.
  • Step 203 The server sends the difference information to the client device.
  • the server may send the difference information to the client device through a wired method (such as network cable, optical fiber), or may also send the difference information to the client device through a wireless method (such as Bluetooth, electromagnetic waves), which is not specifically limited.
  • a wired method such as network cable, optical fiber
  • a wireless method such as Bluetooth, electromagnetic waves
  • Step 204 The client device updates the first data packet stored in the client device according to the difference information.
  • the difference information may include difference data and identification information
  • the client device may update the data in the first data packet to the difference data according to the identification information
  • FIG. 2 is only an exemplary simple description, and the use of the server as the execution body for determining the verification information is only for the convenience of explaining the solution, and does not constitute a limitation on the solution.
  • the client device may also be used as the execution subject for determining the verification information, which is not limited in the embodiment of the present invention.
  • Example 1 uses Example 1 and Example 2 to specifically describe the solution in the embodiment of the present invention.
  • Fig. 3a is a schematic diagram of an embodiment corresponding to a data processing method provided by an embodiment of the present invention.
  • an old resource data package a (abbreviated as data a) is stored on the client device, and the server stores data A corresponds to the new resource data packet b (referred to as data b for short).
  • data a and data b can be 48-bit data.
  • 1 bit may refer to a relatively small amount of data such as 1 byte and 1 bit, or may also refer to a relatively large amount of data such as 1M and 1G, which is not limited.
  • Example 1 and Example 2 are only two exemplary simple descriptions, and the data a and data b listed are only for the convenience of explaining the solution, and do not constitute a solution to the solution. Limited, in specific implementation, the data volume of data a and data b can be much larger than 48 bits, for example, it can be 100G or 10T.
  • the client sends the first information to the server.
  • the client device may divide the data a to obtain data blocks S 1 to S 8 , where each data block of the data blocks S 1 to S 8 may include 6 bits of data.
  • the data block S 1 includes a first to sixth data bit data, i.e., data ABCD12;
  • data block S 2 comprises seventh to twelfth data bits data, i.e., data ACDE23;
  • S 3 comprises a data block The thirteenth data to the eighteenth data, namely data ABDE34;
  • the data block S 4 includes the nineteenth data to the twenty-fourth data, namely data ADEF45;
  • the data block S 5 includes the twenty-fifth data to The thirtieth bit of data is data BDEF56;
  • the data block S 6 includes the thirty-first to thirty-sixth bit data, that is, data BCEF67;
  • the data block S 7 includes the thirty-seventh bit to the forty-second bit
  • the data is the data BEDA78;
  • the data block S 8 includes the
  • the client device may calculate the first detection value and the second detection value of the data block S 1 to the data block S 8 respectively, the first detection value may be a strong check value, and the second detection value may be a weak check value .
  • the client device may determine the data block a check value weakly S 1 through checksum algorithm, and may determine the data block S 1 through MD5 hash algorithm Strong check value.
  • embodiments of the present invention may be calculated check value and the weak strong check value using the data block comprises a block of data, for example, which is a strong data block comprising data ABCD12 S 1 calculates the data block S 1 Check value and weak check value.
  • the client device may generate the first information according to the identification information, data volume information, weak check value, and strong check value of data block S 1 to data block S 8 , and send the first information to server.
  • identification of data bits data block identification information may be included in a data block, such data block identification information S 3 for the Nineteenth ⁇ 24th bit data transactions; data amount of the information data blocks may be data blocks It comprises a number of bits of data, such as data-block S 3 data amount information of six bits.
  • the identification information, data volume information, weak check value, and strong check value of the data block S 1 to the data block S 8 may be stored in the first information in the form of a table.
  • Table 1 is An example of the table stored in the first information. As shown in Table 1, taking the data block S 1 as an example, the identification information of the data block S 1 is the first data to the sixth data, the data amount information is six bits of data, the weak parity value is H 1 , the strong correction The test value is U 1 .
  • Table 1 An illustration of the table in the first message
  • the server divides the data block S 1 to the data block S 8 for the first time.
  • the server After the server receives the first information sent by the client device, it can parse the first information to obtain the identification information, data volume information, weak check value, and strong check value of the data block S 1 to data block S 8 , and can compare the data Block S 1 to data block S 8 are divided for the first time to obtain data group M 11 .
  • the data group M 11 may include data blocks S 1 to S 8 , that is, the identification information of the data group M 11 is the first data to the forty-eighth data, and the data amount information is 48-bit data; and, The server may determine the weak check value of the data group M 11 according to the strong check value of the data block S 1 to the data block S 8 (that is, U 1 to U 8 ).
  • the server may start from the first bit of data b to obtain a target data group with the same amount of data as the data group M 11 , and the target data group may include 48-bit data in data b, as shown in FIG. 3a, the target The data group is data b, and the 48-bit data included in the target data group can be "ABCD12ACDE23ABDE34ADEF45BDEF56BCEF67BEDA78DECA90".
  • the server can divide the target data group into eight data blocks with the same amount of data as data block S 1 to data block S 8 (eight data blocks as shown in Figure 3a), and can determine the weak parity of each data block And the strong check value, and then determine the weak check value of the target data group according to the strong check value of the eight data blocks.
  • the server compares the weak check value of the data group M 11 with the weak check value of the target data group, and determines that the weak check value U 11 of the data group M 11 is different from the weak check value of the target data group. At this time, the server can determine that the data group M 11 is different from the target data group.
  • the server divides the data block S 1 to the data block S 8 for the second time.
  • the server can divide the data block S 1 to the data block S 8 for the second time to obtain the data group M 21 and data group M 22 .
  • the data group M 21 may include data blocks S 1 to data blocks S 4
  • the identification information of the data group M 11 is the first data to the twenty-fourth data
  • the data amount information is 24 bits of data.
  • the data can be "ABCD12ACDE23ABDE34ADEF45"; the data group M 22 can include data block S 5 to data block S 8 , that is, the identification information of the data group M 22 is the twenty-fifth bit data to the forty-eighth bit data, and the data amount information For 24-bit data, the 24-bit data can be "BDEF56BCEF67BEDA78CEDF89".
  • the server can determine the weak check value of the data group M 21 according to the strong check value of the data block S 1 to the data block S 4 (that is, U 1 to U 4 ), and can determine the weak check value of the data group M 21 according to the data block S 5 to the data block S 8
  • the strong check value (that is, U 5 to U 8 ) determines the weak check value of the data group M 22 .
  • the server may begin the first bit of data from the data b, the amount of M data group acquires data 21 of the same target data set (data set shown in Figure 3a M '21).
  • the data sets M '21 b may include data of a first data to the twenty-four-bit data, the amount of data is 24 bits, which may be a 24-bit data "ABCD12ACDE23ABDE34ADEF45".
  • the data server may set M '21 is divided into data blocks and data blocks S 1 ⁇ S 4 in the same amount of data of four data blocks (i.e., each data block may comprise 6-bit data), and each data block is determined weak strong check value and the check value, and to determine the data set M 'check value 21 weak strong check value based on four data blocks.
  • the server can determine the strong check value of the data group M 21 according to the strong check value of the data block S 1 to the data block S 4 (ie U 1 to U 4 ), and can also determine the strong check value of the data group M 21 according to the four data blocks in the target data group M'21 strong check value to determine the target data set M 'strong check value 21; furthermore, the server may set the data and the check value M strong target data set 21 of M' check value 21 is strong contrast, determining the data group M strong target data set and the check value 21 'of the same intensity check value 21, i.e. the data set and the data set M 21 M' identical M 21; in this case, the server may generate the first verification information.
  • Table 2 is an example of the first verification information.
  • the first check information may be: the first bit to the twenty-fourth bit data in the data a are the same as the first bit to the twenty-fourth bit data in the data b.
  • the server can update the data b after generating the first verification information.
  • the server can update the first to twenty-fourth data in the data b to holes, so as to obtain the updated ⁇ b.
  • the first to twenty-fourth data in the updated data b may be holes.
  • the server may perform the verification process according to the updated data b.
  • the server may start with the first non-hole data (ie, the twenty-fifth data) of the updated data b, and obtain the data and M amount of data group data 21 of the same target data set (for ease of description, a data set is called M '22), the data set M' 22 may include twenty-five data b to the forty-eighth bit data ,
  • the data volume information is 24 bits, the 24 bits of data can be "BDEF56BCEF67BEDA78DECA90".
  • the data server may set M '22 is divided into data blocks and data blocks S 1 ⁇ S 4 in the same amount of data of four data blocks, and may determine the weak and strong checksum value for each data block a check value, and further in accordance with strong four check value set data block M determination data weak check value '22.
  • M server data set and the data set of weak check value of 21 M 'weak check value 22 are compared, the data set may be determined and a check value M low data set 21 of M' are different check value 22 is weak, this , the server may determine that the data set 22 'with a different data set M 21 M.
  • the server After executing this step, the server has completed the verification process of the updated data b using the data group M 21. Further, the server can use the data group M 22 to verify the updated data b. For the specific execution process, please refer to The above content will not be repeated in detail.
  • the server uses the data group M 22 to verify the updated data b, the data b does not have the same data as the data group M 22. Therefore, the server does not generate verification information.
  • the server divides the data block S 1 to the data block S 8 for the third time.
  • the server S can be data block S 1 ⁇ 8 a third data block Divide to obtain data group M 31 , data group M 32 , data group M 33 and data group M 34 .
  • the data group M 31 may include a data block S 1 and a data block S 2
  • the identification information of the data group M 31 may be the first data to the twelfth data
  • the data amount information may be 12-bit data.
  • the bit data can be "ABCD12ACDE23";
  • the data group M 32 can include a data block S 3 and a data block S 4 , and the identification information of the data group M 32 can be the thirteenth bit to the twenty-fourth bit data, the data amount information It can be 12-bit data, and the 12-bit data can be "ABDE34ADEF45";
  • the data group M 33 can include data block S 5 and data block S 6 , and the identification information of data group M 33 can be the twenty-fifth data ⁇ the first Thirty-six bits of data, the amount of data information can be 12 bits of data, which can be "BDEF56BCEF67";
  • data group M 34 can include data block S 7 and data block S 8 , that is, the identification information of data group M 33 It is the 37th to the forty-eighth data, the data amount information is 12-bit data, and the 12-bit data can be "BEDA78CEDF89".
  • the server can determine the weak check value of the data group M 31 according to the strong check value of the data block S 1 and the data block S 2 (ie U 1 and U 2 ), and according to the data block S 3 and the data block S 4
  • the strong check value (ie U 3 and U 4 ) determines the weak check value of the data group M 32
  • the data group M is determined according to the strong check value of the data block S 5 and the data block S 6 (ie U 5 and U 6 )
  • the weak check value of the data group M 34 is determined according to the strong check value of the data block S 7 to the data block S 8 (ie, U 7 to U 8 ).
  • the server starts from the first data (ie the twenty-fifth data) of the updated data b, and obtains the target data group with the same amount of data as the data group M 33 (as shown in Figure 3a).
  • Schematic M'31 ).
  • the data sets M '31 may include twenty-five data b is data to the third sixteen, 12-bit data amount information, which may be a 12-bit data "BDEF56BCEF67".
  • the data server may set M '31 is divided into data blocks and data blocks.
  • Data server 33 is set low and M check value set data M 'weak check value 31 are compared, and the data set weak check value M may be determined data set 33 is M' of the same weak check value 31; this , the server S can check the value of intensity of the data block S 2.
  • the server may check value and the strong target data set 33 is a set of M data M' M strong target data set the check value 31 are compared, determining a check value and the data set strong target data set M M 33 'of the same intensity check value 31, i.e., data sets with a data set of M 33 M' identical 31; in this case, the server may generate the first verification information, such as Table 3 shows.
  • the server may turn the 25th to 36th bits of data in the data b into holes to obtain the updated data b.
  • the first data to the thirty-sixth data in the updated data b may be holes.
  • the server may begin 37th data from the data b, and obtain the same amount of data of the M 33 sets target data set, the data set of weak weak parity check value M 33 and the target data set The comparison result of the value, and the comparison result of the strong check value of the data group M 33 and the strong check value of the target data group, the verification process is executed, and the details can be referred to the above content, and will not be repeated.
  • the server divides the data block S 1 to the data block S 8 for the fourth time.
  • the server can compare data block S 1 ⁇ S 8 data blocks for the fourth divided data blocks to obtain a data block Sl - S8.
  • the verification is performed on the data block S1 to the data block S8 and the updated data b to obtain the verification result.
  • the specific implementation process please refer to the above content and will not be repeated.
  • the verification result of the fourth stage may be as shown in Table 4.
  • the 37th to 42nd data of data a are the same as the 37th to 42nd data of data b, and the 42nd to fourth data of data a
  • the eighteen-bit data is different from the forty-second to forty-eighth data of data b.
  • the server sends the difference information to the client device.
  • the server After the server obtains the verification results of the first to fourth stages (that is, as shown in Table 2 to Table 4), it can determine the same data in data b and data a according to the verification results, and can obtain the same data in The identification information in the data a and the identification information in the data b (that is, the difference information), and then the two identification information can be sent to the client device.
  • the first to twenty-fourth data in data a is the same as the first to twenty-fourth data in data b.
  • the 25th to 36th bit of data is the same as the 25th to 36th bit of data b, and the 37th to 42nd bit of data a
  • the data is the same as the 37th to 42nd data in data b
  • the forty-second to forty-eighth data in data a is the same as the forty-second data in data b ⁇
  • the forty-eighth data is different; at this time, the server can send the identification of the same data in data a and the identification in data b to the client device.
  • Table 5 is a schematic table of difference information provided by an embodiment of the present invention.
  • Data a Data b The first data ⁇ the twenty-fourth data The first data ⁇ the twenty-fourth data Twenty-fifth data to thirty-sixth data Twenty-fifth data to thirty-sixth data 37th bit data ⁇ 42nd bit data 37th bit data ⁇ 42nd bit data
  • the client device After the client device receives the difference information sent by the server, it can determine the identification information where the same data in the first data packet and the second data packet are located from the difference information. If it is determined that the same data is in the identification information and data in data a If the identification information in b is the same, there is no need to update data a in the client device according to the verification result; if the server determines that the same data is in data a, the identification information (such as the 4th digit) and the identification in data b If the information (such as the 12th bit of data) is different, the 12th bit of data in data a can be replaced with the 12th bit of data in data a.
  • the server does not need to check the data
  • the first to twenty-fourth data of a is updated, that is, the server does not need to perform any operation on the verification result.
  • the client device can use multiple methods to obtain different data, for example, it can be downloaded through the network, which is not specifically limited.
  • the server may obtain data in data b that is different from data a, and send it to the client device.
  • the server can determine according to the content of Table 4 that the forty-second to forty-eighth data of data a is different from the forty-second to forty-eighth data of data b.
  • the server Can get the 42nd to 48th data of data b (ie DECA90), and send the identification information of DECA90 and data a (42nd to 48th data) to the client The end device, so that the client device updates the forty-second to forty-eighth data (ie CEDF89) of data a to DECA90.
  • Fig. 3b is a schematic diagram of another corresponding embodiment of the data processing method provided by the embodiment of the present invention.
  • the client device stores the old resource data package c (referred to as data c), and the server stores the data c
  • the data d of the corresponding new resource data packet (referred to as data d for short).
  • data c may include 60-bit data, which is "ABCD12ACDE23ABDE34ADEF45BDEF56BCEF67BEDA78CEDF89CFAD90CDFB01"
  • data d may include 44-bit data, which is "38AFABCD12ACDE23ABDE34ADEF45BDEF5691EBCEDF89".
  • the execution process of using data d to update data c is as follows:
  • the client sends the first information to the server.
  • the client device may divide the data c to obtain data blocks Q 1 to Q 10 , where each data block in the data block Q 1 to data block Q 10 may include 6 bits of data.
  • the data block Q 1 includes the first to sixth data, namely data ABCD12;
  • the data block Q 2 includes the seventh to twelfth data, namely data ACDE23;
  • the data block Q 3 includes The thirteenth data to the eighteenth data, namely data ABDE34;
  • the data block Q 4 includes the nineteenth data to the twenty-fourth data, namely data ADEF45;
  • the data block Q 5 includes the twenty fifth data to
  • the 30th bit of data is data BDEF56;
  • the data block Q 6 includes the 31st to the 36th bit of data, that is, data BCEF67;
  • the data block Q 7 includes the 37th to the forty-second bit Data, namely data BEDA78;
  • data block Q 8 includes forty-second data to forty-eighth data, namely data CEDF89;
  • the client device can calculate the checksum value are weak and strong data block check value Q 1 to Q 10 of a data block, and Q may be based on the identification information 10 of the data blocks 1 to Q, the amount of data, weak The check value and the strong check value generate first information, and send the first information to the server.
  • the server divides the data block Q 1 to the data block Q 10 for the first time.
  • the server may divide the data block Q 1 to the data block Q 10 for the first time to obtain a data group E 11 , a data group E 12 and a data group E 13 .
  • the data group E 11 may include data blocks Q 1 to Q 4
  • the identification information of the data group E 11 is the first data to the twenty-fourth data
  • the data amount information is 24 bits of data.
  • the data is "ABCD12ACDE23ABDE34ADEF45"; the data group E 12 can include data block Q 5 to data block Q 8 , the identification information of data group E 12 is the twenty-fifth bit data to the forty-eighth bit data, and the data amount information is 24 Bit data, the 24-bit data is "BDEF56BCEF67BEDA78CEDF89”; data group E 13 may include data block Q 9 to data block Q 10 , and the identification information of data group E 13 is forty-ninth data to sixtieth data, The data amount information is 12-bit data, and the 12-bit data is "CFAD90CDFB01". And, the server may determine the value of the data set E 11 weak calibration check value according to the intensity data blocks Q 4 Q data block. 1 ⁇ , Q strong check value 5 to 8 in accordance with the data blocks determining the data set E 12 Q The weak check value of the data group E 13 can be determined according to the strong check value of the data block Q 9 to the data block Q 10 .
  • the server may start from the first data of the data d, and obtain the target data group with the same amount of data as the data group E 11 .
  • the target data group may include 24-bit data in data b, and the 24-bit data is "38AFABCD12ACDE23ABDE34AD".
  • the server can divide the target data group into four data blocks with the same amount of data as data block Q 1 to data block Q 4 , and can determine the weak check value and strong check value of each data block, and then according to the four The strong check value of the data block determines the weak check value of the target data group.
  • the server compares the weak check value of the data group E 11 with the weak check value of the target data group, and determines that the weak check value of the data group E 11 is different from the weak check value of the target data group. At this time, the server can determine that there is difference information between the data group E 11 and the data group E′ 11 .
  • the server may start with the second bit of data d and obtain a target data group with the same amount of data as the data group E 11 .
  • the target data group may include 24-bit data in data b, and the 24-bit data is "8AFABCD12ACDE23ABDE34ADE".
  • the server determines that the weak check value of the data group E 12 is different from the weak check value of the target data group, and determines that the strong check value of the data group E 12 is different from the strong check value of the target data group; at this time, the server can determine The data group E 11 is different from the target data group. Further, the server may start from the third data of the data d and obtain the target data group with the same data amount as the data group E 11 .
  • the target data group may include 24-bit data in data b, and the 24-bit data is "AFABCD12ACDE23ABDE34ADEF".
  • the server determines that the weak check value of the data group E 12 is different from the weak check value of the target data group. At this time, the server can determine that the data group E 11 is different from the target data group.
  • the server can sequentially execute from the first data of data d to the last data.
  • the server can determine the target data group obtained from the fifth data of data d ( Figure 3b is a schematic of the same data set E '11) and the data group E 11, this time, the server may check value Q 4 is determined intensity data set E according to the intensity check value 11 of the data blocks 1 to Q, strong and four check value set data of the target block is determined target data set E 'strong check value 11; the server data set and the check value E strong target data set of 11 E' strong check value 11 comparing, determining a data checksum value and the set E strong target data set E 11 'of the same intensity check value 11, and therefore, the server may generate a check result.
  • the data set E '11 may include data in 24-bit data d, 24-bit data which is "ABCD12ACDE23ABDE34ADEF45".
  • the server After the server generates the verification result, it can update the fifth to the twenty-ninth data in data d as holes to obtain the updated data d; and can start from the thirtieth data of data d and use Data group E 11 is checked.
  • the server can start with the first data (ie the thirtieth data) of the updated data d, using the data group E 12 and the data group E 13 respectively Check the data d.
  • the specific execution process please refer to the above content, which will not be repeated.
  • data d does not have the same data as data group E 12 or data group E 13 , therefore, the server did not generate a calibration in the process. The results of the test.
  • the verification result of the second stage may be as shown in Table 6.
  • the first data to the twenty-fourth data of the data c are the same as the fifth data to the twenty-ninth data of the data d.
  • the server divides the data block Q 1 to the data block Q 10 for the second time.
  • the server can divide the data block Q 1 to the data block Q 10 a second time to obtain a data group E 21 , a data group E 22 , a data group E 23 , a data group E 24 and a data group E 25 .
  • data group E 21 , data group E 22 , data group E 23 , data group E 24 and data group E 5 data d does not exist in data group E 21 and data group E 22 , Data group E 23 , Data group E 24 or Data group E 25 are the same data, therefore, the server did not generate verification information in the process.
  • the server divides the data block Q 1 to the data block Q 10 for the third time.
  • the server may divide the data block Q 1 to the data block Q 10 for the third time to obtain the data block Q 1 to the data block Q 10 .
  • the verification is performed on the data block Q 1 to the data block Q 10 and the updated data d to obtain a verification result.
  • the verification result of the fourth stage may be as shown in Table 7.
  • the 37th to 42nd data of data a are the same as the 37th to 42nd data of data b, and the 42nd to fourth data of data a
  • the eighteen-bit data is different from the forty-second to forty-eighth data of data b.
  • the server sends the difference information to the client device.
  • the server obtains the verification results of the first to fourth stages (that is, Table 6 and Table 7), for the data with the same verification result, take the verification result shown in Table 6 as an example, the first data c
  • One bit to the twenty-fourth bit data is the same as the fifth bit to the twenty-ninth bit data of data d.
  • the server can assign the same data with the same check result to the identification information in data c (that is, the first bit) Data ⁇ 24th bit data) and the identification information in data d (that is, the fifth bit to the 29th bit data) are sent to the client device, so that the client device is based on the first bit of data in data c ⁇
  • the fifth bit of data in the twenty-fourth data update data c ⁇ the twenty-ninth bit of data, that is, the fifth bit to the twenty-ninth bit of data in data c, 12ACDE23ABDE34ADEF45BDEF is updated to the first in the original data c Bit data ⁇ 24th bit data ABCD12ACDE23ABDE34ADEF45.
  • the server may obtain data in data d that is different from data c, and send it to the client device. As shown in Table 7, the server can determine that the first to fourth data and the thirty-fifth to thirty-eighth data of data d are different from data c. Therefore, the server can obtain the first data of data d.
  • Bit data ⁇ the fourth bit data (ie 38AF) and the thirty-fifth bit data ⁇ the thirty-eighth bit data (ie 91EB), and the identification information of 38AF and data d (the first bit data ⁇ the fourth bit data) , 91EB and the identification information of data d (the thirty-fifth digit to the thirty-eighth digit) are sent to the client device, so that the client device sends the first digit to the fourth digit of data c (ie ABCD ) Is updated to 38AF, and the thirty-fifth to thirty-eighth data (ie 67BE) of data c is updated to 91EB.
  • the server divides the N first data blocks for the first time to obtain M groups of data, and according to at least one first data included in each group of data
  • the check value corresponding to the block and the second data packet corresponding to the first data packet are generated to generate check information corresponding to each group of data, where the first information includes the data amount of the N first data blocks corresponding to the first data packet
  • the check value corresponding to the N first data blocks respectively; further, the server determines the difference information corresponding to the first data packet according to the check information corresponding to the M groups of data and the second data packet, and sends the difference information to the client Terminal device, so that the client device updates the first data packet according to the difference information.
  • the server can transmit the difference information to the client device without sending the entire first data packet to the client device, thereby avoiding Repeated data information is transmitted, the loss of network traffic is reduced, and the efficiency of data synchronization is improved; moreover, the method of determining the check value of the data group by using the check value of the data block in the embodiment of the present invention can make the client device and The synchronization process can be completed by performing only one communication interaction between the servers without performing multiple communication interactions, thereby reducing the pressure of network communication and reducing network transmission traffic.
  • an embodiment of the present invention also provides a server, and the specific content of the server can be implemented with reference to the foregoing method.
  • Fig. 4 is a schematic structural diagram of a server provided by an embodiment of the present invention, including:
  • the obtaining module 401 is configured to obtain first information sent by the client device, where the first information includes the data amount of the N first data blocks corresponding to the first data packet and the calibrations corresponding to the N first data blocks. Test value
  • the processing module 402 is configured to divide the N first data blocks for the first time to obtain M groups of data, and according to the sum of the check value corresponding to at least one first data block included in each group of data and the first data block A second data packet corresponding to the data packet, generating verification information corresponding to each group of data;
  • the transceiver module 403 is configured to determine the difference information corresponding to the first data packet according to the verification information corresponding to the M groups of data and the second data packet, and send the difference information to the client device , So that the client device updates the first data packet according to the difference information.
  • the processing module 402 generates a check value corresponding to each group of data according to a check value corresponding to at least one first data block included in each group of data and a second data packet corresponding to the first data packet. Before verifying information, it is also used to:
  • the maximum amount of data in the M sets of data is greater than or equal to the minimum amount of data in the data amounts of the N first data blocks.
  • the check values respectively corresponding to the N first data blocks included in the first information are first check values corresponding to the N first data blocks respectively;
  • the processing module 402 is used to:
  • the M groups of data include a Yth data group, and the Yth data group includes T first data blocks.
  • the data amount of the T second data blocks is the same as the data amount of the T first data blocks; according to the first check value of the T second data blocks Determine the second check value corresponding to the first target data group; generate the second check value corresponding to the Yth data group and the second check value corresponding to the first target data group Y data group verification information;
  • the second data packet corresponding to the Y-1th group of data is updated to obtain the second data packet corresponding to the Yth data group.
  • processing module 402 is used to:
  • the second check value corresponding to the Yth data group is the same as the second check value corresponding to the first target data group, corresponding to the T first data blocks included in the Yth data group respectively.
  • the first check value corresponding to the Yth data group is determined, and the first check value corresponding to the first target data group is determined according to the first check value of the T second data blocks Check value; if it is determined that the first check value corresponding to the Yth data group is different from the first check value corresponding to the first target data group, then the first check information of the Yth data group is generated ;
  • processing module 402 is further configured to:
  • the first check value corresponding to the Y-th data group is the same as the first check value corresponding to the first target data group, if it is determined that the data amount of the Y-th data group is equal to the N first check values. If it is determined that the data amount of the Yth data group is greater than the minimum data amount in the N first data blocks, then the second check information of the Yth data group is generated; Starting from the i+1-th bit data in the second data packet corresponding to the Y-1th data group, a third target data group with the same amount of data as the Yth data group is acquired.
  • processing module 402 is further configured to:
  • the N first data blocks are divided for the second time.
  • processing module 402 is used to:
  • the server divides the N first data blocks for the first time to obtain M groups of data, and according to each group of data At least one check value corresponding to the first data block and a second data packet corresponding to the first data packet are included to generate check information corresponding to each set of data, where the first information includes N pieces corresponding to the first data packet The data volume of the first data block and the check value corresponding to the N first data blocks respectively; further, the server determines the difference information corresponding to the first data packet according to the check information corresponding to the M groups of data and the second data packet, And the difference information is sent to the client device, so that the client device updates the first data packet according to the difference information.
  • the server can transmit the difference information to the client device without sending the entire first data packet to the client device, thereby avoiding Repeated data information is transmitted, the loss of network traffic is reduced, and the efficiency of data synchronization is improved; moreover, the method of determining the check value of the data group by using the check value of the data block in the embodiment of the present invention can make the client device and The synchronization process can be completed by performing only one communication interaction between the servers without performing multiple communication interactions, thereby reducing the pressure of network communication and reducing network transmission traffic.
  • embodiments of the present invention provide a data processing device. At least one processor; and, a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable The at least one processor can execute the data processing method in the foregoing embodiment.
  • FIG. 5 is a structure of a data processing device provided by an embodiment of the present invention.
  • the data processing device 500 includes: a transceiver 501, a processor 502, a memory 503, and a bus system 504;
  • the memory 503 is used to store programs.
  • the program may include program code, and the program code includes computer operation instructions.
  • the memory 503 may be a random access memory (random access memory, RAM for short), or a non-volatile memory (non-volatile memory), such as at least one disk memory. Only one memory is shown in the figure. Of course, the memory can also be set to multiple as required.
  • the memory 503 may also be a memory in the processor 502.
  • the memory 503 stores the following elements, executable modules or data structures, or their subsets, or their extended sets:
  • Operating instructions including various operating instructions, used to implement various operations.
  • Operating system including various system programs, used to implement various basic services and process hardware-based tasks.
  • the foregoing data processing method in the embodiment of the present invention may be applied to the processor 502, or implemented by the processor 502.
  • the processor 502 may be an integrated circuit chip with signal processing capabilities.
  • the steps of the above-mentioned data processing method can be completed by hardware integrated logic circuits in the processor 502 or instructions in the form of software.
  • the aforementioned processor 502 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware Components.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present invention may be directly embodied as being executed and completed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 503, and the processor 502 reads the information in the memory 503, and performs the following steps in combination with its hardware:
  • the transceiver 501 is configured to obtain first information sent by a client device, where the first information includes the data amount of the N first data blocks corresponding to the first data packet and the N first data blocks respectively correspond to The check value;
  • the processor 502 is configured to divide the N first data blocks for the first time to obtain M groups of data, and according to the sum of the check value corresponding to at least one first data block included in each group of data and the A second data packet corresponding to the first data packet, generating verification information corresponding to each group of data;
  • the transceiver 501 is further configured to determine the difference information corresponding to the first data packet according to the check information corresponding to the M groups of data and the second data packet, and send the difference information to the The client device, so that the client device updates the first data packet according to the difference information.
  • the processor 502 generates a check value corresponding to each group of data according to a check value corresponding to at least one first data block included in each group of data and a second data packet corresponding to the first data packet. Before verifying information, it is also used to:
  • the maximum amount of data in the M sets of data is greater than or equal to the minimum amount of data in the data amounts of the N first data blocks.
  • the check values respectively corresponding to the N first data blocks included in the first information are first check values corresponding to the N first data blocks respectively;
  • the processor 502 is configured to:
  • the M groups of data include a Yth data group, and the Yth data group includes T first data blocks.
  • the data amount of the T second data blocks is the same as the data amount of the T first data blocks; according to the first check value of the T second data blocks Determine the second check value corresponding to the first target data group; generate the second check value corresponding to the Yth data group and the second check value corresponding to the first target data group Y data group verification information;
  • the second data packet corresponding to the Y-1th group of data is updated to obtain the second data packet corresponding to the Yth data group.
  • the processor 502 is configured to:
  • the second check value corresponding to the Yth data group is the same as the second check value corresponding to the first target data group, corresponding to the T first data blocks included in the Yth data group respectively.
  • the first check value corresponding to the Yth data group is determined, and the first check value corresponding to the first target data group is determined according to the first check value of the T second data blocks Check value; if it is determined that the first check value corresponding to the Yth data group is different from the first check value corresponding to the first target data group, then the first check information of the Yth data group is generated ;
  • processor 502 is further configured to:
  • the first check value corresponding to the Y-th data group is the same as the first check value corresponding to the first target data group, if it is determined that the data amount of the Y-th data group is equal to the N first check values. If it is determined that the data amount of the Yth data group is greater than the minimum data amount in the N first data blocks, then the second check information of the Yth data group is generated; Starting from the i+1-th bit data in the second data packet corresponding to the Y-1th data group, a third target data group with the same amount of data as the Yth data group is acquired.
  • processor 502 is further configured to:
  • the N first data blocks are divided for the second time.
  • the processor 502 is configured to:
  • the data processing device of the embodiment of the present invention exists in various forms, including but not limited to:
  • Ultra-mobile personal computer equipment This type of equipment belongs to the category of personal computers, has calculation and processing functions, and generally also has mobile Internet features.
  • Such terminals include: PDA, MID and UMPC devices, such as iPad.
  • the program is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer). , A chip, etc.) or a processor (processor) executes all or part of the steps of the method in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code .
  • embodiments of the present invention also provide a computer-readable storage medium, including instructions, which when run on a computer, cause the computer to execute the data processing method described in FIG. 1 or FIG. 2 above.
  • the embodiments of the present invention also provide a computer program product, which when running on a computer, causes the computer to execute the data processing method described in FIG. 1 or FIG. 2 above.
  • the embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
  • a computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明实施例公开了一种数据处理方法及服务器、设备,其中方法包括:服务器获取客户端设备发送的第一信息后,对N个第一数据块进行划分,得到M组数据,并确定每组数据对应的校验信息;进一步地,服务器根据M组数据对应的校验信息和第二数据包,确定差异信息,并将差异信息发送给客户端设备,以使客户端设备根据差异信息更新第一数据包。本发明实施例中,服务器可以将差异信息传输给客户端设备,而无需将整个第一数据包发送给客户端设备,从而可以避免传输重复的数据信息,降低网络流量的损失,提高数据同步的效率;且,本发明实施中客户端设备与服务器之间可以仅执行一次通信交互过程,从而可以减轻网络通信的压力,降低网络传输流量。

Description

一种数据处理方法及服务器、设备
相关申请的交叉引用
本申请要求在2019年04月04日提交中国专利局、申请号为201910272205.8、申请名称为“一种数据处理方法及服务器”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本发明涉及计算机技术领域,尤其涉及一种数据处理方法及服务器、设备。
背景技术
数据同步是一种常用的数据处理方法,通过对不同设备(比如客户端设备与服务器、客户端设备与存储设备或服务器与存储设备)上的数据进行同步,可以保持不同设备上数据的完整性与统一性。以客户端设备与服务器之间同步资源数据包数据为例,在一种常用的数据同步方式中,若服务器检测到服务器上安装的资源数据包从资源数据包1升级为资源数据包2,则可以将资源数据包2发送给客户端设备,以使客户端设备将客户端设备上安装的资源数据包从资源数据包1更新为资源数据包2,从而保持服务器与客户端设备上的资源数据包的一致性。
然而,在实际操作中,资源数据包2与资源数据包1之间可能只存在部分数据不同,而大部分的数据是相同的;比如,资源数据包1和资源数据包2的总数据量均为50M,资源数据包2中仅存在5M的数据与资源数据包1不同,而资源数据包2中其它45M的数据与资源数据包1相同。若采用上述方式,则客户端设备会向服务器发送资源数据包2所包括的50M的数据,其中,这50M的数据中可以包括与资源数据包1相同的45M的数据和与资源数据包1不同的5M数据。由此可知,采用上述方式服务器可能会传输重复的数据, 从而损失较多的网络流量,导致数据同步的效率较低。
综上,目前亟需一种数据处理方法,用以提高数据同步的效率。
发明内容
本发明实施例提供一种数据处理方法及服务器、设备,用以提高数据同步的效率。
第一方面,本发明实施例提供的一种数据处理方法,所述方法包括:
服务器获取客户端设备发送的第一信息,所述第一信息包括第一数据包对应的N个第一数据块的校验值,所述服务器对所述N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息;进一步地,所述服务器根据所述M组数据对应的校验信息和所述第二数据包,确定所述第一数据包对应的差异信息,并将所述差异信息发送给所述客户端设备,以使所述客户端设备根据所述差异信息更新所述第一数据包。
上述技术方案中,通过使用第一数据包对第二数据包进行校验,可以使得服务器将差异信息传输给客户端设备,而无需将整个第一数据包发送给客户端设备,从而可以避免传输重复的数据信息,降低网络流量的损失,提高数据同步的效率;且,本发明实施例中通过采用数据块的校验值来确定数据组的校验值的方式,可以使得客户端设备与服务器之间仅执行一次通信交互即可完成同步过程,而无需执行多次通信交互,从而可以减轻网络通信的压力,降低网络传输流量。也就是说,本发明实施例中的数据处理方法可以适应于低带宽和流量敏感的网络环境。
可选地,所述根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息之前,还包括:所述服务器确定所述M组数据中的最大数据量大于或等于所述N个第一数据块的数据量中的最小数据量。
可选地,所述第一信息中还包括所述N个第一数据块的数据量,所述第一 信息中包括的所述N个第一数据块分别对应的校验值为所述N个第一数据块分别对应的第一校验值;所述根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息,包括:所述M组数据中包括第Y数据组,所述第Y数据组中包括T个第一数据块,针对于所述M组数据中的第Y数据组,执行:所述服务器根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第二校验值;所述服务器从第Y-1数据组对应的第二数据包中的第i位数据开始,获取与所述第Y数据组的数据量相同的第一目标数据组,并对所述第一目标数据组进行划分,得到T个第二数据块,所述T个第二数据块的数据量与所述T个第一数据块的数据量相同;根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第二校验值;根据所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息;所述服务器对所述第Y-1组数据对应的第二数据包进行更新,得到第Y数据组对应的第二数据包。
上述技术方案中,第一校验值可以为强校验值,第二校验值可以为弱校验值,通过采用先对比弱校验值再对比强校验值的方式,一方面可以通过弱校验值过滤掉目的数据组中与第Y数据组明显相同的数据信息,从而减轻工作量、提高数据处理的效率;另一方面,通过采用强校验值和弱校验值共同确定第Y数据组对应的校验信息,可以提高第Y数据组对应的校验信息的准确性。
可选地,所述根据所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息,包括:所述服务器若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值相同,则根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第一校验值,并根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第一校验值;若确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值不同,则生成所述第Y数据组的第一校验信息;所述服务器若确定所述第Y数据组对应的第二 校验值与所述第一目标数据组对应的第二校验值不同,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第二目标数据组。
上述技术方案中,通过先对比弱校验值再对比强校验值的方式,可以简化数据处理的过程,提高数据处理的效率。
可选地,所述方法还包括:所述服务器确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值相同时,若确定所述第Y数据组的数据量等于所述N个第一数据块中的最小数据量,则生成所述第Y数据组的第二校验信息;若确定所述第Y数据组的数据量大于所述N个第一数据块中的最小数据量,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第二目标数据组。
上述技术方案中,校验信息可以包括第一校验信息与第二校验信息,第一校验信息为目标数据组中与第Y数据组相同的数据信息对应的校验信息,第二校验信息可以为目标数据组中与第Y数据组不同的数据信息对应的校验信息。通过获取第一校验信息和第二校验信息,服务器可以获取更为完整的校验信息,从而使得服务器向客户端设备同步的差异信息更为准确。
可选地,所述方法还包括:所述服务器若确定所述第M组数据对应的第二数据包的数据量大于或等于预设数据量阈值,则对所述N个第一数据块进行第二次划分。
上述技术方案中,通过采用多轮校验的方式获取校验信息,可以提高服务器获取到的校验信息的准确性,从而使得客户端设备与服务器之间的数据同步过程更为精确;进一步地,第二轮校验对应每组数据的数据量可以小于第一轮校验对应的每组数据的数据量,也就是说,上述技术方案通过多轮校验的方式,可以避免仅采用一轮校验所导致的数据信息重复传输的技术问题,从而可以节省网络流量,提高数据同步的效率。
可选地,所述服务器对所述N个第一数据块进行第二次划分得到P组数据;所述服务器若确定所述P组数据中的最大数据量小于所述N个第一数据块的数 据量中的最小数据量,则将第一至第M组数据的校验信息发送给所述客户端设备。
第二方面,本发明实施例提供的一种服务器,所述服务器包括:
获取模块,用于获取客户端设备发送的第一信息,所述第一信息包括第一数据包对应的N个第一数据块的校验值;
处理模块,用于对所述N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息;
收发模块,用于根据所述M组数据对应的校验信息和所述第二数据包,确定所述第一数据包对应的差异信息,并将所述差异信息发送给所述客户端设备,以使所述客户端设备根据所述差异信息更新所述第一数据包。
可选地,所述处理模块根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息之前,还用于:确定所述M组数据中的最大数据量大于或等于所述N个第一数据块的数据量中的最小数据量。
可选地,所述第一信息中还包括所述N个第一数据块的数据量,所述第一信息中包括的所述N个第一数据块分别对应的校验值为所述N个第一数据块分别对应的第一校验值;所述处理模块用于:所述M组数据中包括第Y数据组,所述第Y数据组中包括T个第一数据块,针对于所述M组数据中的第Y数据组,执行:根据所述第Y数据组包括的T个第一数据块的第一校验值,确定所述第Y数据组对应的第二校验值;从第Y-1数据组对应的第二数据包中的第i位数据开始,获取与所述第Y数据组的数据量相同的第一目标数据组,并对所述第一目标数据组进行划分,得到T个第二数据块,所述T个第二数据块的数据量与所述T个第一数据块的数据量相同;根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第二校验值;根据所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息;对所述第Y-1组数据对应的第二数据包进行更新,得到第Y数据组对应的 第二数据包;
可选地,所述处理模块用于:若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值相同,则根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第一校验值,并根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第一校验值;若确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值不同,则生成所述第Y数据组的第一校验信息;若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值不同,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第二目标数据组。
可选地,所述处理模块还用于:确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值相同时,若确定所述第Y数据组的数据量等于所述N个第一数据块中的最小数据量,则生成所述第Y数据组的第二校验信息;若确定所述第Y数据组的数据量大于所述N个第一数据块中的最小数据量,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第三目标数据组。
可选地,所述处理模块还用于:若确定所述第M组数据对应的第二数据包的数据量大于或等于预设数据量阈值,则对所述N个第一数据块进行第二次划分。
可选地,所述处理模块用于:对所述N个第一数据块进行第二次划分得到P组数据;若确定所述P组数据中的最大数据量小于所述N个第一数据块的数据量中的最小数据量,则将第一至第M组数据的校验信息发送给所述客户端设备。
第三方面,本发明实施例提供的一种计算机可读存储介质,包括指令,当其在计算机上运行时,使得计算机执行如上述第一方面或第一方面任意所述的数据处理方法。
第四方面,本发明实施例提供的一种计算机程序产品,当其在计算机上运行时,使得计算机执行如上述第一方面或第一方面任意所述的数据处理方 法。
第五方面,本发明实施例提供的一种数据处理设备,包括:
至少一个处理器;以及,
与所述至少一个处理器通信连接的存储器;其中,
所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器能够执行如上述第一方面或第一方面任意所述的数据处理方法。
本发明的这些实现方式或其他实现方式在以下实施例的描述中会更加简明易懂。
附图说明
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简要介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域的普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的一种数据处理方法对应的流程示意图;
图2为本发明实施例提供的一种客户端设备与服务器之间交互的示意图;
图3a为本发明实施例中提供的一种实施例示意图;
图3b为本发明实施例中提供的一种实施例示意图;
图4为本发明实施例提供的一种服务器的结构示意图;
图5为本发明实施例提供的一种数据处理设备对应的结构示意图。
具体实施方式
为了使本发明的目的、技术方案和优点更加清楚,下面将结合附图对本发明作进一步地详细描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例,都属于本发明保护的 范围。
在客户端设备与服务器进行数据同步的过程中,为了避免传输重复的数据信息,一种可能的实现方式为:将服务器中安装的资源数据包2与客户端设备中安装的资源数据包1进行对比,确定资源数据包2与资源数据包1之间的校验信息,校验信息可以用于标识文件a2与文件a1之间的差异信息,从而使得服务器可以将差异信息发送给客户端设备,以使客户端设备对资源数据包1进行更新。在该实现方式中,确定校验信息的执行主体可以是客户端设备,或者也可以是服务器,具体不作限定。
以确定校验信息的执行主体为服务器为例,根据上述内容可知,若服务器确定的校验信息较为准确,则服务器可以根据准确的校验信息得到准确的差异信息,从而使得服务器可以将准确的差异信息发送给客户端设备,提高数据同步的效率。也就是说,校验信息的准确性会影响差异信息的准确性,若想保证差异信息的准确性,则需要服务器获取到准确的校验信息。然而,现阶段还没有一种准确获取校验信息的方式,因此,现阶段在客户端设备与服务器之间进行数据同步时,存在传输重复的数据信息、网络流量的损失较大以及数据同步的效率较低的技术问题。
综上,目前亟需一种数据处理方法,用以提高数据同步的效率。
图1为本发明实施例提供的一种数据处理方法对应的流程示意图,该方法包括:
步骤101,服务器获取客户端设备发送的第一信息。
本发明实施例中,客户端设备中可以存储有第一数据包,服务器中可以存储有第二数据包。其中,服务器中的第二数据包可以为对服务器中存储的第三数据包进行更新得到的数据包,第三数据包可以与第一数据包相同,或者也可以与第一数据包不同,具体不作限定。举例说明,服务器和客户端设备上均可以安装有同一资源数据包,客户端设备上安装的资源数据包为版本V1.1对应的资源数据包(即第一数据包),而服务器上安装的资源数据包为版本V1.3对应的资源数据包(即第二数据包);在一个示例中,若服务器上安 装的版本V1.3对应的资源数据包为对服务器上原安装的版本V1.1对应的资源数据包(即第三数据包)进行升级得到的,即版本V1.3为对版本V1.1进行升级得到的,则在该示例中,第一数据包和第三数据包可以相同(均为版本V1.1对应的资源数据包);在另一个示例中,若服务器上安装的版本V1.3对应的资源数据包为对服务器上原安装的版本V1.2对应的资源数据包(即第三数据包)进行升级得到的,即版本V1.3为对版本V1.2进行升级得到的,而版本V1.2是对版本V1.1进行升级得到的,则在该示例中,第一数据包和第三数据包可以不同,第一数据包可以为版本V1.1对应的资源数据包,第三数据包可以为版本V1.2对应的资源数据包。
本发明的下列实施例以使用服务器中的第二数据包对客户端设备中的第一数据包进行数据同步的过程为例说明本发明实施例中的方案,可以理解的,其它设备之间进行数据同步的过程也可以参照本发明实施例中的数据同步过程进行实现,具体不作限定。
本发明实施例中,第一信息可以包括第一数据包对应的N个第一数据块的数据量和N个第一数据块分别对应的校验值。具体实施中,客户端设备可以根据第一数据包的数据量将第一数据包划分为N个第一数据块。其中,对第一数据包进行划分的方式可以有多种,在一种可能的实现方式中,客户端设备可以按照预设数据量将第一数据包划分为N个第一数据块,每个第一数据块的数据量可以相同。举例来说,第一数据包的数据量为905M,对第一数据包进行划分可以得到第一数据块b1~第一数据块b10;其中,第一数据块b1~第九第一数据块b9中每个第一数据块的数据量可以为100M,第一数据块b10的数据量可以为5M。
进一步地,针对于第一数据块b1~第一数据块b10中的每个第一数据块,可以确定每个第一数据块对应的校验值。每个第一数据块对应的校验值可以为每个第一数据块的第一校验值(强校验值),或者也可以为每个数据块的第二校验值(强校验值)。相应地,第一信息中包括的第一数据块b1~第一数据块b10的校验值可以为第一数据块b1~第一数据块b10的第一校验值,或者也可 以为第一数据块b1~第一数据块b10的第二校验值,或者还可以第一数据块b1~第一数据块b10的第一校验值和第一数据块b1~第一数据块b10的第二校验值,具体不作限定。
第一校验值可以为一种强校验值,比如MD5校验值。MD5校验值可以为通过带加盐的MD5哈希方式或者不带加盐的MD5哈希方式得到的密钥,其获取过程为不可逆的加密过程,即通过某一数据对应的MD5校验值无法获取该数据的信息。相应地,若采用相同的加密方式(比如带加盐的MD5哈希方式)对文件第一数据和第二数据分别进行加密后,第一数据对应的密钥与第二数据对应的密钥相同,则可以说明第一数据与第二数据相同。以确定第一数据块b1的第一校验值为例,具体实施中,可以预先将第一数据块b1划分为512个第一分组,并将每个第一分组划分为32个第一子分组。进一步地,通过使用MD5哈希的方式对512个第一分组进行加密,可以得到四个32位的第二子分组,将这四个32位的第二子分组进行级联后,可以得到一个128位的散列值。其中,该散列值即可以为第一数据块b1对应的第一校验值。第一校验值的可靠性较强,若采用N个第一数据块分别对应的第一校验值来获取校验信息,可以提交校验信息的准确性。
第二校验值可以为一种弱校验值,比如基于校验和算法(即Asler32算法)确定的校验值。具体地说,第一数据块的弱校验值可以为对第一数据组中包括的各个字段进行加和得到的,可以将得到的和值作为第一校验和值;相应地,可以将在确定第一校验和值的过程中每一步的阶段值之和作为第二校验和值。其中,第一校验和值与第二校验和值均可以为16位的数值,进一步地,可以将16位的第一校验和值与16位的第二校验和值拼接为一个32位的整数,即第一数据块的第二校验值。
步骤102,服务器对N个第一数据块进行划分,得到M组数据。
具体实施中,对N个第一数据块进行划分的方式可以有多种,在一种可能的实现方式中,服务器可以按照预设的数据块数量将N个第一数据块划分为M组数据,其中,每组数据包括的数据块的数量可以相同。举例来说,服务器 可以将第一数据块b1~第一数据块b10划分为第一数据组~第四数据组,其中,第一数据组~第三数据组中的每个数据组包括的数据块的数量可以为3个,第四数据组包括的数据块的数量可以为1个。比如,第一数据组包括第一数据块b1~第一数据块b3,第二数据组包括第一数据块b4~第一数据块b6,第三数据组包括第一数据块b7~第一数据块b9,第四数据组包括第一数据块b10。也就是说,第一数据组~第三数据组中每个数据组的数据量可以为300M,第四数据组的数据量可以为5M。
步骤103,服务器确定M组数据中每组数据的校验信息。
在一种可能的实现方式中,服务器若确定M组数据中的最大数据量大于或等于N个第一数据块的数据量中的最小数据量,则说明服务器对第二数据包进行校验的过程未结束,可以继续执行后续的步骤。此处,N个第一数据块的数据量中的最小数据量可以是指N个第一数据块中数据量最小的第一数据块的数据量。比如,第一数据块b1~第一数据块b10的数据量中的最小数据量可以是指第一数据块b10的数据量,即5M。
需要说明的是,上述实现方式仅是一种示例性的简单说明,其所列举的确定服务器对第二数据包进行校验的过程未结束的条件仅是为了便于说明方案,并不构成对方案的限定。在具体实施中,确定服务器对第二数据包进行校验的过程未结束的条件可以由本领域技术人员根据实际情况进行设置,比如可以设置预设数据量阈值,若确定M组数据中的最大数据量大于或等于预设数据量阈值,则可以确定服务器对第二数据包进行校验的过程未结束;其中,预设数据量阈值可以为固定的数值,或者也可以为根据第一数据块中的最小数据量确定得到的数值,比如可以为第一数据块中的最小数据量的3/4,具体不作限定。
以第二数据组为例,第二数据组中可以包括第一数据块b4~第一数据块b6,具体实施中,可以根据第二数据组包括的第一数据块b4~第一数据块b6分别对应的校验值,确定第二数据组对应的校验值。下面分别描述确定第二数据组对应的校验值的几种可能的情况。
第一种情形
在第一种情形中,第一信息中包括的第一数据块的校验值可以为第一数据块的第一校验值,即步骤101中发送的是第一数据块b4~第一数据块b6分别对应的第一校验值。在第一种情形中,可以根据第一数据块b4~第一数据块b6分别对应的第一校验值确定第二数据组对应的第一校验值,或者也可以根据第一数据块b4~第一数据块b6分别对应的第一校验值确定第二数据组对应的第二校验值,或者还可以根据第一数据块b4~第一数据块b6分别对应的第一校验值确定第二数据组对应的第一校验值和第二校验值,具体不作限定。下面分别从这三种子情形进行描述。
第一子情形
若根据第一数据块b4~第一数据块b6分别对应的第一校验值确定第二数据组对应的第一校验值,则可以将第一数据块b4~第一数据块b6分别对应的第一校验值划分为3个分组,并将每个分组划分为32个子分组。进一步地,通过使用MD5哈希的方式对3个分组进行加密,可以得到多个32位的子分组,将多个32位的子分组进行级联后,可以得到一个128位的散列值。该散列值即可以为第二数据组对应的第一校验值。
第二子情形
若根据第一数据块b4~第一数据块b6分别对应的第一校验值确定第二数据组对应的第二校验值,则可以对第一数据块b4~第一数据块b6的第一校验值加和,并可以将得到的和值作为第一校验和值;相应地,可以将在确定第一校验和值的过程中每一步的阶段值之和作为第二校验和值。进一步地,可以将第一校验和值与第二校验和值拼接为一个整数,该整数既可以为第二数据组对应的第二校验值。
举例来说,若第一数据块b4~第一数据块b6对应的第一校验值分别为R1、R2和R3,则可以对R1、R2和R3中的每个字节进行加和,将得到的和值作为第一校验和值;相应地,可以将在确定第一校验和值的过程中每一步的阶段值之和作为第二校验和值。其中,第一校验和值与第二校验和值均可以为16位 的数值,进一步地,可以将16位的第一校验和值与16位的第二校验和值拼接为一个32位的整数,即第二数据组对应的第二校验值。
第三子情形
若根据第一数据块b4~第一数据块b6分别对应的第一校验值确定第二数据组对应的第一校验值和第二校验值,则可以按照上述方式分别确定第二数据组对应的第一校验值和第二校验值,此处不再赘述。
第二种情形
在第二种情形中,第一信息中包括的第一数据块的校验值可以为第一数据块的第二校验值,即步骤101中发送的是第一数据块b4~第一数据块b6分别对应的第二校验值。在第二种情形中,可以根据第一数据块b4~第一数据块b6分别对应的第二校验值确定第二数据组对应的第一校验值,或者也可以根据第一数据块b4~第一数据块b6分别对应的第二校验值确定第二数据组对应的第二校验值,或者还可以根据第一数据块b4~第一数据块b6分别对应的第二校验值确定第二数据组对应的第一校验值和第二校验值,具体不作限定。下面分别从这三种子情形进行描述。
第四子情形
若根据第一数据块b4~第一数据块b6分别对应的第二校验值确定第二数据组对应的第一校验值,则可以将第一数据块b4~第一数据块b6分别对应的第二校验值划分为3个分组,并将每个分组划分为32个子分组。进一步地,通过使用MD5哈希的方式对3个分组进行加密,可以得到多个32位的子分组,将多个32位的子分组进行级联后,可以得到一个128位的散列值。该散列值即可以为第二数据组对应的第二校验值。
第五子情形
若根据第一数据块b4~第一数据块b6分别对应的第二校验值确定第二数据组对应的第二校验值,则可以对第一数据块b4~第一数据块b6的第二校验值加和,并可以将得到的和值作为第一校验和值;相应地,可以将在确定第一校验和值的过程中每一步的阶段值之和作为第二校验和值。进一步地,可以 将第一校验和值与第二校验和值拼接为一个整数,该整数既可以为第二数据组对应的第二校验值。
第六子情形
若根据第一数据块b4~第一数据块b6分别对应的第二校验值确定第二数据组对应的第一校验值和第二校验值,则可以按照上述方式分别确定第二数据组对应的第一校验值和第二校验值,此处不再赘述。
进一步地,服务器可以根据第二数据组的校验值对第一数据组对应的第二数据包进行校验。其中,第一数据组对应的第二数据包可以为针对于第一数据组执行步骤103后得到的数据包。具体地说,在根据第一数据组的校验值对第二数据包进行校验后,可以对第二数据包进行更新,得到第一数据组对应的第二数据包;相应地,在根据第二数据组的校验值对第一数据组对应的第二数据包进行校验后,可以对第一数据组对应的第二数据包进行更新,得到第二数据组对应的第二数据包。需要说明的是,得到第三数据组对应的第二数据包和第四数据组对应的第二数据包的过程可以参照第一数据组和第二数据组执行,不再赘述。
具体实施中,服务器可以在获取服务器中存储的第一数据组对应的第二数据包后,从第一数据组对应的第二数据包中的第i位数据开始,获取与第一数据组的数据量相同的目标数据组,并可以对目标数据组进行划分,得到T个第二数据块;其中,T个第二数据块的数据量与T个第一数据块的数据量可以相同。进一步地,服务器可以根据T个第二数据块的校验值确定目标数据组对应的校验值,比如,在第一子情形中,服务器可以根据T个第二数据块的第一校验值确定目标数据组对应的第一校验值,并可以根据第二数据组的第一校验值和目标数据组对应的第一校验值,对第二数据包进行校验,生成第二数据组对应的校验信息;在第二子情形中,服务器可以根据T个第二数据块的第一校验值确定目标数据组对应的第二校验值,并可以根据第二数据组的第二校验值和目标数据组对应的第二校验值,对第二数据包进行校验,生成第二数据组对应的校验信息;在第三子情形中,服务器可以根据T个第二数据块的 第一校验值确定目标数据组对应的第一校验值和第二校验值,并可以根据第二数据组的第一校验值和目标数据组对应的第一校验值、第二数据组的第二校验值和目标数据组对应的第二校验值,对第二数据包进行校验,生成第二数据组对应的校验信息。相应地,在第四子情形中,服务器可以根据T个第二数据块的第二校验值确定目标数据组对应的第一校验值,并可以根据第二数据组的第一校验值和目标数据组对应的第一校验值,对第二数据包进行校验,生成第二数据组对应的校验信息;在第五子情形中,服务器可以根据T个第二数据块的第二校验值确定目标数据组对应的第二校验值,并可以根据第二数据组的第二校验值和目标数据组对应的第二校验值,对第二数据包进行校验,生成第二数据组对应的校验信息;在第六子情形中,服务器可以根据T个第二数据块的第二校验值确定目标数据组对应的第一校验值和第二校验值,并可以根据第二数据组的第一校验值和目标数据组对应的第一校验值、第二数据组的第二校验值和目标数据组对应的第二校验值,对第二数据包进行校验,生成第二数据组对应的校验信息。
本发明的下列实施例中以第二子情形为例具体描述对第二数据包进行校验以及生成第二数据组对应的校验信息的实现过程。
具体实施中,以第一次执行的过程为例(从第一数据组对应的第二数据包中的第1位数据开始),若第二数据组中包括第一数据块b4~第一数据块b6,且第一数据块b4~第一数据块b6中的每个第一数据块的数据量均为100M,则在第一次执行过程中,服务器可以从第一数据组对应的第二数据包中的第一位数据开始,依次获取300M的数据作为目标数据组,并将目标数据组划分为第二数据块c4~第二数据块c6;其中,第二数据块c4~第二数据块c6中的每个第二数据块的数据量可以均为100M。进一步地,服务器可以确定第二数据块c4~第二数据块c6的第一校验值,并可以根据第二数据块c4~第二数据块c6的第一校验值确定目标数据组的第二校验值(可以参照确定第二数据组的第二校验值的过程进行执行,不再赘述)。
进一步地,服务器可以将第二数据组对应的第二校验值与目标数据组对 应的第二校验值进行对比,若确定第二数据组对应的第二校验值与目标数据组对应的第二校验值相同,则可以根据第二数据组包括的T个第一数据块分别对应的第一校验值,确定第二数据组对应的第一校验值,且可以根据T个第二数据块的第一校验值确定目标数据组对应的第一校验值。以确定第二数据组对应的第一校验值的过程为例,具体地说,若第二数据组包括的第一数据块b4~第一数据块b6对应的第一校验值分别为R1、R2和R3,则可以通过将R1、R2和R3组合成一个数值,进而对该数值求HD5哈希值,即第二数据组对应的第一校验值。
若确定第二数据组对应的第一校验值与目标数据组对应的第一校验值相同,则服务器可以生成第二数据组的第一校验信息。其中,第二数据组的第一校验信息中可以包括第二数据包中与第一数据包相同的目标数据组所对应的位信息(即标识信息)。在一个示例中,若第一数据组对应的第二数据包即为第二数据包,则第二数据组的第一校验信息可以为:第二数据包中第1M至第300M的数据与第一数据包中第二数据组(第301M的数据至第601M的数据)相同。
本发明实施例中,在生成第二数据组的第一校验信息后,还可以对第一数据组对应的第二数据包进行更新。比如,可以将第一数据组对应的第二数据包中的目标数据组(即第1M至第300M的数据)替换为空洞,或者也可以将第1M至第300M的数据修改为特殊字符,或者还可以对第1M至第300M的数据进行标注,具体不作限定。
进一步地,服务器可以从目标数据组的下一位数据开始,执行上述的步骤。需要说明的是,若目标数据组与第二数据组相同,则在第二次执行的过程中,服务器可以从第二数据包中除目标数据组后的第1位数据开始,执行上述过程。当使用第二数据组对第一数据组对应的第二数据包中的每个数据执行完毕后,服务器可以通过上述的执行步骤依次对第一数据组对应的第二数据包进行更新,得到第二数据组对应的第二数据包。
在一种可能的实现方式中,若确定第二数据组对应的第二校验值与目标 数据组对应的第二校验值不同,则说明目标数据组与第二数据组不同,此时,服务器可以确定划分过程是否结束(第二数据包中的全部位数据变为空洞和/或全部已被标注,或第二数据组的数据量等于N个第一数据块中的最小数据量,则确定划分过程已结束)。若服务器确定划分过程未结束,则可以使用第三数据组从第二数据组对应的第一位数据开始,执行上述校验过程。若服务器确定划分过程已结束(即第二数据组的数据量等于N个第一数据块中的最小数据量),则服务器可以生成第二数据组的第二校验信息,并可以使用第三数据组从第二数据组对应的第一位数据开始,执行上述校验过程。其中,第二校验信息中可以包括第二数据包中与第一数据包不同的目标数据组所对应的位信息(即标识信息)。在一个示例中,若第一数据组对应的第二数据包即为第二数据包,则第二数据组的第二校验信息可以为:第二数据包中第1M至第300M的数据与第一数据包不同。
本发明实施例中,第一校验值可以为强校验值,第二校验值可以为弱校验值,通过采用先对比弱校验值再对比强校验值的方式,一方面可以通过弱校验值过滤掉目的数据组中与第Y数据组明显相同的数据信息,从而减轻工作量、提高数据处理的效率;另一方面,通过采用强校验值和弱校验值共同确定第Y数据组对应的校验信息,可以提高第Y数据组对应的校验信息的准确性。进一步地,校验信息可以包括第一校验信息与第二校验信息,第一校验信息为目标数据组中与第Y数据组相同的数据信息对应的校验信息,第二校验信息可以为目标数据组中与第Y数据组不同的数据信息对应的校验信息。通过获取第一校验信息和第二校验信息,服务器可以获取更为完整的校验信息,从而使得客户端设备向服务器同步的数据更为准确。
步骤104,服务器若确定第M数据组对应的第二数据包的数据量小于预设数据量阈值,则执行步骤105;若确定第M数据组对应的第二数据包的数据量大于或等于预设数据量阈值,则执行步骤102。
本发明实施例中,服务器在使用第三数据组对第二数据组对应的第二数据包进行校验后,可以对第二数据组对应的第二数据包进行更新,生成第三 数据组对应的第二数据包。进一步地,服务器若确定第三数据组对应的第二数据包的数据量小于预设数据量阈值,则说明第二数据包中的全部数据与第一数据包中的数据相同(即第二数据包中的全部位数据变为空洞和/或全部已被标注,校验过程结束);此时,服务器可以执行步骤105。其中,预设数据量阈值可以由本领域技术人员根据经验进行设置,具体不作限定。
相应地,服务器若确定第三数据组对应的第二数据包的数据量大于或等于预设数据量阈值,则说明第二数据包中存在与第一数据包中的数据不同的数据(即校验过程未结束);此时,服务器可以再次执行步骤102。具体实施中,服务器可以对N个第一数据块进行第二次划分,若将N个第一数据划分为P个数据组,则P个数据组中的每个数据组中包括的数据块的数量可以小于或等于第一次划分得到的M个数据组中的每个数据组中包括的数据块的数量。也就是说,P可以大于或等于M。
进一步地,服务器若确定P组数据中每组数据的数据量均小于N个第一数据块的数据量中的最小数据量,则说明服务器对第二数据包进行校验的过程已结束,此时,服务器可以将M组数据对应的校验信息发送给客户端设备。相应地,服务器若确定P组数据中数据量最大的数据量大于或等于N个第一数据块中的最小数据量,则说明服务器对第二数据包进行校验的过程未结束,此时,服务器可以针对于P个数据组中的每个数据组执行步骤103~步骤105(或102)。
本发明实施例中,通过采用多轮校验的方式获取校验信息,可以提高服务器获取到的校验信息的准确性,从而使得客户端设备与服务器之间的数据同步过程更为精确;进一步地,第二轮校验对应每组数据的数据量可以小于第一轮校验对应的每组数据的数据量,也就是说,本发明实施例通过多轮校验的方式,可以避免仅采用一轮校验所导致的数据信息重复传输的技术问题,从而可以节省网络流量,提高数据同步的效率。
步骤105,服务器根据M组数据的校验信息和第二数据包确定差异信息,并将差异信息发送给客户端设备。
此处,若服务器在第一次划分后,对第二数据包的校验过程即结束,则M组数据的校验信息可以仅包括第一校验信息,或者也可以仅包括第二校验信息,或者还可以包括第一校验信息和第二校验信息。若M组数据的校验信息仅包括第一校验信息,说明第二数据包中的数据与第一数据包中的数据相同;若M组数据的校验信息仅包括第二校验信息,说明第二数据包中的数据与第一数据包中的数据完全不同;若M组数据的校验信息包括第一校验信息和第二校验信息,说明第二数据包中的一部分数据与第一数据包中的数据相同,另一部分数据与第一数据包中的数据不同。
以第二数据组为例,若第二数据组的校验信息为:第二数据包中第1M至第300M的数据与第一数据包不同,则服务器可以获取第二数据包中的第1M至第300M的数据(即差异信息),并根据差异信息生成第二信息,将第二信息发送给客户端设备。相应地,客户端设备可以在接收到第二信息后,解析第二信息,得到差异信息,并根据差异信息更新第二数据包。比如,若客户端设备解析得到的差异信息为第二数据包中的第1M至第300M的数据,则客户端设备可以将第一数据包中的第1M至第300M的数据更新为第二数据包中的第1M至第300M的数据。
需要说明的是,上述步骤编号仅为执行流程的一种示例,并不构成对各个步骤的执行先后顺序的限定。
本发明的上述实施例中,通过使用第一数据包对第二数据包进行校验,可以使得服务器将差异信息传输给客户端设备,而无需将整个第一数据包发送给客户端设备,从而可以避免传输重复的数据信息,降低网络流量的损失,提高数据同步的效率;且,本发明实施例中通过采用数据块的校验值来确定数据组的校验值的方式,可以使得客户端设备与服务器之间仅执行一次通信交互即可完成同步过程,而无需执行多次通信交互,从而可以减轻网络通信的压力,降低网络传输流量。也就是说或,本发明实施例中的数据处理方法可以适应于低带宽和流量敏感的网络环境。
下面从客户端设备与服务器之间进行数据交互的角度描述本发明实施例 中的方案,其与图1所示意的数据处理方法相对应。
图2为本发明实施例提供的一种客户端设备与服务器交互的流程示意图,该方法包括:
步骤201,客户端设备向服务器发送第一信息。
此处,第一信息中可以包括N个数据块的数据量和N个数据块分别对应的第一校验值,其中,N个数据块可以为客户端设备对第一数据包进行划分得到的。
在一个示例中,第一信息中还可以包括其它与第一数据包相关的信息,比如第一数据包的标识信息、N个数据块的位信息和N个数据块的更新时间等,具体不作限定。
步骤202,服务器根据第一信息确定校验信息,并根据校验信息和第二数据包确定差异信息。
在一种可能的实现方式中,服务器可以在接收客户端设备发送的第一信息后,对第一信息进行解析,获取第一信息中包括的第一数据包的标识信息。进一步地,服务器可以根据第一数据包的标识信息,获取服务器内存储的与第一数据包对应的第二数据包,并可以根据第一信息和第二数据包,确定校验信息。具体的实现过程可以参照步骤102~步骤105(或步骤102)中示意的过程,具体不作限定。
若校验信息为第二数据包中的第三位数据至第九位数据与第一数据包不同,则服务器可以确定第二数据包中的第三位数据至第九位数据为差异信息。其中,位是指串行通信中的位数据,可以包括6个比特位,或者也可以包括9个比特位,具体不作限定。
步骤203,服务器将差异信息发送给客户端设备。
此处,服务器可以通过有线方式(比如网线、光纤)将差异信息发送给客户端设备,或者也可以通过无线方式(比如蓝牙、电磁波)将差异信息发送给客户端设备,具体不作限定。
步骤204,客户端设备根据差异信息更新客户端设备中存储的第一数据包。
在一种可能的实现方式中,差异信息中可以包括差异数据和标识信息,客户端设备可以根据标识信息将第一数据包中的数据更新为差异数据。
需要说明的是,图2仅是一种示例性的简单说明,其采用服务器作为确定校验信息的执行主体仅是为了便于说明方案,并不构成对方案的限定。在具体实施中,也可以采用客户端设备作为确定校验信息的执行主体,本发明实施例不作限定。
下面用示例一和示例二来具体描述本发明实施例中的方案。
示例一
图3a为本发明实施例提供的一种数据处理方法对应的实施例的示意图,如图3a所示,客户端设备上存储有旧的资源数据包a(简称为数据a),服务器存储有数据a对应的新的资源数据包b(简称为数据b)。其中,如图3a所示,数据a和数据b均可以为48位数据。其中,1位可以是指可以为1字节、1比特位等较小的数据量,或者也可以是指1M、1G等较大的数据量,对此不做限定。
需要说明的是,本发明的下列示例(即示例一和示例二)仅是两种示例性的简单说明,其所列举的数据a和数据b仅是为了便于说明方案,并不构成对方案的限定,在具体实施中,数据a和数据b的数据量可以远远大于48位,比如可以为100G或10T。
具体执行过程如下:
第一阶段,客户端将第一信息发送给服务器。
具体实施中,客户端设备可以对数据a进行划分,得到数据块S 1~数据块S 8,其中,数据块S 1~数据块S 8中的每个数据块均可以包括6位数据。如图3a所示,数据块S 1包括第一位数据~第六位数据,即数据ABCD12;数据块S 2包括第七位数据~第十二位数据,即数据ACDE23;数据块S 3包括第十三位数据~第十八位数据,即数据ABDE34;数据块S 4包括第十九位数据~第二十四位数据,即数据ADEF45;数据块S 5包括第二十五位数据~第三十位数据,即数据BDEF56;数据块S 6包括第三十一位数据~第三十六位数据,即数据 BCEF67;数据块S 7包括第三十七位数据~第四十二位数据,即数据BEDA78;数据块S 8包括第四十二位数据~第四十八位数据,即数据CEDF89。
进一步地,客户端设备可以分别计算数据块S 1~数据块S 8的第一检测值和第二检测值,第一检测值可以为强校验值,第二检测值可以为弱校验值。以数据块S 1为例,在一种可能的实现方式中,客户端设备可以通过校验和算法确定数据块S 1的弱校验值,并可以通过MD5哈希算法确定数据块S 1的强校验值。需要说明的是,本发明实施例可以使用数据块包括的数据来计算数据块的强校验值和弱校验值,比如,可以使用数据块S 1包括的数据ABCD12计算数据块S 1的强校验值和弱校验值。
本发明实施例中,客户端设备可以根据数据块S 1~数据块S 8的标识信息、数据量信息、弱校验值和强校验值生成第一信息中,并将第一信息发送给服务器。其中,数据块的标识信息可以为数据块包括的位数据的标识,比如数据块S 3的标识信息为第十九位数据~第二十四位数据;数据块的数据量信息可以为数据块包括的位数据的数量,比如数据块S 3的数据量信息为六位数据。
在一种可能的实现方式中,数据块S 1~数据块S 8的标识信息、数据量信息、弱校验值和强校验值可以以表格的形式存储在第一信息中,表1为第一信息中存储的表格的一种示例。如表1所示,以数据块S 1为例,数据块S 1的标识信息为第一位数据~第六位数据,数据量信息为六位数据,弱校验值为H 1,强校验值为U 1
表1:第一信息中表格的一种示意
数据块 标识信息 数据量信息 弱校验值 强校验值
S 1 1~6 6 H 1 U 1
S 2 2~12 6 H 2 U 2
S 3 13~18 6 H 3 U 3
S 4 19~24 6 H 4 U 4
S 5 25~30 6 H 5 U 5
S 6 31~36 6 H 6 U 6
S 7 37~42 6 H 7 U 7
S 8 43~48 6 H 8 U 8
第二阶段,服务器对数据块S 1~数据块S 8进行第一次划分。
服务器接收到客户端设备发送的第一信息后,可以解析第一信息得到数据块S 1~数据块S 8的标识信息、数据量信息、弱校验值和强校验值,并可以对数据块S 1~数据块S 8进行第一次划分,得到数据组M 11。其中,数据组M 11中可以包括数据块S 1~数据块S 8,即数据组M 11的标识信息为第一位数据~第四十八位数据,数据量信息为48位数据;且,服务器可以根据数据块S 1~数据块S 8的强校验值(即U 1~U 8)确定数据组M 11的弱校验值。
进一步地,服务器可以从数据b的第一位数据开始,获取与数据组M 11的数据量相同的目标数据组,目标数据组可以包括数据b中的48位数据,如图3a所示,目标数据组即为数据b,目标数据组包括的48位数据可以为“ABCD12ACDE23ABDE34ADEF45BDEF56BCEF67BEDA78DECA90”。服务器可以将目标数据组划分为与数据块S 1~数据块S 8数据量相同的八个数据块(如图3a所示意的八个数据块),并可以确定每个数据块的弱校验值和强校验值,进而根据八个数据块的强校验值确定目标数据组的弱校验值。服务器将数据组M 11的弱校验值和目标数据组的弱校验值进行对比,确定数据组M 11的弱校验值U 11和目标数据组的弱校验值不同。此时,服务器可以确定数据组M 11与目标数据组不同。
第三阶段,服务器对数据块S 1~数据块S 8进行第二次划分。
由于数据组M 11的数据量大于数据块S 1~数据块S 8中任一数据块的数据量,因此,服务器可以对数据块S 1~数据块S 8进行第二次划分,得到数据组M 21和数据组M 22。其中,数据组M 21中可以包括数据块S 1~数据块S 4,数据组M 11的标识信息为第一位数据~第二十四位数据,数据量信息为24位数据,这24位数据可以为“ABCD12ACDE23ABDE34ADEF45”;数据组M 22中可以包 括数据块S 5~数据块S 8,即数据组M 22的标识信息为第二十五位数据~第四十八位数据,数据量信息为24位数据,这24位数据可以为“BDEF56BCEF67BEDA78CEDF89”。且,服务器可以根据数据块S 1~数据块S 4的强校验值(即U 1~U 4)确定数据组M 21的弱校验值,并可以根据数据块S 5~数据块S 8的强校验值(即U 5~U 8)确定数据组M 22的弱校验值。
进一步地,服务器可以从数据b的第一位数据开始,获取与数据组M 21的数据量相同的目标数据组(如图3a所示的数据组M' 21)。其中,数据组M' 21可以包括数据b中的第一位数据至第二十四位数据,数据量信息为24位,这24位数据可以为“ABCD12ACDE23ABDE34ADEF45”。服务器可以将数据组M' 21划分为与数据块S 1~数据块S 4的数据量相同的四个数据块(即每个数据块可以包括6位数据),并确定每个数据块的弱校验值和强校验值,进而根据四个数据块的强校验值确定数据组M' 21的弱校验值。服务器将数据组M 21的弱校验值和数据组M' 21的弱校验值进行对比,可以确定M 21的弱校验值和数据组M' 21的弱校验值相同;此时,服务器可以根据数据块S 1~数据块S 4的强校验值(即U 1~U 4)确定数据组M 21的强校验值,并可以根据目标数据组M' 21中四个数据块的强校验值确定目标数据组M' 21的强校验值;进一步地,服务器可以将数据组M 21的强校验值和目标数据组M' 21的强校验值进行对比,确定数据组M 21的强校验值和目标数据组M' 21的强校验值相同,即数据组M 21与数据组M' 21相同;此时,服务器可以生成第一校验信息。
表2为第一校验信息的一种示例。如表2所示,第一校验信息可以为:数据a中的第一位数据~第二十四位数据与数据b中的第一位数据~第二十四位数据相同。
表2:第二阶段的校验结果
数据a 数据b 校验结果
第一位数据~第二十四位数据 第一位数据~第二十四位数据 相同
服务器可以在生成第一校验信息后,对数据b进行更新,在一种更新方式中,服务器可以将数据b中的第一位数据~第二十四位数据更新为空洞,从而得到更新后的数据b。其中,更新后的数据b中的第一位数据~第二十四位数据可以为空洞。进一步地,服务器可以根据更新后的数据b执行校验的过程,具体地说,服务器可以从更新后的数据b的第一位非空洞的数据(即第二十五位数据)开始,获取与数据组M 21的数据量相同的目标数据组(为便于描述,称为数据组M' 22),数据组M' 22可以包括数据b中的第二十五位数据至第四十八位数据,数据量信息为24位,这24位数据可以为“BDEF56BCEF67BEDA78DECA90”。服务器可以将数据组M' 22划分为与数据块S 1~数据块S 4的数据量相同的四个数据块,并可以确定每个数据块的弱校验值和强校验值,进而根据四个数据块的强校验值确定数据组M' 22的弱校验值。服务器将数据组M 21的弱校验值和数据组M' 22的弱校验值进行对比,可以确定数据组M 21的弱校验值和数据组M' 22的弱校验值不同,此时,服务器可以确定数据组M 21与数据组M' 22不同。
执行至此步骤,服务器即完成了使用数据组M 21对更新后的数据b的校验过程,进一步地,服务器可以使用数据组M 22对更新后的数据b进行校验,具体地执行过程可以参照上述内容,具体不再赘述。服务器使用数据组M 22对更新后的数据b进行校验的过程中,数据b中不存在与数据组M 22相同的数据,因此,服务器并未生成校验信息。
第三阶段,服务器对数据块S 1~数据块S 8进行第三次划分。
由于数据组M 21和数据组M 22的数据量大于数据块S 1~数据块S 8中任一数据块的数据量,因此,服务器可以对数据块S 1~数据块S 8进行第三次划分,得到数据组M 31、数据组M 32、数据组M 33和数据组M 34。其中,数据组M 31 中可以包括数据块S 1和数据块S 2,数据组M 31的标识信息可以为第一位数据~第十二位数据,数据量信息可以为12位数据,这12位数据可以为“ABCD12ACDE23”;数据组M 32中可以包括数据块S 3和数据块S 4,数据组M 32的标识信息可以为第十三位数据~第二十四位数据,数据量信息可以为12位数据,这12位数据可以为“ABDE34ADEF45”;数据组M 33中可以包括数据块S 5和数据块S 6,数据组M 33的标识信息可以为第二十五位数据~第三十六位数据,数据量信息可以为12位数据,这12位数据可以为“BDEF56BCEF67”;数据组M 34中可以包括数据块S 7和数据块S 8,即数据组M 33的标识信息为第三十七位数据~第四十八位数据,数据量信息为12位数据,这12位数据可以为“BEDA78CEDF89”。
进一步地,服务器可以根据数据块S 1和数据块S 2的强校验值(即U 1和U 2)确定数据组M 31的弱校验值,根据数据块S 3和数据块S 4的强校验值(即U 3和U 4)确定数据组M 32的弱校验值,根据数据块S 5和数据块S 6的强校验值(即U 5和U 6)确定数据组M 34的弱校验值,根据数据块S 7~数据块S 8的强校验值(即U 7~U 8)确定数据组M 34的弱校验值。
以数据组M 33为例,服务器从更新后的数据b的第一位数据(即第二十五位数据)开始,获取与数据组M 33的数据量相同的目标数据组(如图3a所示意的M' 31)。其中,数据组M' 31可以包括数据b中的第二十五位数据至第三十六位数据,数据量信息为12位,这12位数据可以为“BDEF56BCEF67”。服务器可以将数据组M' 31划分为与数据块S 1和数据块S 2的数据量相同的两个数据块,并确定每个数据块的弱校验值和强校验值,进而根据两个数据块的强校验值确定数据组M' 31的弱校验值。服务器将数据组M 33的弱校验值和数据组M' 31的弱校验值进行对比,可以确定数据组M 33的弱校验值和数据组M' 31的弱校验值相同;此时,服务器可以根据数据块S 1和数据块S 2的强校验值(即U 1和U 2)确定数据组M 33的强校验值,并可以根据目标数据组M' 31中两个数据块的强校验值确定目标数据组M' 31的强校验值;进一步地,服务器可以将数据组M 33的强校验值和目标数据组M' 31的强校验值进行对比,确定数据组 M 33的强校验值和目标数据组M' 31的强校验值相同,即数据组M 33与数据组M' 31相同;此时,服务器可以生成第一校验信息,如表3所示。
表3:第三阶段的校验结果
Figure PCTCN2020072336-appb-000001
进一步地,服务器可以将数据b中的第二十五位数据~第三十六位数据变成空洞,得到更新后的数据b。其中,更新后的数据b中的第一位数据~第三十六位数据可以为空洞。进一步地,服务器可以从数据b的第三十七位数据开始,获取与数据组M 33的数据量相同的目标数据组,根据数据组M 33的弱校验值与目标数据组的弱校验值的对比结果,以及数据组M 33的强校验值与目标数据组的强校验值的对比结果,执行校验过程,具体可以参照上述内容,不再赘述。
第四阶段,服务器对数据块S 1~数据块S 8进行第四次划分。
由于数据组M 31、数据组M 32、数据组M 33和数据组M 34的数据量大于数据块S 1~数据块S 8中任一数据块的数据量,因此,服务器可以对数据块S 1~数据块S 8进行第四次划分,得到数据块S1~数据块S8。针对于数据块S1~数据块S8和更新后的数据b进行校验,得到校验结果,具体实现过程可以参照上述内容,不再赘述。
在一个示例中,第四阶段的校验结果可以如表4所示。其中,数据a的第三十七位数据~第四十二位数据与数据b的第三十七位数据~第四十二位数据相同,且数据a的第四十二位数据~第四十八位数据与数据b的第四十二位数据~第四十八位数据不同。
表4:第四阶段的校验结果
Figure PCTCN2020072336-appb-000002
第五阶段,服务器将差异信息发送给客户端设备。
服务器在获取到第一阶段~第四阶段的校验结果(即表2~表4所示)后,可以根据校验结果确定数据b与数据a中相同的数据,并可以获取相同的数据在数据a中的标识信息和在数据b中的标识信息(即差异信息),进而可以将两个标识信息发送给客户端设备。举例来说,根据表2~表4的内容可知,数据a中的第一位数据~第二十四位数据与数据b中的第一位数据~第二十四位数据相同,数据a中的第二十五位数据~第三十六位数据与数据b中的第二十五位数据~第三十六位数据相同,数据a中的第三十七位数据~第四十二位数据与数据b中的第三十七位数据~第四十二位数据相同,且数据a中的第四十二位数据~第四十八位数据与数据b中的第四十二位数据~第四十八位数据不同;此时,服务器可以将相同数据在数据a中的标识和在数据b中的标识发送给客户端设备。
表5为本发明实施例提供的一种差异信息的示意表。
表5:一种差异信息的示意
数据a 数据b
第一位数据~第二十四位数据 第一位数据~第二十四位数据
第二十五位数据~第三十六位数据 第二十五位数据~第三十六位数据
第三十七位数据~第四十二位数据 第三十七位数据~第四十二位数据
客户端设备接收到服务器发送的差异信息后,可以从差异信息中确定第 一数据包与第二数据包中相同的数据所在的标识信息,若确定相同的数据在数据a中的标识信息与数据b中的标识信息相同,则可以无需根据该校验结果更新客户端设备中的数据a;服务器若确定相同的数据在数据a中的标识信息(比如第4位数据)与数据b中的标识信息(比如第12位数据)不同,则可以将数据a中的第12位数据替换为数据a中的第12位数据。举例说明,如表2所示的校验结果,数据a的第一位数据~第二十四位数据与数据b的第一位数据~第二十四位数据相同,则服务器可以无需对数据a的第一位数据~第二十四位数据进行更新,即服务器可以无需针对于该校验结果执行任何操作。
进一步地,针对于数据a与数据b不同的数据,客户端设备可以采用多种方法获取不同的数据,比如可以通过网络下载,具体不作限定。在一种可能的实现方式中,针对于校验结果不同的数据,服务器可以获取数据b中不同于数据a的数据,并发送给客户端设备。具体地说,服务器可以根据表4的内容确定数据a的第四十二位数据~第四十八位数据与数据b的第四十二位数据~第四十八位数据不同,因此,服务器可以获取数据b的第四十二位数据~第四十八位数据(即DECA90),并将DECA90与数据a的标识信息(第四十二位数据~第四十八位数据)发送给客户端设备,以使客户端设备将数据a的第四十二位数据~第四十八位数据(即CEDF89)更新为DECA90。
示例二
图3b为本发明实施例提供的另一个数据处理方法对应的实施例的示意图,如图3b所示,客户端设备上存储有旧资源数据包c(简称为数据c),服务器存储有数据c对应的新资源数据包的数据d(简称为数据d)。其中,数据c可以包括60位数据,这60位数据为“ABCD12ACDE23ABDE34ADEF45BDEF56BCEF67BEDA78CEDF89CFAD90CDFB01”;数据d可以包括44位数据,这44位数据为“38AFABCD12ACDE23ABDE34ADEF45BDEF5691EBCEDF89”。使用数据d对数据c进行更新的执行过程如下:
第一阶段,客户端将第一信息发送给服务器。
具体实施中,客户端设备可以对数据c进行划分,得到数据块Q 1~数据块Q 10,其中,数据块Q 1~数据块Q 10中的每个数据块均可以包括6位数据。如图3b所示,数据块Q 1包括第一位数据~第六位数据,即数据ABCD12;数据块Q 2包括第七位数据~第十二位数据,即数据ACDE23;数据块Q 3包括第十三位数据~第十八位数据,即数据ABDE34;数据块Q 4包括第十九位数据~第二十四位数据,即数据ADEF45;数据块Q 5包括第二十五位数据~第三十位数据,即数据BDEF56;数据块Q 6包括第三十一位数据~第三十六位数据,即数据BCEF67;数据块Q 7包括第三十七位数据~第四十二位数据,即数据BEDA78;数据块Q 8包括第四十二位数据~第四十八位数据,即数据CEDF89;数据块Q 9包括第四十九位数据~第五十四位数据,即数据CFAD90;数据块Q 10包括第五十五位数据~第六十位数据,即数据CDFB01。
进一步地,客户端设备可以分别计算数据块Q 1~数据块Q 10的弱校验值和强校验值,并可以根据数据块Q 1~数据块Q 10的标识信息、数据量信息、弱校验值和强校验值生成第一信息,将第一信息发送给服务器。
第二阶段,服务器对数据块Q 1~数据块Q 10进行第一次划分。
服务器接收到客户端设备发送的第一信息后,可以对数据块Q 1~数据块Q 10进行第一次划分,得到数据组E 11、数据组E 12和数据组E 13。其中,数据组E 11中可以包括数据块Q 1~数据块Q 4,数据组E 11的标识信息为第一位数据~第二十四位数据,数据量信息为24位数据,这24位数据为“ABCD12ACDE23ABDE34ADEF45”;数据组E 12中可以包括数据块Q 5~数据块Q 8,数据组E 12的标识信息为第二十五位数据~第四十八位数据,数据量信息为24位数据,这24位数据为“BDEF56BCEF67BEDA78CEDF89”;数据组E 13中可以包括数据块Q 9~数据块Q 10,数据组E 13的标识信息为第四十九位数据~第六十位数据,数据量信息为12位数据,这12位数据为“CFAD90CDFB01”。且,服务器可以根据数据块Q 1~数据块Q 4的强校验值确定数据组E 11的弱校验值,根据数据块Q 5~数据块Q 8的强校验值确定数据组E 12的弱校验值,并可以根据数据块Q 9~数据块Q 10的强校验值确定数据组E 13的弱校验值。
进一步地,服务器可以从数据d的第一位数据开始,获取与数据组E 11的数据量相同的目标数据组。其中,目标数据组可以包括数据b中的24位数据,这24位数据为“38AFABCD12ACDE23ABDE34AD”。服务器可以将目标数据组划分为与数据块Q 1~数据块Q 4的数据量相同的四个数据块,并可以确定每个数据块的弱校验值和强校验值,进而根据四个数据块的强校验值确定目标数据组的弱校验值。服务器将数据组E 11的弱校验值和目标数据组的弱校验值进行对比,确定数据组E 11的弱校验值和目标数据组的弱校验值不同。此时,服务器可以确定数据组E 11与数据组E' 11之间存在差异信息。
服务器可以从数据d的第二位数据开始,获取与数据组E 11的数据量相同的目标数据组。其中,目标数据组可以包括数据b中的24位数据,这24位数据为“8AFABCD12ACDE23ABDE34ADE”。服务器确定数据组E 12的弱校验值和目标数据组的弱校验值不同,并确定数据组E 12的强校验值和目标数据组的强校验值不同;此时,服务器可以确定数据组E 11与目标数据组不同。进一步地,服务器可以从数据d的第三位数据开始,获取与数据组E 11的数据量相同的目标数据组。其中,目标数据组可以包括数据b中的24位数据,这24位数据为“AFABCD12ACDE23ABDE34ADEF”。服务器确定数据组E 12的弱校验值和目标数据组的弱校验值不同,此时,服务器可以确定数据组E 11与目标数据组不同。
按照上述过程,服务器可以从数据d的第一位数据依次执行至最后一位数据,在执行至第五位数据时,服务器可以确定从数据d的第五位数据开始获取到的目标数据组(如图3b所示意的数据组E' 11)与数据组E 11相同,此时,服务器可以根据数据块Q 1~数据块Q 4的强校验值确定数据组E 11的强校验值,并目标数据组的四个数据块的强校验值确定目标数据组E' 11的强校验值;服务器将数据组E 11的强校验值和目标数据组E' 11的强校验值进行对比,确定数据组E 11的强校验值和目标数据组E' 11的强校验值相同,因此,服务器可以生成校验结果。其中,数据组E' 11可以包括数据d中的24位数据,这24位数据为“ABCD12ACDE23ABDE34ADEF45”。服务器在生成校验结果后,可以将数 据d中的第五位数据~第二十九位数据更新为空洞,得到更新后的数据d;并可以从数据d的第三十位数据开始,使用数据组E 11进行校验。
若数据组E 11对数据d的校验过程结束,则服务器可以针对于更新后的数据d的第一位数据(即第三十位数据)开始,使用数据组E 12和数据组E 13分别对数据d进行校验。具体执行过程可以参照上述内容,不再赘述。在使用数据组E 12和数据组E 13对数据d进行校验的过程中,数据d中不存在与数据组E 12或数据组E 13相同的数据,因此,服务器在该过程中未生成校验结果。
在一个示例中,第二阶段的校验结果可以如表6所示。其中,数据c的第一位数据~第二十四位数据与数据d的第五位数据~第二十九位数据相同。
表6:第二阶段的校验结果
数据c 数据d 校验结果
第一位数据~第二十四位数据 第五位数据~第二十九位数据 相同
第三阶段,服务器对数据块Q 1~数据块Q 10进行第二次划分。
服务器可以对数据块Q 1~数据块Q 10进行第二次划分,得到数据组E 21、数据组E 22、数据组E 23、数据组E 24和数据组E 25。在使用数据组E 21、数据组E 22、数据组E 23、数据组E 24和数据组E 5对数据d校验的过程中,数据d中不存在与数据组E 21、数据组E 22、数据组E 23、数据组E 24或数据组E 25相同的数据,因此,服务器在该过程中未生成校验信息。
第四阶段,服务器对数据块Q 1~数据块Q 10进行第三次划分。
服务器可以对数据块Q 1~数据块Q 10进行第三次划分,得到数据块Q 1~数据块Q 10。针对于数据块Q 1~数据块Q 10和更新后的数据d进行校验,得到校验结果。
在一个示例中,第四阶段的校验结果可以如表7所示。其中,数据a的第三十七位数据~第四十二位数据与数据b的第三十七位数据~第四十二位数据相同,且数据a的第四十二位数据~第四十八位数据与数据b的第四十二位数据~第四十八位数据不同。
表7:第四阶段的校验结果
Figure PCTCN2020072336-appb-000003
第五阶段,服务器将差异信息发送给客户端设备。
服务器在获取到第一阶段~第四阶段的校验结果(即表6和表7)后,针对于校验结果相同的数据,以表6所示的校验结果为例,数据c的第一位数据~第二十四位数据与数据d的第五位数据~第二十九位数据相同,因此,服务器可以将校验结果相同的数据在数据c中的标识信息(即第一位数据~第二十四位数据)与数据d中的标识信息(即第五位数据~第二十九位数据)发送给客户端设备,以使客户端设备根据数据c中的第一位数据~第二十四位数据更新数据c中的第五位数据~第二十九位数据,即将数据c中的第五位数据~第二十九位数据12ACDE23ABDE34ADEF45BDEF更新为原数据c中的第一位数据~第二十四位数据ABCD12ACDE23ABDE34ADEF45。
在一种可能的实现方式中,针对于校验结果不同的数据,服务器可以获取数据d中不同于数据c的数据,并发送给客户端设备。如表7所示,服务器可以确定数据d的第一位数据~第四位数据、第三十五位数据~第三十八位数据与数据c不同,因此,服务器可以获取数据d的第一位数据~第四位数据(即38AF)和第三十五位数据~第三十八位数据(即91EB),并将38AF与数据d的标识信息(第一位数据~第四位数据)、91EB与数据d的标识信息(第三十五位数据~第三十八位数据)发送给客户端设备,以使客户端设备将数据c的第一位数据~第四位数据(即ABCD)更新为38AF,将数据c的第三十五 位数据~第三十八位数据(即67BE)更新为91EB。
本发明的上述实施例中,服务器获取客户端设备发送的第一信息后,对N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块对应的校验值和与第一数据包对应的第二数据包,生成每组数据对应的校验信息,其中,第一信息包括第一数据包对应的N个第一数据块的数据量和N个第一数据块分别对应的校验值;进一步地,服务器根据M组数据对应的校验信息和第二数据包,确定第一数据包对应的差异信息,并将差异信息发送给客户端设备,以使客户端设备根据差异信息更新第一数据包。本发明实施例中,通过使用第一数据包对第二数据包进行校验,可以使得服务器将差异信息传输给客户端设备,而无需将整个第一数据包发送给客户端设备,从而可以避免传输重复的数据信息,降低网络流量的损失,提高数据同步的效率;且,本发明实施例中通过采用数据块的校验值来确定数据组的校验值的方式,可以使得客户端设备与服务器之间仅执行一次通信交互即可完成同步过程,而无需执行多次通信交互,从而可以减轻网络通信的压力,降低网络传输流量。
针对上述方法流程,本发明实施例还提供一种服务器,该服务器的具体内容可以参照上述方法实施。
图4为本发明实施例提供的一种服务器的结构示意图,包括:
获取模块401,用于获取客户端设备发送的第一信息,所述第一信息包括第一数据包对应的N个第一数据块的数据量和所述N个第一数据块分别对应的校验值;
处理模块402,用于对所述N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息;
收发模块403,用于根据所述M组数据对应的校验信息和所述第二数据包,确定所述第一数据包对应的差异信息,并将所述差异信息发送给所述客户端设备,以使所述客户端设备根据所述差异信息更新所述第一数据包。
可选地,所述处理模块402根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息之前,还用于:
确定所述M组数据中的最大数据量大于或等于所述N个第一数据块的数据量中的最小数据量。
可选地,所述第一信息中包括的所述N个第一数据块分别对应的校验值为所述N个第一数据块分别对应的第一校验值;
所述处理模块402用于:
所述M组数据中包括第Y数据组,所述第Y数据组中包括T个第一数据块,针对于所述M组数据中的第Y数据组,执行:
根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第二校验值;
从第Y-1数据组对应的第二数据包中的第i位数据开始,获取与所述第Y数据组的数据量相同的第一目标数据组,并对所述第一目标数据组进行划分,得到T个第二数据块,所述T个第二数据块的数据量与所述T个第一数据块的数据量相同;根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第二校验值;根据所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息;
对所述第Y-1组数据对应的第二数据包进行更新,得到第Y数据组对应的第二数据包。
可选地,所述处理模块402用于:
若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值相同,则根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第一校验值,并根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第一校验值;若确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值不同,则生成所述第Y数据组的第一校验信息;
若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值不同,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第二目标数据组。
可选地,所述处理模块402还用于:
确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值相同时,若确定所述第Y数据组的数据量等于所述N个第一数据块中的最小数据量,则生成所述第Y数据组的第二校验信息;若确定所述第Y数据组的数据量大于所述N个第一数据块中的最小数据量,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第三目标数据组。
可选地,所述处理模块402还用于:
若确定所述第M组数据对应的第二数据包的数据量大于或等于预设数据量阈值,则对所述N个第一数据块进行第二次划分。
可选地,所述处理模块402用于:
对所述N个第一数据块进行第二次划分得到P组数据;
若确定所述P组数据中的最大数据量小于所述N个第一数据块的数据量中的最小数据量,则将第一至第M组数据的校验信息发送给所述客户端设备。
从上述内容可以看出:本发明的上述实施例中,服务器获取客户端设备发送的第一信息后,对N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块对应的校验值和与第一数据包对应的第二数据包,生成每组数据对应的校验信息,其中,第一信息包括第一数据包对应的N个第一数据块的数据量和N个第一数据块分别对应的校验值;进一步地,服务器根据M组数据对应的校验信息和第二数据包,确定第一数据包对应的差异信息,并将差异信息发送给客户端设备,以使客户端设备根据差异信息更新第一数据包。本发明实施例中,通过使用第一数据包对第二数据包进行校验,可以使得服务器将差异信息传输给客户端设备,而无需将整个第一数据包发送给客户端设备,从而可以避免传输重复的数据信息,降低网络 流量的损失,提高数据同步的效率;且,本发明实施例中通过采用数据块的校验值来确定数据组的校验值的方式,可以使得客户端设备与服务器之间仅执行一次通信交互即可完成同步过程,而无需执行多次通信交互,从而可以减轻网络通信的压力,降低网络传输流量。
基于相同的技术构思,本发明实施例提供一种数据处理设备。至少一个处理器;以及,与所述至少一个处理器通信连接的存储器;所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器能够执行上述实施例中的数据处理方法。
以一个处理器为例,图5为本发明实施例提供的数据处理设备的结构,该数据处理设备500包括:收发器501、处理器502、存储器503和总线系统504;
其中,存储器503,用于存放程序。具体地,程序可以包括程序代码,程序代码包括计算机操作指令。存储器503可能为随机存取存储器(random access memory,简称RAM),也可能为非易失性存储器(non-volatile memory),例如至少一个磁盘存储器。图中仅示出了一个存储器,当然,存储器也可以根据需要,设置为多个。存储器503也可以是处理器502中的存储器。
存储器503存储了如下的元素,可执行模块或者数据结构,或者它们的子集,或者它们的扩展集:
操作指令:包括各种操作指令,用于实现各种操作。
操作系统:包括各种系统程序,用于实现各种基础业务以及处理基于硬件的任务。
上述本发明实施例数据处理方法可以应用于处理器502中,或者说由处理器502实现。处理器502可能是一种集成电路芯片,具有信号的处理能力。在实现过程中,上述数据处理方法的各步骤可以通过处理器502中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器502可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可 以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本发明实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成,或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器503,处理器502读取存储器503中的信息,结合其硬件执行以下步骤:
所述收发器501,用于获取客户端设备发送的第一信息,所述第一信息包括第一数据包对应的N个第一数据块的数据量和所述N个第一数据块分别对应的校验值;
所述处理器502,用于对所述N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息;
所述收发器501,还用于根据所述M组数据对应的校验信息和所述第二数据包,确定所述第一数据包对应的差异信息,并将所述差异信息发送给所述客户端设备,以使所述客户端设备根据所述差异信息更新所述第一数据包。
可选地,所述处理器502根据每组数据包括的至少一个第一数据块对应的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息之前,还用于:
确定所述M组数据中的最大数据量大于或等于所述N个第一数据块的数据量中的最小数据量。
可选地,所述第一信息中包括的所述N个第一数据块分别对应的校验值为所述N个第一数据块分别对应的第一校验值;
所述处理器502用于:
所述M组数据中包括第Y数据组,所述第Y数据组中包括T个第一数据块,针对于所述M组数据中的第Y数据组,执行:
根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定 所述第Y数据组对应的第二校验值;
从第Y-1数据组对应的第二数据包中的第i位数据开始,获取与所述第Y数据组的数据量相同的第一目标数据组,并对所述第一目标数据组进行划分,得到T个第二数据块,所述T个第二数据块的数据量与所述T个第一数据块的数据量相同;根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第二校验值;根据所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息;
对所述第Y-1组数据对应的第二数据包进行更新,得到第Y数据组对应的第二数据包。
可选地,所述处理器502用于:
若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值相同,则根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第一校验值,并根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第一校验值;若确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值不同,则生成所述第Y数据组的第一校验信息;
若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值不同,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第二目标数据组。
可选地,所述处理器502还用于:
确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值相同时,若确定所述第Y数据组的数据量等于所述N个第一数据块中的最小数据量,则生成所述第Y数据组的第二校验信息;若确定所述第Y数据组的数据量大于所述N个第一数据块中的最小数据量,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第三目标数据组。
可选地,所述处理器502还用于:
若确定所述第M组数据对应的第二数据包的数据量大于或等于预设数据量阈值,则对所述N个第一数据块进行第二次划分。
可选地,所述处理器502用于:
对所述N个第一数据块进行第二次划分得到P组数据;
若确定所述P组数据中的最大数据量小于所述N个第一数据块的数据量中的最小数据量,则将第一至第M组数据的校验信息发送给所述客户端设备。
本发明实施例的数据处理设备以多种形式存在,包括但不限于:
(1)超移动个人计算机设备:这类设备属于个人计算机的范畴,有计算和处理功能,一般也具备移动上网特性。这类终端包括:PDA、MID和UMPC设备等,例如iPad。
(2)其他具有数据处理功能的电子装置。
本领域技术人员可以理解实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序存储在一个存储介质中,包括若干指令用以使得一个设备(可以是单片机,芯片等)或处理器(processor)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
基于同一发明构思,本发明实施例还提供一种计算机可读存储介质,包括指令,当其在计算机上运行时,使得计算机执行如上述图1或图2所述的数据处理方法。
基于同一发明构思,本发明实施例还提供一种计算机程序产品,当其在计算机上运行时,使得计算机执行如上述图1或图2所述的数据处理方法。
本领域内的技术人员应明白,本发明的实施例可提供为方法、或计算机程序产品。因此,本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、 CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明的优选实施例,但本领域内的技术人员一旦得知了基本创造性概念,则可对这些实施例作出另外的变更和修改。所以,所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。
显然,本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样,倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内,则本发明也意图包含这些改动和变型在内。

Claims (17)

  1. 一种数据处理方法,其特征在于,所述方法包括:
    服务器获取客户端设备发送的第一信息,所述第一信息包括所述第一数据包对应的N个第一数据块的校验值;
    所述服务器对所述N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息;
    所述服务器根据所述M组数据对应的校验信息和所述第二数据包,确定所述第一数据包对应的差异信息,并将所述差异信息发送给所述客户端设备,以使所述客户端设备根据所述差异信息更新所述第一数据包。
  2. 根据权利要求1所述的方法,其特征在于,所述根据每组数据包括的至少一个第一数据块的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息之前,还包括:
    所述服务器确定所述M组数据中的最大数据量大于或等于所述N个第一数据块的数据量中的最小数据量。
  3. 根据权利要求1所述的方法,其特征在于,所述第一信息中还包括所述N个第一数据块的数据量,所述第一信息中包括的所述N个第一数据块分别对应的校验值为所述N个第一数据块分别对应的第一校验值;
    所述根据每组数据包括的至少一个第一数据块的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息,包括:
    所述M组数据中包括第Y数据组,所述第Y数据组中包括T个第一数据块,针对于所述M组数据中的第Y数据组,执行:
    所述服务器根据所述第Y数据组包括的T个第一数据块的第一校验值,确定所述第Y数据组对应的第二校验值;
    所述服务器从第Y-1数据组对应的第二数据包中的第i位数据开始,获取与所述第Y数据组的数据量相同的第一目标数据组,并对所述第一目标数据组进 行划分,得到T个第二数据块,所述T个第二数据块的数据量与所述T个第一数据块的数据量相同;根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第二校验值;根据所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息;
    所述服务器对所述第Y-1组数据对应的第二数据包进行更新,得到第Y数据组对应的第二数据包。
  4. 根据权利要求3所述的方法,其特征在于,所述根据所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息,包括:
    所述服务器若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值相同,则根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第一校验值,并根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第一校验值;若确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值相同,则生成所述第Y数据组的第一校验信息;
    所述服务器若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值不同,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第二目标数据组。
  5. 根据权利要求4所述的方法,其特征在于,所述方法还包括:
    所述服务器确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值相同时,若确定所述第Y数据组的数据量等于所述N个第一数据块中的最小数据量,则生成所述第Y数据组的第二校验信息;若确定所述第Y数据组的数据量大于所述N个第一数据块中的最小数据量,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第三目标数据组。
  6. 根据权利要求1至5中任一项所述的方法,其特征在于,所述方法还包括:
    所述服务器若确定所述第M组数据对应的第二数据包的数据量大于或等于预设数据量阈值,则对所述N个第一数据块进行第二次划分。
  7. 根据权利要求6所述的方法,其特征在于,所述服务器对所述N个第一数据块进行第二次划分得到P组数据;
    所述服务器若确定所述P组数据中的最大数据量小于所述N个第一数据块的数据量中的最小数据量,则将第一至第M组数据的校验信息发送给所述客户端设备。
  8. 一种服务器,其特征在于,所述服务器包括:
    获取模块,用于获取客户端设备发送的第一信息,所述第一信息包括第一数据包对应的N个第一数据块的校验值;
    处理模块,用于对所述N个第一数据块进行第一次划分,得到M组数据,并根据每组数据包括的至少一个第一数据块的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息;
    收发模块,用于根据所述M组数据对应的校验信息和所述第二数据包,确定所述第一数据包对应的差异信息,并将所述差异信息发送给所述客户端设备,以使所述客户端设备根据所述差异信息更新所述第一数据包。
  9. 根据权利要求8所述的服务器,其特征在于,所述处理模块根据每组数据包括的至少一个第一数据块的校验值和与所述第一数据包对应的第二数据包,生成所述每组数据对应的校验信息之前,还用于:
    确定所述M组数据中的最大数据量大于或等于所述N个第一数据块的数据量中的最小数据量。
  10. 根据权利要求8所述的服务器,其特征在于,所述第一信息中还包括所述N个第一数据块的数据量,所述第一信息中包括的所述N个第一数据块分别对应的校验值为所述N个第一数据块分别对应的第一校验值;
    所述处理模块用于:
    所述M组数据中包括第Y数据组,所述第Y数据组中包括T个第一数据块,针对于所述M组数据中的第Y数据组,执行:
    根据所述第Y数据组包括的T个第一数据块的第一校验值,确定所述第Y数据组对应的第二校验值;
    从第Y-1数据组对应的第二数据包中的第i位数据开始,获取与所述第Y数据组的数据量相同的第一目标数据组,并对所述第一目标数据组进行划分,得到T个第二数据块,所述T个第二数据块的数据量与所述T个第一数据块的数据量相同;根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第二校验值;根据所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值,生成所述第Y数据组的校验信息;
    对所述第Y-1组数据对应的第二数据包进行更新,得到第Y数据组对应的第二数据包。
  11. 根据权利要求10所述的服务器,其特征在于,所述处理模块用于:
    若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值相同,则根据所述第Y数据组包括的T个第一数据块分别对应的第一校验值,确定所述第Y数据组对应的第一校验值,并根据所述T个第二数据块的第一校验值确定所述第一目标数据组对应的第一校验值;若确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值不同,则生成所述第Y数据组的第一校验信息;
    若确定所述第Y数据组对应的第二校验值与所述第一目标数据组对应的第二校验值不同,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第二目标数据组。
  12. 根据权利要求11所述的服务器,其特征在于,所述处理模块还用于:
    确定所述第Y数据组对应的第一校验值与所述第一目标数据组对应的第一校验值相同时,若确定所述第Y数据组的数据量等于所述N个第一数据块中的最小数据量,则生成所述第Y数据组的第二校验信息;若确定所述第Y数据组的数据量大于所述N个第一数据块中的最小数据量,则从所述第Y-1数据组对应的第二数据包中的第i+1位数据开始,获取与所述第Y数据组的数据量相同的第三目标数据组。
  13. 根据权利要求8至12中任一项所述的服务器,其特征在于,所述处理模块还用于:
    若确定所述第M组数据对应的第二数据包的数据量大于或等于预设数据量阈值,则对所述N个第一数据块进行第二次划分。
  14. 根据权利要求13所述的服务器,其特征在于,所述处理模块用于:
    对所述N个第一数据块进行第二次划分得到P组数据;
    若确定所述P组数据中的最大数据量小于所述N个第一数据块的数据量中的最小数据量,则将第一至第M组数据的校验信息发送给所述客户端设备。
  15. 一种计算机可读存储介质,其特征在于,包括指令,当其在计算机上运行时,使得计算机执行如权利要求1至7任一项所述的方法。
  16. 一种计算机程序产品,其特征在于,当其在计算机上运行时,使得计算机执行如权利要求1至7任一项所述的方法。
  17. 一种数据处理设备,其特征在于,包括:
    至少一个处理器;以及,
    与所述至少一个处理器通信连接的存储器;其中,
    所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器能够执行权利要求1-7任一所述数据处理方法。
PCT/CN2020/072336 2019-04-04 2020-01-15 一种数据处理方法及服务器、设备 WO2020199724A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910272205.8 2019-04-04
CN201910272205.8A CN110083614B (zh) 2019-04-04 2019-04-04 一种数据处理方法及服务器

Publications (1)

Publication Number Publication Date
WO2020199724A1 true WO2020199724A1 (zh) 2020-10-08

Family

ID=67414244

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/072336 WO2020199724A1 (zh) 2019-04-04 2020-01-15 一种数据处理方法及服务器、设备

Country Status (2)

Country Link
CN (1) CN110083614B (zh)
WO (1) WO2020199724A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023241582A1 (zh) * 2022-06-14 2023-12-21 华为技术有限公司 数据同步方法、系统、介质、程序产品和电子设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110083614B (zh) * 2019-04-04 2023-06-06 中国银联股份有限公司 一种数据处理方法及服务器

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103297482A (zh) * 2012-03-05 2013-09-11 联想(北京)有限公司 信息处理方法和设备
CN104243508A (zh) * 2013-06-07 2014-12-24 富鸿康科技(深圳)有限公司 服务器、客户端及文件同步方法
CN106528125A (zh) * 2016-10-26 2017-03-22 腾讯科技(深圳)有限公司 一种数据文件的增量更新方法和服务器、客户端以及系统
CN110083614A (zh) * 2019-04-04 2019-08-02 中国银联股份有限公司 一种数据处理方法及服务器

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101854241A (zh) * 2009-03-30 2010-10-06 上海聚力传媒技术有限公司 一种用于对网络传输的数据块进行验证的方法和装置
CN104579605B (zh) * 2013-10-23 2018-04-10 华为技术有限公司 一种数据传输方法及装置
CN105682183B (zh) * 2016-01-14 2017-06-16 美的集团股份有限公司 控制家电设备入网的方法和装置
CN108255647B (zh) * 2018-01-18 2021-03-23 湖南麒麟信安科技股份有限公司 一种samba服务器集群下的高速数据备份方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103297482A (zh) * 2012-03-05 2013-09-11 联想(北京)有限公司 信息处理方法和设备
CN104243508A (zh) * 2013-06-07 2014-12-24 富鸿康科技(深圳)有限公司 服务器、客户端及文件同步方法
CN106528125A (zh) * 2016-10-26 2017-03-22 腾讯科技(深圳)有限公司 一种数据文件的增量更新方法和服务器、客户端以及系统
CN110083614A (zh) * 2019-04-04 2019-08-02 中国银联股份有限公司 一种数据处理方法及服务器

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023241582A1 (zh) * 2022-06-14 2023-12-21 华为技术有限公司 数据同步方法、系统、介质、程序产品和电子设备

Also Published As

Publication number Publication date
CN110083614A (zh) 2019-08-02
CN110083614B (zh) 2023-06-06

Similar Documents

Publication Publication Date Title
CN109345388B (zh) 区块链智能合约验证方法、装置及存储介质
US20200145222A1 (en) Optimal data storage configuration in a blockchain
US9292548B2 (en) Digest generation
WO2020199724A1 (zh) 一种数据处理方法及服务器、设备
US11977637B2 (en) Technique for authentication and prerequisite checks for software updates
CN106484453B (zh) 一种实现系统升级的方法及装置
CN103595661B (zh) 一种报文分片重组方法及装置
US8948037B1 (en) Checksum trailer in timing protocols
US20150333766A1 (en) Run time compression method for a vehicle communication bus
CN114661318A (zh) 针对资源受限设备定制的高效后量子安全软件更新
CN110213018B (zh) 车载总线的数据通信方法、装置及车辆
WO2022116088A1 (zh) 固件数据处理方法及装置
CN105300436A (zh) 传感器系统中的校准数据
CN104580202B (zh) 报文的匹配方法和装置
US7453879B1 (en) Method and apparatus for determining the landing zone of a TCP packet
WO2017211076A1 (zh) 一种互联文档的自动填报方法
CN117014318B (zh) 多尺度网络节点间链路的添加方法、装置、设备及介质
CN105704207A (zh) 一种生成唯一标识的方法、终端和网络侧
WO2017211077A1 (zh) 一种跨平台的互联文档id生成方法
US20160218828A1 (en) Sensor interface that provides a long package crc to improve functional safety
CN111193730B (zh) 一种IoT可信场景构建方法及装置
CN112905542A (zh) 一种文件处理方法、装置、服务器、设备及存储介质
CN113448764A (zh) 校验码生成方法、装置、电子设备及计算机存储介质
CN108270508A (zh) 一种循环冗余校验crc实现方法、装置及网络设备
CN109194674A (zh) 一种通信数据的处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20784165

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20784165

Country of ref document: EP

Kind code of ref document: A1