CN106921734B - Client and server data interaction method based on selective caching and synchronization - Google Patents
Client and server data interaction method based on selective caching and synchronization Download PDFInfo
- Publication number
- CN106921734B CN106921734B CN201710076363.7A CN201710076363A CN106921734B CN 106921734 B CN106921734 B CN 106921734B CN 201710076363 A CN201710076363 A CN 201710076363A CN 106921734 B CN106921734 B CN 106921734B
- Authority
- CN
- China
- Prior art keywords
- data
- client
- processing
- server
- stage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5683—Storage of data provided by user terminals, i.e. reverse caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
- H04L67/5651—Reducing the amount or size of exchanged application data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5681—Pre-fetching or pre-delivering data based on network characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Computer And Data Communications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a client and server data interaction method based on selective caching and synchronization, which adopts a non-strict real-time method of selective caching and synchronization to synchronize data of a client and a server. The method abstracts a data interaction process into a data processing task, and each processing task is divided into a plurality of same or similar processing stages. The smallest unit of "selective caching and synchronization" is the so-called "processing stage". The characteristics and the process of the method can be seen that the method is more suitable for the scene that a client needs to do more data processing and analyzing tasks. Compared with the traditional real-time synchronization mode, the method has the advantages of saving bandwidth, reducing the requirement on network quality, ensuring real-time data processing of the client, reducing the storage pressure of the server and the like.
Description
Technical Field
The method relates to a method for data interaction between a client and a server in a C/S mode, in particular to a client and server data interaction method based on selective caching and synchronization.
Background
The C/S and B/S modes are two well-known software architectures. The C/S architecture is called Client/Server in its entirety, i.e. Client-Server architecture. The method is characterized in that the analysis and processing tasks of the data are reasonably distributed to the client and the server to be realized. However, the B/S architecture is different, and almost all data analysis and processing are distributed to the server side, and the browser as the client side is only responsible for displaying, and here, we focus on discussing the C/S architecture.
Since the tasks of data processing and analysis are distributed to the front end and the back end of the system in the C/S architecture, the architecture is suitable for the case that the front end (i.e. the client) needs to process more complex and heavy data analysis and processing tasks. Also because of this feature, data is distributed between the client and the server, and it is conceivable that the data in both places must be kept synchronized in order to ensure accurate operation of the system.
The traditional way of maintaining two-place data synchronization focuses on "real-time" synchronization. Although the method ensures the accurate operation of the system, the method puts high requirements on the communication quality, the bandwidth and the network speed between two places, and under the condition that some clients need to process a large amount of data and have strict requirements on timeliness, if the network quality does not meet the requirements, the defects of the traditional real-time synchronization mode are obvious. As shown in fig. 1: in the traditional real-time synchronization process of the client and the server, the client needs to interact with the server once for each operation. If n operations are required to complete a stage of data analysis and processing, then n interactions are required for the entire stage. As shown in fig. 2: if a data analysis task contains m identical test stages, it is calculated that the whole task needs n × m interactions.
Disclosure of Invention
In order to overcome the defects caused by the traditional real-time synchronization mode, the invention provides a client and server data interaction method based on selective caching and synchronization, and the high efficiency and accuracy of data interaction are improved by adopting a selective caching mode. The method abandons the mode of synchronizing all data interacted between the client and the server in real time, selects the data with low timeliness requirement of the server and strict timeliness requirement of the client to the server to be cached in the local of the client, and synchronizes the processing result to the server after the whole processing stage is finished.
In order to achieve the purpose, the invention adopts the following technical scheme:
a client and server data interaction method based on selective caching and synchronization comprises the following steps:
1) the whole transmission process of the client and the server is used as a processing task, and the task is divided into a plurality of same or similar processing stages;
2) the client-side firstly interacts with the server once to obtain basic data required by a plurality of processing stages under one processing task, and the data respectively provide functions similar to parameters for each processing stage of the client-side; the data are cached in the local client after being acquired, and the use of each processing stage is waited;
3) the client only needs to call the data in the local cache each time to complete the data analysis and processing of each processing stage, the processing result is continuously cached in the local, and then the data analysis and processing of the next stage are continuously carried out;
4) each processing stage produces a series of detail data which are stored locally at the client as a whole, with respect to which the server is not concerned, and which are hidden from view;
5) the detailed data of each processing stage and the final result are stored and managed in a proper way at the client side; if necessary, the managed complete set of detail data and data processing results can be viewed or exported at the client;
6) and the other thread in the client detects the network environment, does not perform any operation if the network environment is not good, and informs the client to synchronize the results of each processing stage cached in the local to the server if the network environment is good.
Under the directory of the client, each task has a plurality of data processing stages, and the processing stages are the minimum units of cache and synchronization; each processing stage consists of a stage processing result and processing details, and the processing details are different along with different application scenes and also generate different whole sets of detail data; the processing procedure of the client to each processing stage is as follows:
1) the client reads all data required by a single processing stage from the local cache, and in order to reduce the information interaction amount between the client and the server, the group of data needs to be simplified as much as possible, and the data required by the single processing stage are assumed to be S1, S2, S3, … and Sm;
2) after the group of data is obtained, the data is substituted into a local single processing stage process like real parameters of a function; as this process progresses, a series of derivative data also ensues, P1, P2, P3, …, Px, Q1, Q2, Q3, … Qy, … …;
3) until the whole processing stage is finished, a final processing result W1, W2, W3 … … Wn and a series of derivative data P1, P2, P3, …, Px, Q1, Q2, Q3, … Qy and … … which are generated concomitantly in the process are obtained;
4) the processing result and a series of derived data generated above will be stored locally in a suitable manner, and they will be embedded in a series of processing stages in a certain processing task in a whole according to a given sequence, and wait for the client to synchronize its result to the server, or wait for the client to open the stage in some special cases for querying specific processing details.
The client synchronizes the results cached in each local processing stage to the server, and the receiving process of the server to the data of each stage specifically comprises the following steps:
1) when the whole interaction process starts, the first interaction of the client and the server synchronizes data required by the whole processing process to the local part of the client;
2) then, the server side waits for the client side to send the processing results of all the stages;
3) the server arranges the received series of results according to a specific sequence to obtain a final result of the whole data processing.
The method is characterized by the following aspects:
"non-strict real-time" of client and server data transmission: the traditional real-time data synchronization requires that all data can be synchronized as quickly as possible, but in the method, only necessary data is selected and synchronization is carried out under the condition that the network is proper, the network condition is not good, and data synchronization cannot be carried out under the condition that the network is disconnected under the extreme condition.
Selectivity of caching and synchronization: when the timeliness requirement of the client on some data is high, and the timeliness requirement of the server on the data is not high, the data is cached in the local of the client. And after the client finishes one stage of data processing, synchronizing the processing result with the server.
Judging the network environment by the client: and if the network environment is suitable for transmitting data, transmitting the data, and otherwise, continuously caching the data in the local.
Management of the client on the cached local data: the client stores the data cached locally reasonably and orderly according to time or other standards. The operator can view or derive the data details at any time.
Hiding data processing details which are not concerned by a client side to a server side: the client is mainly used for processing data, the processing result is uploaded after one processing stage is finished, and details generated in the processing process hide the server.
Compared with the prior art, the invention has the following advantages:
the method selectively caches and synchronizes so that only necessary data is transmitted between the client and the server, and derivative data generated by the client in the data processing and analyzing process is not transmitted, thereby reducing the data amount transmitted between the client and the server and saving the bandwidth. The non-strict real-time of the interaction process reduces the requirement on the network quality. The single processing stage is carried out locally at the client, and the required data is buffered locally, so that the requirement of the system on data timeliness is met, and errors in the local processing process caused by network delay are avoided. The processing details are cached in the client, so that the storage pressure of the server is reduced, local checking at the client is allowed, and the independence of the client is improved.
Drawings
Fig. 1 shows a data interaction process of a single processing stage in a conventional real-time manner.
Fig. 2 is a data interaction process of a plurality of processing stages in a conventional real-time manner.
Fig. 3 is an interaction process of a client and a server under the selective caching and synchronization method.
Fig. 4 is a process of a client for a single processing stage.
Fig. 5 is a process of receiving data of each phase by the server.
Detailed Description
The following further describes a specific embodiment of the present invention with reference to the drawings.
As shown in fig. 3, a client and server data interaction method based on selective caching and synchronization includes the following steps:
1) the whole transmission process of the client and the server is used as a processing task, and the task is divided into a plurality of same or similar processing stages;
2) the client-side firstly interacts with the server once to obtain basic data required by a plurality of processing stages under one processing task, and the data respectively provide functions similar to parameters for each processing stage of the client-side; the data are cached in the local client after being acquired, and the use of each processing stage is waited;
3) the client only needs to call the data in the local cache each time to complete the data analysis and processing of each processing stage, the processing result is continuously cached in the local, and then the data analysis and processing of the next stage are continuously carried out;
4) each processing stage produces a series of detail data which are stored locally at the client as a whole, with respect to which the server is not concerned, and which are hidden from view;
5) the detailed data of each processing stage and the final result are stored and managed in a proper way at the client side; if necessary, the managed complete set of detail data and data processing results can be viewed or exported at the client;
6) and the other thread in the client detects the network environment, does not perform any operation if the network environment is not good, and informs the client to synchronize the results of each processing stage cached in the local to the server if the network environment is good.
As shown in fig. 4, under the directory of the client, there are several data processing stages under each task, and these processing stages are the minimum units of caching and synchronization; each processing stage consists of a stage processing result and processing details, and the processing details are different along with different application scenes and also generate different whole sets of detail data; the processing procedure of the client to each processing stage is as follows:
1) the client reads all data required by a single processing stage from the local cache, and in order to reduce the information interaction amount between the client and the server, the group of data needs to be simplified as much as possible, and the data required by the single processing stage are assumed to be S1, S2, S3, … and Sm;
2) after the group of data is obtained, the data is substituted into a local single processing stage process like real parameters of a function; as this process progresses, a series of derivative data also ensues, P1, P2, P3, …, Px, Q1, Q2, Q3, … Qy, … …;
3) until the whole processing stage is finished, a final processing result W1, W2, W3 … … Wn and a series of derivative data P1, P2, P3, …, Px, Q1, Q2, Q3, … Qy and … … which are generated concomitantly in the process are obtained;
4) the processing result and a series of derived data generated above will be stored locally in a suitable manner, and they will be embedded in a series of processing stages in a certain processing task in a whole according to a given sequence, and wait for the client to synchronize its result to the server, or wait for the client to open the stage in some special cases for querying specific processing details.
As shown in fig. 5, the client synchronizes the results of each processing stage cached locally to the server, and the receiving process of the server for each stage data specifically includes:
1) when the whole interaction process starts, the first interaction of the client and the server synchronizes data required by the whole processing process to the local part of the client;
2) then, the server side waits for the client side to send the processing results of all the stages;
3) the server arranges the received series of results according to a specific sequence to obtain a final result of the whole data processing.
Claims (3)
1. A client and server data interaction method based on selective caching and synchronization is characterized by comprising the following steps:
1) taking the complete data interaction process of the client and the server as a processing task, and dividing the task into a plurality of same or similar processing stages;
2) the client-side firstly interacts with the server once to obtain basic data required by a plurality of processing stages under one processing task, and the data respectively provide functions similar to parameters for each processing stage of the client-side; the data are cached in the local client after being acquired, and the use of each processing stage is waited;
3) the client only needs to call the data in the local cache each time to complete the data analysis and processing of each processing stage, the processing result is continuously cached in the local, and then the data analysis and processing of the next stage are continuously carried out;
4) each processing stage will generate a series of detail data, which will be stored locally at the client, respectively, according to the different stages, and the server is not concerned about the detail data, and the server is hidden;
5) the detailed data of each processing stage and the final result are stored and managed in a proper way at the client side; if necessary, the detail data and data processing results of the managed stages can be viewed or exported at the client;
6) the other thread in the client detects the network environment, if the network environment is found to be poor, no operation is carried out, and if the network environment is found to be good, the client is informed to synchronize the results of each processing stage cached in the local to the server;
the selectivity of the cache and the synchronization is as follows: when the timeliness requirement of the client on some data is high, and the timeliness requirement of the server on the data is not high, the data is cached in the local of the client.
2. The method for client-server data interaction based on selective caching and synchronization of claim 1, wherein under the directory of the client, there are several data processing stages under each task, and these processing stages are the minimum units for caching and synchronization; each processing stage consists of a stage processing result and processing details, and the processing details are different along with different application scenes and also generate different whole sets of detail data; the processing procedure of the client to each processing stage is as follows:
1) the client reads all data required by a single processing stage from the local cache, and in order to reduce the information interaction amount between the client and the server, the group of data needs to be simplified as much as possible, and the data required by the single processing stage are assumed to be S1, S2, S3, … and Sm;
2) after the group of data is obtained, the data is substituted into a local single processing stage process like real parameters of a function; as this process progresses, a series of derivative data also ensues, P1, P2, P3, …, Px, Q1, Q2, Q3, … Qy, … …;
3) until the whole processing stage is finished, a final processing result W1, W2, W3 … … Wn and a series of derivative data P1, P2, P3, …, Px, Q1, Q2, Q3, … Qy and … … which are generated concomitantly in the process are obtained;
4) the processing result and a series of derived data generated above will be stored locally in a suitable manner, and they will be embedded in the corresponding processing stage in the corresponding processing task in a whole according to a predetermined sequence, and wait for the client to open the stage in some special cases for inquiring specific processing details.
3. The method for client-server data interaction based on selective caching and synchronization according to claim 1, wherein the client synchronizes the results of the processing stages cached locally to the server, and the receiving process of the server for the data of each stage specifically comprises:
1) when the whole interactive process starts, when the service function starts to run, the first interaction of the client and the server synchronizes data required by the processing process to the local part of the client;
2) then, the server side waits for the client side to send the processing results of all the stages;
3) the server arranges the received series of results according to a specific sequence to obtain a final result of the whole data processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710076363.7A CN106921734B (en) | 2017-02-13 | 2017-02-13 | Client and server data interaction method based on selective caching and synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710076363.7A CN106921734B (en) | 2017-02-13 | 2017-02-13 | Client and server data interaction method based on selective caching and synchronization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106921734A CN106921734A (en) | 2017-07-04 |
| CN106921734B true CN106921734B (en) | 2020-04-03 |
Family
ID=59453555
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710076363.7A Active CN106921734B (en) | 2017-02-13 | 2017-02-13 | Client and server data interaction method based on selective caching and synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106921734B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107844363B (en) * | 2017-10-27 | 2020-08-28 | 东软集团股份有限公司 | Business transaction processing method, device, storage medium and equipment |
| CN108494859B (en) * | 2018-03-28 | 2020-07-14 | 烽火通信科技股份有限公司 | Network data synchronization method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101374158A (en) * | 2007-08-24 | 2009-02-25 | 国际商业机器公司 | Selectively delivering cached content or processed content to clients based upon a result completed percentage |
| CN101421726A (en) * | 2004-05-20 | 2009-04-29 | Bea系统公司 | Occasionally-connected application server |
| CN102227901A (en) * | 2008-09-29 | 2011-10-26 | 苹果公司 | Trickle sync protocol |
| CN102291467A (en) * | 2011-09-15 | 2011-12-21 | 电子科技大学 | Communication platform and method suitable for private cloud environment |
| CN104281476A (en) * | 2014-10-13 | 2015-01-14 | 中国外汇交易中心 | Data download method and data download manager for computer system |
| CN104796498A (en) * | 2015-05-19 | 2015-07-22 | 郑州悉知信息技术有限公司 | Data processing method, device and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020023127A1 (en) * | 2000-08-15 | 2002-02-21 | Sabeti Roya Rezvani | Asynchronous hyperlink object |
| US20140289428A1 (en) * | 2013-03-20 | 2014-09-25 | Microsoft Corporation | Dynamic Intervals for Synchronizing Data |
-
2017
- 2017-02-13 CN CN201710076363.7A patent/CN106921734B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101421726A (en) * | 2004-05-20 | 2009-04-29 | Bea系统公司 | Occasionally-connected application server |
| CN101374158A (en) * | 2007-08-24 | 2009-02-25 | 国际商业机器公司 | Selectively delivering cached content or processed content to clients based upon a result completed percentage |
| CN102227901A (en) * | 2008-09-29 | 2011-10-26 | 苹果公司 | Trickle sync protocol |
| CN102291467A (en) * | 2011-09-15 | 2011-12-21 | 电子科技大学 | Communication platform and method suitable for private cloud environment |
| CN104281476A (en) * | 2014-10-13 | 2015-01-14 | 中国外汇交易中心 | Data download method and data download manager for computer system |
| CN104796498A (en) * | 2015-05-19 | 2015-07-22 | 郑州悉知信息技术有限公司 | Data processing method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106921734A (en) | 2017-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11064012B2 (en) | File downloading method, apparatus, and non-transitory computer-readable storage medium | |
| CN108737467B (en) | Server log viewing method, device and system | |
| US8983902B2 (en) | Transparent caching of configuration data | |
| EP3063925B1 (en) | Synchronizing event history for multiple clients | |
| CN111800443B (en) | Data processing system and method, device and electronic equipment | |
| US20150331944A1 (en) | Method, apparatus and video monitoring system for providing monitoring video information | |
| CN101815087A (en) | Method for caching information of mobile communication terminal | |
| CN108259533B (en) | Data transmission method and device | |
| JP2007011928A5 (en) | ||
| CN106921734B (en) | Client and server data interaction method based on selective caching and synchronization | |
| CN105791401B (en) | Client and server-side data interactive method, system under net and off-network state | |
| CN104866339B (en) | Distributed persistence management method, system and the device of FOTA data | |
| US11588890B2 (en) | System, method and apparatus having a redundant cluster for processing data | |
| CN106708826A (en) | Data processing method and apparatus, and data query method and apparatus | |
| CN111209467A (en) | Data real-time query system under multi-concurrency multi-channel environment | |
| CN102026228B (en) | Statistical method and equipment for communication network performance data | |
| CN107046551B (en) | A kind of request of data, update method and related device | |
| CN109308287A (en) | A kind of partitions of file storage method, terminal and medium based on block chain | |
| CN113779094B (en) | Batch-flow-integration-based data processing method and device, computer equipment and medium | |
| CN105338260B (en) | Signal source switch method and system | |
| CN105302651A (en) | Method and apparatus for supplier and user to remotely and synchronously browse commodity | |
| CN114584808A (en) | Video stream acquisition method, device, system, equipment and medium | |
| CN110795328A (en) | Interface testing method and device | |
| AU2014274826A1 (en) | System and method for providing advertising consistency | |
| CN103546433A (en) | Information synchronization system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |