CN110191168A - Processing method, device, computer equipment and the storage medium of online business datum - Google Patents
Processing method, device, computer equipment and the storage medium of online business datum Download PDFInfo
- Publication number
- CN110191168A CN110191168A CN201910432966.5A CN201910432966A CN110191168A CN 110191168 A CN110191168 A CN 110191168A CN 201910432966 A CN201910432966 A CN 201910432966A CN 110191168 A CN110191168 A CN 110191168A
- Authority
- CN
- China
- Prior art keywords
- business datum
- request
- data
- written
- local cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Abstract
The invention discloses a kind of processing method of online data and its relevant devices.Wherein, this method comprises: receiving user's request, and the access address in user's request is determined；When user's request is data read request, the corresponding business datum of access address is obtained using synchronous interaction mode；When user's request is data write request, obtain business datum to be written, and it uses asynchronous interactive mode to be written into business datum and local cache is written to batch writing mode, and be written to the synchronizing traffic data to be written of local cache to network database servers.Thousands of secondary write requests are merged into single request, can remove unnecessary network write request, reduce unnecessary error checking, network interaction bring delay problem is greatly lowered by this method in such a way that local cache batch is written.
Description
Technical field
The present invention relates to data processing field more particularly to a kind of processing methods of online business datum, device, computer
Equipment and storage medium.
Background technique
With the development of development of Mobile Internet technology, for the service product of big flow, can exist and apply flexibly up to ten million days
Family, service line need to dig for according to the real-time behavioural information (such as search behavior, browsing behavior) of user in conjunction with history in real time
Pick customized information (such as point of interest, educational background, permanent residence) carries out pushing information, news, problem, advertisement etc. to user, to mention
Rise the sense of ownership and user's viscosity of user.
In the related technology, it is buffered in offline storage service using by the customized information of user, even if using advanced
Storage service (such as Mola2/3 or RDict) and solid state hard disk low latency hardware, extraction time, which not can guarantee in 10s, yet (deposits
" update " delay and network delay inside storage service).
The problem of mentioning in the related technology, key point are all caused by network interaction.Under the premise of aging performance allows,
If can be reduced or eliminate network interaction, it will alleviate or eliminate problem above to a certain extent.Therefore, how network is reduced or eliminated
Interaction, will alleviate to a certain extent or eliminate problem above, and have become urgent problem to be solved.
Summary of the invention
The present invention is directed to solve at least some of the technical problems in related technologies.
For this purpose, this method passes through this object of the present invention is to provide a kind of processing method of online business datum
Thousands of secondary write requests are merged into single request, can remove unnecessary network and write by the mode of ground caching batch write-in
Request, reduces unnecessary error checking, network interaction bring delay problem is greatly lowered.
Second object of the present invention is to propose a kind of processing unit of online data.
Third object of the present invention is to propose a kind of computer equipment.
Fourth object of the present invention is to propose a kind of computer readable storage medium.
In order to achieve the above objectives, the processing method for the online business datum that first aspect present invention embodiment proposes, it is described
Method includes: to receive user's request, and determine the access address in user's request；When user request is reading data
When request, the corresponding business datum of the access address is obtained using synchronous interaction mode；When the user requests to write for data
When entering to request, business datum to be written is obtained, and uses asynchronous interactive mode by the business datum to be written to be written in batches
Mode is written to local cache, and is written to the synchronizing traffic data to be written of the local cache to network data base
Server.
The processing method of online business datum according to an embodiment of the present invention can receive user's request, and determine that user asks
It is corresponding to obtain access address using synchronous interaction mode when user's request is data read request for access address in asking
Business datum；When user's request is data write request, business datum to be written is obtained, and will be to using asynchronous interactive mode
Business datum is written, local cache is written to batch writing mode, and the business datum to be written for being written to local cache is same
It walks to network database servers.This method closes thousands of secondary write requests in such a way that local cache batch is written
And be single request, unnecessary network write request can be removed, unnecessary error checking is reduced, network interaction is greatly lowered
Bring delay problem.
In order to achieve the above objectives, the processing unit for the online business datum that second aspect of the present invention embodiment proposes, comprising:
Request receiving module for receiving user's request, and determines the access address in user's request；Data read module is used
In when user request is data read request, the corresponding business number of the access address is obtained using synchronous interaction mode
According to；Data write. module, for obtaining business datum to be written, and use when user request is data write request
The business datum to be written is written to local cache with batch writing mode by asynchronous interactive mode, and is written to described
The synchronizing traffic data to be written of ground caching is to network database servers.
The processing unit of online business datum according to an embodiment of the present invention can receive user's request, and determine that user asks
It is corresponding to obtain access address using synchronous interaction mode when user's request is data read request for access address in asking
Business datum；When user's request is data write request, business datum to be written is obtained, and will be to using asynchronous interactive mode
Business datum is written, local cache is written to batch writing mode, and the business datum to be written for being written to local cache is same
It walks to network database servers.Thousands of secondary write requests can be closed in such a way that local cache batch is written as a result,
And be single request, unnecessary network write request can be removed, unnecessary error checking is reduced, network interaction is greatly lowered
Bring delay problem.
In order to achieve the above objectives, the computer equipment that third aspect present invention embodiment proposes, comprising: memory, processing
Device and it is stored in the computer program that can be run on the memory and on the processor, the processor executes the meter
When calculation machine program, the processing method of online business datum described in first aspect present invention embodiment is realized.
In order to achieve the above objectives, the computer readable storage medium that fourth aspect present invention embodiment proposes, stores thereon
There is computer program, online industry described in first aspect present invention embodiment is realized when the computer program is executed by processor
The processing method for data of being engaged in.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments
Obviously and it is readily appreciated that, in which:
Fig. 1 is the flow chart of the processing method of online business datum according to an embodiment of the invention.
Fig. 2 is the flow chart of the processing method of provided online business datum accord to a specific embodiment of that present invention.
Fig. 3 is the read access address corresponding business datum according to an embodiment of the present invention from network database servers
And it is saved in the flow chart in local cache.
Fig. 4 is the structural schematic diagram of overall system structure according to an embodiment of the present invention.
Fig. 5 be according to the user of the embodiment of the present invention request be the pure read request method of data exemplary diagram.
Fig. 6 be according to the user of the embodiment of the present invention request be the pure write request method of data exemplary diagram.
Fig. 7 be according to the user of the embodiment of the present invention request be reading and writing data request exemplary diagram.
Fig. 8 is the structural schematic diagram of online operation system according to an embodiment of the present invention.
Fig. 9 is the structural schematic diagram of the processing unit of online business datum according to an embodiment of the invention.
Figure 10 is the structural schematic diagram of the processing unit of online business datum according to an embodiment of the invention.
Figure 11 is the structural schematic diagram of computer equipment according to an embodiment of the invention.
Specific embodiment
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to is used to explain the present invention, and is not considered as limiting the invention.
Below with reference to the accompanying drawings describe the processing method of the online business datum of the embodiment of the present invention, device, computer equipment,
Computer readable storage medium.
Fig. 1 is the flow chart of the processing method of online business datum according to an embodiment of the invention.It needs to illustrate
It is that the processing method of the online business datum of the embodiment of the present invention can be applied to the place of the online business datum of the embodiment of the present invention
Device is managed, which can be configured in computer equipment.
As shown in Figure 1, the processing method of the online business datum may include:
S110 receives user's request, and determines the access address in user's request.
It should be noted that user's request can be the access request of user, for example, user request access to shopping website,
Social network sites, gaming platform or search platform etc..
In an embodiment of the present invention, when user is requested access to by terminal device sending, it can be received and be used by server
Family request, wherein user's request includes the access address in user's request.
For example, when user open a terminal the browser of equipment access when, i.e., send access request to server,
Server can receive user access request, wherein each access request has corresponding access address.
S120 obtains the corresponding industry of access address using synchronous interaction mode when user's request is data read request
Business data.
It should be noted that synchronous interaction mode refers to that server after receiving user's request, need to carry out the request
Response, and result is returned, next request is then received again.
In an embodiment of the present invention, when receiving user's request, it can determine that user requests according to user's request content
It whether is data read request, for example, user's request content is " content for checking novel A ", i.e. user's request is reading data
Request obtains the corresponding business datum of access address using synchronous interaction mode when user's request is data read request,
In, business datum is the data generated for operation system.
For example, server is when receiving user's request, it is known that user's request content is " to check the correlation of enterprise A
Introduce ", it may be determined that user's request is data read request, obtains " related introduction for checking enterprise A " using synchronous interaction mode
The corresponding type of business in address, place city, registered capital simultaneously return.
Buffer update mechanism based on business service, optionally, in one embodiment of the invention, when user's request is
When data read request, the corresponding business datum of the access address is obtained using synchronous interaction mode, i.e., when user's request is
When data read request, it can be inquired in local cache and whether there is the corresponding business datum of access address, and if it exists, then from this
Business datum is read in ground caching, if it does not exist, then the corresponding business number in read access address from network database servers
According to and be saved in local cache.Specific implementation process can be found in the description of subsequent embodiment.
S130 obtains business datum to be written, and use asynchronous interactive mode when user's request is data write request
It is written into business datum and local cache is written to batch writing mode, and be written to the business number to be written of local cache
According to being synchronized to network database servers.
It should be noted that asynchronous interactive mode refers to server after receiving user's request, without waiting for server
Response, next request can be received again.
In an embodiment of the present invention, it when receiving multiple users request, can be determined according to multiple user's request contents
Whether multiple user's requests are data write request, when multiple users request is data write request, can be obtained to be written
Business datum, and use asynchronous interactive mode to be written into business datum and local cache is written to batch writing mode, and will
The synchronizing traffic data to be written of local cache is written to network database servers.
For example, server can obtain the industry to be written in each user's request when receiving multiple users request
Business data, and asynchronous interactive mode is used to be written to the business datum to be written in each user request with batch writing mode
Local cache, and the synchronizing traffic data to be written of local cache is written to network database servers.Pass through this as a result,
Ground caches batch writing mode, and thousands of secondary write requests are merged into single request, can remove unnecessary network and writes and ask
It asks, network interaction bring delay problem is greatly lowered.
In order to which the occupancy of on-line storage resource is greatly lowered, in an embodiment of the present invention, it is being written into business number
According to before being written to local cache, business datum to be written is compressed, and compressed business datum to be written is written
To local cache.
That is, obtaining business datum to be written, and to business to be written when user's request is data write request
Data are compressed, and are used asynchronous interactive mode to be written into business datum and be written to local cache with batch writing mode,
And the synchronizing traffic data to be written of local cache is written to network database servers.
The processing method of online business datum according to an embodiment of the present invention can receive user's request, and determine that user asks
It is corresponding to obtain access address using synchronous interaction mode when user's request is data read request for access address in asking
Business datum；When user's request is data write request, business datum to be written is obtained, and will be to using asynchronous interactive mode
Business datum is written, local cache is written to batch writing mode, and the business datum to be written for being written to local cache is same
It walks to network database servers.This method closes thousands of secondary write requests in such a way that local cache batch is written
And be single request, unnecessary network write request can be removed, unnecessary error checking is reduced, network interaction is greatly lowered
Bring delay problem.
Fig. 2 is the flow chart of the processing method of provided online business datum accord to a specific embodiment of that present invention.
As shown in Fig. 2, the processing method of the online business datum may include:
S210 receives user's request, and determines the access address in user's request.
It should be noted that user's request can be the access request of user, for example, user request access to shopping website,
Social network sites, gaming platform or search platform etc..
In an embodiment of the present invention, when user is requested access to by terminal device sending, it can be received and be used by server
Family request, wherein user's request includes the access address in user's request.
For example, when user open a terminal the browser of equipment access when, i.e., send access request to server,
Server can receive user access request, wherein each access request has corresponding access address.
S220, when user's request is data read request, inquiry is corresponding with the presence or absence of access address in local cache
Business datum.
In an embodiment of the present invention, when receiving user's request, it can determine that user requests according to user's request content
It whether is data read request, when user's request is data read request, inquiry whether there is access in local cache
The corresponding business datum in location.
For example, when receiving user's request, it is known that user's request content is " related data for checking star A ",
I.e. user's request is data read request, and inquiry whether there is " related data for checking star A " address pair in local cache
The brief introduction answered, representative works.
S230, if reading business from local cache there are the corresponding business datum of access address in local cache
Data.
For example, there is the corresponding brief introduction in " related data for checking star A " address, representative works in local cache
Etc. business datums, the business datums such as brief introduction, representative works can be read directly from local cache.
S240, if the corresponding business datum of access address is not present in local cache, from network database servers
The corresponding business datum in upper read access address is simultaneously saved in local cache.
In order to realize that the individual business of real-time rank calculates, in an embodiment of the present invention, it is not present in local cache
The corresponding business datum of access address, it may be determined that need to read data, need to read by detection network database servers reading
Whether the operation for evidence of fetching successfully determines the corresponding business datum in read access address from network database servers and protects
It is stored in local cache.Concrete implementation process is as follows:
Fig. 3 is the read access address corresponding business datum according to an embodiment of the present invention from network database servers
And it is saved in the flow chart in local cache.As shown in figure 3, the corresponding industry in read access address from network database servers
The business data and specific implementation process that is saved in local cache may include following steps:
The corresponding business datum of access address is determined as needing to read data by S310.
In embodiments of the present invention, the corresponding business datum of access address is not present in local cache, by access address
Corresponding business datum is determined as needing to read data.
That is, inquiry is accessed less than read requests in local cache when user's request is data read request
The corresponding business datum of access address can be determined as needing to read data by the corresponding business datum in address.
For example, when user's request is data read request, it is known that the request content of user is " to check school A
Data " inquires the business of " brief introduction of school A " corresponding less than " data for checking school A " access address in local cache
The business datum of " brief introduction of school A " can be determined as needing to read data by data.
The code key KEY for needing to read data is put into high priority and reads queue to ask to network database servers by S320
It asks and needs to read data.
Wherein, in an embodiment of the present invention, database refers to Key-Value (KV, storing data library).
It should be noted that the code key KEY for reading data is similar to index, each reads data and corresponds to unique KEY.
That is, the index for needing to read data, which is put into high priority, reads queue, wherein every data is corresponding unique
Index, by corresponding data known to index, to need to read data to network database servers request.
For example, the index for reading data is key1, and the index key1 of the reading data is put into high priority and reads team
Column, it is known that the corresponding data of key1 are " data 1 ", to need to read " data 1 " to network database servers request.
S330 is detected to whether the operation that network database servers request reading needs to read data succeeds.
In an embodiment of the present invention, the index for needing to read data is put into high priority and reads queue, with to network number
After needing to read data according to the request of library server, whether there can be returned data by detection function detection service device, with judgement
Whether the operation for needing to read data to network database servers request reading succeeds.
For example, there is returned data by detection function detection service device, then request to read to network database servers
The operation success for needing to read data is taken, does not have returned data by detection function detection service device, is then taken to network data base
The operation failure for needing to read data is read in business device request.
S340, network database servers request read the operation success for needing to read data, then obtain network data base
The corresponding business datum of access address that server returns.
That is, local cache detects that server has returned data by detection function, so as to obtain server
The corresponding business datum of the access address of return.
S350, the corresponding business datum of access address that network database servers are returned is with the write-in of batch writing mode
To local cache.
That is, the corresponding business datum of access address that the network database servers that will acquire return is with batch
Writing mode is written to local cache.
In order to further embody the stability of system, in one embodiment of the invention, detecting to network data
When server request reading in library needs to read the operation failure of data, the code key KEY for needing to read data is put into low priority
Read queue.
That is, local cache detects that server does not have returned data by detection function, will need to read data
Code key KEY be put into low priority read queue, wherein every data correspond to unique KEY, by KEY know corresponding data, and
The corresponding business datum of access address that network database servers return is obtained, later returns to network database servers
The corresponding business datum of access address is written to local cache with batch writing mode.
For example, the KEY for reading data is key2, and the index key2 of the reading data is put into low priority and reads team
Column, it is known that the corresponding data of key2 are " data 2 ", and obtain the corresponding business of access address of network database servers return
Data " data 2 ", the corresponding business datum of access address " data 2 " for later returning to network database servers to write in batches
Enter mode and is written to local cache.
In order to guarantee the safety of internal storage data, the occupancy of storage resource is saved, the data of real-time update local cache are optional
Ground determines the timeliness for the business datum being present in local cache in an embodiment of the present invention, eliminates timeliness in local cache
More than the business datum of preset duration.
That is, the corresponding business datum of access address that network database servers return is written to local slow
Deposit, can the timeliness of business datum in real-time monitoring local cache will be in local cache and by the way that timeliness is arranged in local cache
Timeliness carries out superseded more than the business datum of preset duration.Wherein, in an embodiment of the present invention, the timeliness of the business datum
It can be determined according to the type of the business datum, for example, it is assumed that business datum is historical data, then can be determined according to the time
The timeliness of the business datum, for example, can detect and eliminate the historical data that timeliness in local cache was more than 1 week.
S250 obtains business datum to be written, and use asynchronous interactive mode when user's request is data write request
It is written into business datum and local cache is written to batch writing mode, and be written to the business number to be written of local cache
According to being synchronized to network database servers.
It should be noted that asynchronous interactive mode refers to server after receiving user's request, without waiting for server
Response, next request can be received again.
In an embodiment of the present invention, it when receiving multiple users request, can be determined according to multiple user's request contents
Whether multiple user's requests are data write request, when multiple users request is data write request, can be obtained to be written
Business datum, and use asynchronous interactive mode to be written into business datum and local cache is written to batch writing mode, and will
The synchronizing traffic data to be written of local cache is written to network database servers.
For example, server can obtain the industry to be written in each user's request when receiving multiple users request
Business data, and asynchronous interactive mode is used to be written to the business datum to be written in each user request with batch writing mode
Local cache, and the synchronizing traffic data to be written of local cache is written to network database servers.Pass through this as a result,
Ground caches batch writing mode, and thousands of secondary write requests are merged into single request, can remove unnecessary network and writes and ask
It asks, network interaction bring delay problem is greatly lowered.
The processing method of online business datum according to an embodiment of the present invention can receive user's request, and determine that user asks
Access address in asking, when user's request is data read request, inquiry whether there is access address pair in local cache
The business datum answered then reads business datum there are the corresponding business datum of access address in local cache from local cache,
If it does not exist, then it from the corresponding business datum in read access address in network database servers and is saved in local cache；
When user's request is data write request, business datum to be written is obtained, and business is written into using asynchronous interactive mode
Data are written to local cache with batch writing mode, and are written to the synchronizing traffic data to be written of local cache to network
Database server.Thousands of secondary write requests are merged into single in such a way that local cache batch is written by this method
Request, can remove unnecessary network write request, reduce unnecessary error checking, when network interaction bring is greatly lowered
Prolong problem, while under the premise of most saving hardware resource, the realization real-time behavioural information of user is fast with characteristic of history information
Speed docking realizes that the personalized also error count of real-time rank is calculated.
Overall structure of the invention is more apparent from order to facilitate those skilled in the art, Fig. 4 is according to the present invention
The structural schematic diagram of the overall system structure of embodiment, as shown in figure 4, in an embodiment of the present invention, in user interface response
Operation interface comprising whole dblib is obtained data and is returned by the way of synchronous when user's request is data read request
It returns, wherein if data exist in cache, directly return；If it is not, the code key key for needing to read data is put into high preferential reading
Queue does not stop to check whether data are updated successfully, and returns after success, and the data of return are written to this with batch writing mode
Ground caching carries out processing and the thread interaction on backstage according to expiration policy by failure detection module, takes the shape of asynchronous interactive
Formula will read failure key and be put into low preferential reading queue, reads data by background synchronization, take settable batch read method,
Preferential selection high-priority queue；Data are written by background synchronization, that is, take settable batch wiring method.
Clearly understanding user in order to facilitate those skilled in the art to request is the pure read request of data, user's request is
The pure write request of data and user's request are the difference of reading and writing data request, and introducing lower user's request separately below is that the pure reading of data is asked
It asks, user's request is the pure write request of data and user's request is the method for reading and writing data request.
Fig. 5 be according to user of the embodiment of the present invention request be the pure read request method of data exemplary diagram.As shown in figure 5, with
User orientation server sends read requests, and determines the access address in user's read requests, after server receives the read requests
Backward reference address corresponding data, local cache are detected by thread function, and whether detection service device has returned data,
It then needs to update if any returned data, more new data is put into and is read in queue, updates local cache, sheet in the updated later
The data read before are returned in ground caching.
Fig. 6 be according to user of the embodiment of the present invention request be the pure write request method of data exemplary diagram.As shown in fig. 6, with
User orientation server sends write request, and determines the access address in user's write request, after server receives the write request
Backward reference address corresponding data, local cache will return to number and be put into write-in queue, update local cache later, in the updated
Local cache in by the data being written before return.
Fig. 7 be according to user of the embodiment of the present invention request be reading and writing data request exemplary diagram.As shown in fig. 7, of the invention
Embodiment can receive user's request, and determine the access address in user's request, when user's request is data read request,
It is inquired in local cache and whether there is the corresponding business datum of access address, there are the corresponding business of access address in local cache
Data then read business datum from local cache, if it does not exist, then the read access address pair from network database servers
The business datum answered simultaneously is saved in local cache；When user's request is data write request, business datum to be written is obtained,
And it uses asynchronous interactive mode to be written into business datum and local cache is written to batch writing mode, and be written to local
The synchronizing traffic data to be written of caching is to network database servers.
In an embodiment of the present invention, the processing method of online business datum can be applied to online operation system, and Fig. 8 is root
According to the structural schematic diagram of the online operation system of the embodiment of the present invention, as shown in figure 8, the online operation system includes: example mould
Block 810, adapter system module 820, Database Systems module 830, user's module 840 and decorative system module 850,
In:
Example module 810 is used to provide the DB example of different storage classes for user, and can be obtained by these DB examples
Take some interfaces of this storage medium, mainly be the interface provided in Adapter System.
Adapter system module 820 is used to provide the read-write interface of the DB of various different storage classes for user, at present all
It is the interface of KV type.
Database Systems module 830 realizes the original library DB in the process for Adapter System.
User's module 840 is used for the user using dblib, is not belonging to dblib；User in use, initial
After selecting storage class to be used when change, the difference of different storage classes will be ignored, is directly connect using what Adapter unitized
Mouthful.
Decorative system module 850 wraps up original class for increasing a modification class, the mode of package generally by
Using original object as the parameter of the constructed fuction of modification class.
Wherein, in an embodiment of the present invention, decorative system module 850 is by the processing dress of online business datum of the invention
Set realization.Due to the processing unit of online business datum provided in an embodiment of the present invention and above-mentioned several embodiments provide
The processing method of line service data is corresponding, therefore the embodiment of the processing method in online business datum is also applied for this reality
The processing unit for applying the online business datum of example offer, is not described in detail in the present embodiment.Fig. 9 is one according to the present invention
The structural schematic diagram of the processing unit of the online business datum of embodiment.As shown in figure 9, the processing unit of the online business datum
It include: request receiving module 910, data read module 920 and Data write. module 930, in which:
Request receiving module 910 determines the access address in user's request for receiving user's request.
Data read module 920 is used to obtain when user request is data read request using synchronous interaction mode
Take the corresponding business datum of the access address.
In one embodiment of the invention, the data read module 920 is specifically used for: inquiring in local cache is
It is no that there are the corresponding business datums of the access address；If it exists, then the business datum is read from the local cache；If
It is not present, then reads the corresponding business datum of the access address from network database servers and is saved in the local and is slow
In depositing.
In one embodiment of the invention, the data read module 920 is specifically used for: the access address is corresponding
Business datum be determined as needing to read data；By the code key KEY for needing to read data be put into high priority read queue with
It needs to read data to network database servers request is described；It detects to the network database servers and requests to read
Whether the operation for needing to read data succeeds；If so, obtaining the access that the network database servers return
The corresponding business datum in address；The corresponding business datum of the access address that the network database servers are returned is to criticize
Amount writing mode is written to the local cache.
In one embodiment of the invention, the data read module 920 is also used to: being detected to the network number
When requesting to read the operation failure for needing to read data according to library server, the code key KEY for needing to read data is put
Enter low priority and reads queue.
Data write. module 930 is used to obtain business datum to be written when user request is data write request,
And use asynchronous interactive mode that the business datum to be written is written to local cache with batch writing mode, and be written to
The synchronizing traffic data to be written of the local cache is to network database servers.
In one embodiment of the invention, the Data write. module 930 is also used to the business datum to be written
It is compressed, and uses asynchronous interactive mode that compressed business datum to be written is written to described with batch writing mode
Ground caching.
In one embodiment of the invention, as shown in Figure 10, the processing unit of the online business datum further include: number
According to superseded module 940, wherein in an embodiment of the present invention, data eliminate module 940 for determining that being present in the local delays
The timeliness of business datum in depositing, and eliminate the business datum that timeliness in the local cache is more than preset duration.
The processing unit of online business datum according to an embodiment of the present invention can receive user's request, and determine that user asks
It is corresponding to obtain access address using synchronous interaction mode when user's request is data read request for access address in asking
Business datum；When user's request is data write request, business datum to be written is obtained, and will be to using asynchronous interactive mode
Business datum is written, local cache is written to batch writing mode, and the business datum to be written for being written to local cache is same
It walks to network database servers.Thousands of secondary write requests can be closed in such a way that local cache batch is written as a result,
And be single request, unnecessary network write request can be removed, unnecessary error checking is reduced, network interaction is greatly lowered
Bring delay problem.
In order to realize above-described embodiment, the invention also provides a kind of computer equipment,
Figure 11 is the structural schematic diagram of computer equipment according to an embodiment of the invention.As shown in figure 11, the calculating
Machine equipment 1100 may include: memory 1110, processor 1120 and be stored on memory 1110 and can be in processor 1120
The computer program 1130 of upper operation when processor 1120 executes program, is realized described in any of the above-described embodiment of the present invention
The processing method of online business datum.
In order to realize above-described embodiment, the invention also provides a kind of computer readable storage mediums, are stored thereon with meter
Calculation machine program realizes the processing method of the present invention online business datum described in any of the above embodiments when program is executed by processor.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiment or examples.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
Any process described otherwise above or method description are construed as in flow chart or herein, and expression includes
It is one or more for realizing specific logical function or process the step of executable instruction code module, segment or portion
Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discussed suitable
Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, Lai Zhihang function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system, including the system of processor or other can be held from instruction
The instruction fetch of row system, device or equipment and the system executed instruction) it uses, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium ", which can be, any may include, stores, communicates, propagates or pass
Defeated program is for instruction execution system, device or equipment or the dress used in conjunction with these instruction execution systems, device or equipment
It sets.The more specific example (non-exhaustive list) of computer-readable medium include the following: there is the electricity of one or more wirings
Interconnecting piece (electronic device), portable computer diskette box (magnetic device), random access memory (RAM), read-only memory
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium can even is that the paper that can print described program on it or other are suitable
Medium, because can then be edited, be interpreted or when necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each section of the invention can be realized with hardware, software, firmware or their combination.Above-mentioned
In embodiment, software that multiple steps or method can be executed in memory and by suitable instruction execution system with storage
Or firmware is realized.It, and in another embodiment, can be under well known in the art for example, if realized with hardware
Any one of column technology or their combination are realized: having a logic gates for realizing logic function to data-signal
Discrete logic, with suitable combinational logic gate circuit specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are understood that realize all or part of step that above-described embodiment method carries
It suddenly is that relevant hardware can be instructed to complete by program, the program can store in a kind of computer-readable storage medium
In matter, which when being executed, includes the steps that one or a combination set of embodiment of the method.
It, can also be in addition, each functional unit in each embodiment of the present invention can integrate in a processing module
It is that each unit physically exists alone, can also be integrated in two or more units in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized and when sold or used as an independent product in the form of software function module, also can store in a computer
In read/write memory medium.
Storage medium mentioned above can be read-only memory, disk or CD etc..Although having been shown and retouching above
The embodiment of the present invention is stated, it is to be understood that above-described embodiment is exemplary, and should not be understood as to limit of the invention
System, those skilled in the art can be changed above-described embodiment, modify, replace and become within the scope of the invention
Type.
Claims (14)
1. a kind of processing method of online business datum characterized by comprising
User's request is received, and determines the access address in user's request；
When user request is data read request, the corresponding business of the access address is obtained using synchronous interaction mode
Data；
When user request is data write request, business datum to be written is obtained, and use asynchronous interactive mode by institute
It states business datum to be written and local cache is written to batch writing mode, and be written to the described to be written of the local cache
Enter synchronizing traffic data to network database servers.
2. the method according to claim 1, wherein described obtain the access address using synchronous interaction mode
Corresponding business datum, comprising:
Inquiry whether there is the corresponding business datum of the access address in local cache；
If it exists, then the business datum is read from the local cache；
If it does not exist, then the corresponding business datum of the access address is read from network database servers and is saved in described
In local cache.
3. according to the method described in claim 2, it is characterized in that, described read the access from network database servers
The corresponding business datum in address is simultaneously saved in the local cache, comprising:
The corresponding business datum of the access address is determined as needing to read data；
The code key KEY for needing to read data is put into high priority and reads queue to request to the network database servers
It is described to need to read data；
It detects to network database servers request and reads whether the operation for needing to read data succeeds；
If so, obtaining the corresponding business datum of the access address that the network database servers return；
The corresponding business datum of the access address that the network database servers are returned is with the write-in of batch writing mode
To the local cache.
4. according to the method described in claim 3, it is characterized by further comprising:
Detect request to read the operation failure for needing to read data to the network database servers when, will be described
The code key KEY for needing to read data is put into low priority and reads queue.
5. method according to claim 1 to 4, which is characterized in that further include:
Determine the timeliness for the business datum being present in the local cache；
Eliminate the business datum that timeliness in the local cache is more than preset duration.
6. the method according to claim 1, wherein being written to the local cache being written into business datum
Before, the method also includes:
The business datum to be written is compressed, and compressed business datum to be written is written to the local and is delayed
It deposits.
7. a kind of processing unit of online data characterized by comprising
Request receiving module for receiving user's request, and determines the access address in user's request；
Data read module, for being obtained using synchronous interaction mode described when user request is data read request
The corresponding business datum of access address；
Data write. module, for obtaining business datum to be written, and use when user request is data write request
The business datum to be written is written to local cache with batch writing mode by asynchronous interactive mode, and is written to described
The synchronizing traffic data to be written of ground caching is to network database servers.
8. device according to claim 7, which is characterized in that the data read module is specifically used for:
Inquiry whether there is the corresponding business datum of the access address in local cache；
If it exists, then the business datum is read from the local cache；
If it does not exist, then the corresponding business datum of the access address is read from network database servers and is saved in described
In local cache.
9. device according to claim 8, which is characterized in that the data read module is specifically used for:
The corresponding business datum of the access address is determined as needing to read data；
The code key KEY for needing to read data is put into high priority and reads queue to request to the network database servers
It is described to need to read data；
It detects to network database servers request and reads whether the operation for needing to read data succeeds；
If so, obtaining the corresponding business datum of the access address that the network database servers return；
The corresponding business datum of the access address that the network database servers are returned is with the write-in of batch writing mode
To the local cache.
10. device according to claim 9, which is characterized in that the data read module is also used to: detecting to institute
When stating the network database servers request reading operation failure for needing to read data, by the code for needing to read data
Key KEY is put into low priority and reads queue.
11. device according to any one of claims 7 to 10, which is characterized in that further include:
Data eliminate module, for determining the timeliness for the business datum being present in the local cache, and eliminate the local
Timeliness is more than the business datum of preset duration in caching.
12. device according to claim 7, which is characterized in that the Data write. module is also used to described to be written
Business datum is compressed, and uses asynchronous interactive mode by compressed business datum to be written with the write-in of batch writing mode
To the local cache.
13. a kind of computer equipment characterized by comprising memory, processor and be stored on the memory and can be
The computer program run on the processor, when the processor executes the computer program, realize as claim 1 to
The processing method of online business datum described in any one of 6.
14. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
The processing method such as online business datum described in any one of claims 1 to 6 is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910432966.5A CN110191168A (en) | 2019-05-23 | 2019-05-23 | Processing method, device, computer equipment and the storage medium of online business datum |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910432966.5A CN110191168A (en) | 2019-05-23 | 2019-05-23 | Processing method, device, computer equipment and the storage medium of online business datum |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110191168A true CN110191168A (en) | 2019-08-30 |
Family
ID=67717537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910432966.5A Pending CN110191168A (en) | 2019-05-23 | 2019-05-23 | Processing method, device, computer equipment and the storage medium of online business datum |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110191168A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110727523A (en) * | 2019-09-30 | 2020-01-24 | 歌尔股份有限公司 | Service request processing method, processing device and readable storage medium |
| CN110928656A (en) * | 2019-11-18 | 2020-03-27 | 浙江大搜车软件技术有限公司 | Service processing method, device, computer equipment and storage medium |
| CN111241455A (en) * | 2020-01-22 | 2020-06-05 | 北京字节跳动网络技术有限公司 | Data processing apparatus, computer device, and storage medium |
| CN111400406A (en) * | 2020-04-09 | 2020-07-10 | 上海东普信息科技有限公司 | Data processing method and device, electronic equipment and computer storage medium |
| CN111414389A (en) * | 2020-03-19 | 2020-07-14 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111831446A (en) * | 2020-07-15 | 2020-10-27 | 北京思特奇信息技术股份有限公司 | Data read-write method, device and medium based on Disproptor |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130031060A1 (en) * | 2000-02-07 | 2013-01-31 | Parallel Networks, Llc | Method and apparatus for content synchronization |
| CN104503710A (en) * | 2015-01-23 | 2015-04-08 | 福州瑞芯微电子有限公司 | Method and device for increasing writing speed of nand flash |
| CN105549905A (en) * | 2015-12-09 | 2016-05-04 | 上海理工大学 | Method for multiple virtual machines to access distributed object storage system |
| CN105791439A (en) * | 2016-04-29 | 2016-07-20 | 中国农业银行股份有限公司 | Data processing method, data receiving apparatus and data processing system |
| US20180097814A1 (en) * | 2000-02-07 | 2018-04-05 | Parallel Networks Llc | Method and Apparatus for Prioritization of Data Requests |
| CN108572970A (en) * | 2017-03-09 | 2018-09-25 | 腾讯科技（深圳）有限公司 | A kind of processing method and distributed processing system(DPS) of structural data |
-
2019
- 2019-05-23 CN CN201910432966.5A patent/CN110191168A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130031060A1 (en) * | 2000-02-07 | 2013-01-31 | Parallel Networks, Llc | Method and apparatus for content synchronization |
| US20180097814A1 (en) * | 2000-02-07 | 2018-04-05 | Parallel Networks Llc | Method and Apparatus for Prioritization of Data Requests |
| CN104503710A (en) * | 2015-01-23 | 2015-04-08 | 福州瑞芯微电子有限公司 | Method and device for increasing writing speed of nand flash |
| CN105549905A (en) * | 2015-12-09 | 2016-05-04 | 上海理工大学 | Method for multiple virtual machines to access distributed object storage system |
| CN105791439A (en) * | 2016-04-29 | 2016-07-20 | 中国农业银行股份有限公司 | Data processing method, data receiving apparatus and data processing system |
| CN108572970A (en) * | 2017-03-09 | 2018-09-25 | 腾讯科技（深圳）有限公司 | A kind of processing method and distributed processing system(DPS) of structural data |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110727523A (en) * | 2019-09-30 | 2020-01-24 | 歌尔股份有限公司 | Service request processing method, processing device and readable storage medium |
| CN110727523B (en) * | 2019-09-30 | 2023-08-29 | 歌尔股份有限公司 | Service request processing method, service request processing device and readable storage medium |
| CN110928656A (en) * | 2019-11-18 | 2020-03-27 | 浙江大搜车软件技术有限公司 | Service processing method, device, computer equipment and storage medium |
| CN110928656B (en) * | 2019-11-18 | 2023-02-28 | 浙江大搜车软件技术有限公司 | Service processing method, device, computer equipment and storage medium |
| CN111241455A (en) * | 2020-01-22 | 2020-06-05 | 北京字节跳动网络技术有限公司 | Data processing apparatus, computer device, and storage medium |
| CN111241455B (en) * | 2020-01-22 | 2023-08-25 | 抖音视界有限公司 | Data processing apparatus, computer device, and storage medium |
| CN111414389A (en) * | 2020-03-19 | 2020-07-14 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111414389B (en) * | 2020-03-19 | 2023-09-22 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111400406A (en) * | 2020-04-09 | 2020-07-10 | 上海东普信息科技有限公司 | Data processing method and device, electronic equipment and computer storage medium |
| CN111400406B (en) * | 2020-04-09 | 2023-05-12 | 上海东普信息科技有限公司 | Data processing method and device, electronic equipment and computer storage medium |
| CN111831446A (en) * | 2020-07-15 | 2020-10-27 | 北京思特奇信息技术股份有限公司 | Data read-write method, device and medium based on Disproptor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110191168A (en) | Processing method, device, computer equipment and the storage medium of online business datum | |
| JP6122199B2 (en) | System, method and storage medium for improving access to search results | |
| CN106649349B (en) | Data caching method, device and system for game application | |
| CN109947668B (en) | Method and device for storing data | |
| US8117358B2 (en) | Revenue management system and method utilizing database backup | |
| CN101557427A (en) | Method for providing diffluent information and realizing the diffluence of clients, system and server thereof | |
| CN110275983A (en) | The search method and device of traffic monitoring data | |
| CN102780603B (en) | Web traffic control method and device | |
| CN1234086C (en) | System and method for high speed buffer storage file information | |
| CN107092628B (en) | Time series data processing method and device | |
| CN103207882A (en) | Shop visiting data processing method and system | |
| CN111314238A (en) | Token management method and device, storage medium and electronic device | |
| CN107911799B (en) | Method for utilizing intelligent route | |
| CN112434015B (en) | Data storage method and device, electronic equipment and medium | |
| CN111966938B (en) | Configuration method and system for realizing loading speed improvement of front-end page of cloud platform | |
| US9405785B2 (en) | Online propagation of data updates | |
| CN108780458A (en) | A kind of page cache processing method, device and server | |
| CN113010549A (en) | Data processing method based on remote multi-active system, related equipment and storage medium | |
| CN110020305A (en) | Webpage loading method, device, computer equipment and storage medium | |
| CN104933045B (en) | Network information browsing method and network information browsing apparatus | |
| CN104281581B (en) | The monitoring method and system of the recommendation position content exposure amount of webpage | |
| CN106547751A (en) | Data cached update method and device | |
| CN107295059A (en) | The statistical system and method for service propelling amount | |
| CN108021464B (en) | Bottom-pocketing processing method and device for application response data | |
| US7571446B2 (en) | Server, computer system, object management method, server control method, computer program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211013 Address after: 100176 101, floor 1, building 1, yard 7, Ruihe West 2nd Road, Beijing Economic and Technological Development Zone, Daxing District, Beijing Applicant after: Apollo Zhilian (Beijing) Technology Co.,Ltd. Address before: 2 / F, baidu building, 10 Shangdi 10th Street, Haidian District, Beijing 100085 Applicant before: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190830 |