CN106777085A - A kind of data processing method, device and data query system - Google Patents
A kind of data processing method, device and data query system Download PDFInfo
- Publication number
- CN106777085A CN106777085A CN201611147798.8A CN201611147798A CN106777085A CN 106777085 A CN106777085 A CN 106777085A CN 201611147798 A CN201611147798 A CN 201611147798A CN 106777085 A CN106777085 A CN 106777085A
- Authority
- CN
- China
- Prior art keywords
- inquiry request
- target
- target data
- inquiry
- external cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Abstract
This application provides a kind of data processing method, device and data query system, when the first inquiry request of client is received, it is processed into be inquired about in outside caching system in the second inquiry request of outside caching system inquiry and based on the second inquiry request, if not inquiring the second inquiry request, target data corresponding with the second inquiry request is then obtained from target database, the target data is buffered in external cache system.Second inquiry request and corresponding target data can be buffered in external cache system by the application, because the buffer memory capacity of external cache system is generally larger or expansible, therefore, it can realize the caching to all Query Results, and, due to each shared external cache system of inquiry server, as long as this causes to be cached with target data in external cache system, any one inquiry server faster to client feedback target data corresponding with inquiry request, and can avoid the unnecessary wasting of resources.
Description
Technical field
Data query technique field of the present invention, more particularly to a kind of data processing method, device and data query system.
Background technology
Apache Kylin are the second level SQL query engines of big data quantity, and its principle is all possible by precomputation
In Hbase, parsing SQL is scanned from hbase and obtained again to obtain dimension and metric, then during inquiry for dimension combination storage
Access is according to return.
In Kylin SQL engines use Calcite carry out SQL parsings, optimization and part operator computing, due to
The calculating of Calcite is based entirely on internal memory, therefore, when an inquiry needs to obtain a large amount of records from hbase in Kylin
When, internal memory can be increasingly becoming bottleneck.Historical data is normally based in view of SQL query, and the most important spy of historical data
Property is immutableness, for the data for infrequently changing, caching be it is the most frequently used be also maximally effective improving performance method.
Kylin is not that each Query Result is cached, and for each inquiry, it can be total according to the record of the inquiry scan
Whether number judges whether to need buffered results more than given threshold, if the record sum of inquiry scan exceedes given threshold,
Cached.
Fig. 1 is referred to, Kylin frameworks of the prior art are shown, from figure 1 it appears that many in Kylin frameworks
Individual inquiry server is independent, and for each inquiry server, it can only be by data buffer storage in the internal memory of the machine, so
And the limited memory of the machine, it possibly cannot be cached to all Query Results, also, because each inquiry server will
Data buffer storage is in the internal memory of the machine, and each inquiry server is separate, and this can cause to inquire about server customer in response
Hold the speed of inquiry request sometimes fast and sometimes slow, and cause the unnecessary wasting of resources.
The content of the invention
In view of this, the invention provides a kind of data processing method, device and data query system, it is used to solve existing
By data buffer storage in the internal memory of the machine, and local memory is limited for data query server in technology, and it is right that it may lead to not
All Query Results are cached, also, the speed of inquiry server customer in response end inquiry request is sometimes fast and sometimes slow, and are caused
The problem of the unnecessary wasting of resources, its technical scheme is as follows：
A kind of data processing method, is applied to the inquiry server in data query system, and the inquiry server is set
There is the interface of connection external cache system, and be connected with the external cache system by the interface, the external cache system
Unite for caching query request and target data corresponding with the inquiry request；
Preferably, the data processing method includes：
When the first inquiry request from client is received, target query ginseng is extracted from first inquiry request
Number；
The target query parameter is processed into the second inquiry request by preset rules, and in the external cache system
Inquire about second inquiry request；
When second inquiry request is not inquired in the external cache system, second inquiry request is deposited
In storage to the external cache system, and target data corresponding with second inquiry request is inquired about from target database；
Will target data corresponding with second inquiry request with second inquiry request corresponding storage described outer
In portion's caching system.
The data processing method also includes：
When second inquiry request is inquired in the external cache system, obtained in the external cache system
Take target data corresponding with second inquiry request；
Target data corresponding with second inquiry request is fed back into the client.
Preferably, before target data corresponding with second inquiry request is obtained in the external cache system,
The data processing method also includes：
Judging the time that target data corresponding with second inquiry request is cached in the external cache system is
The no latest update time being later than in the target database；
When the time that target data corresponding with second inquiry request is cached in the external cache system is later than
In the latest update time of the target database, obtained from the external cache system described in execution and looked into described second
Ask the corresponding target data of request.
Preferably, the data processing method also includes：
The time cached in the external cache system when target data corresponding with second inquiry request earlier than
In the latest update time of the target database, removed from the external cache system and second inquiry request pair
The target data answered；
Inquire about target data corresponding with second inquiry request again from the target database, and will look into again
The target data ask is corresponding with second inquiry request to be stored into the external cache system.
Preferably, stored to the external cache system by second inquiry request, the data processing side
Method also includes：
In the external cache system, write at the storage location of target data corresponding with second inquiry request
Target indication information, the Target indication information is used to indicate inquiring about number of targets corresponding with second inquiry request
According to；
It is described will target data corresponding with second inquiry request with second inquiry request it is corresponding storage in institute
In stating external cache system, including：
After target data corresponding with second inquiry request is obtained from target database, will be with described second
The Target indication information updating at the storage location of the corresponding target data of inquiry request is and second inquiry request
Corresponding target data.
Preferably, the data processing method also includes：
Judge target data corresponding with second inquiry request storage location storage whether be that the target refers to
Show information；
When target data corresponding with second inquiry request storage location store be the Target indication information
When, wait, until the Target indication information updating is target data corresponding with second inquiry request, sentence described in execution
Whether the time that target data corresponding with second inquiry request of breaking is cached in the external cache system is later than in institute
State the latest update time of target database；
What is stored when the storage location of target data corresponding with second inquiry request is not Target indication letter
During breath, perform it is described judge that corresponding with second inquiry request target data caches in the external cache system when
Between whether be later than latest update time in the target database.
A kind of data processing equipment, is applied to the inquiry server in data query system, and the inquiry server is set
There is the interface of connection external cache system, and be connected with the external cache system by the interface, the external cache system
Unite for caching query request and target data corresponding with the inquiry request；
The data processing equipment includes：Extraction module, processing module, the first enquiry module, the first memory module, second
Enquiry module and the second memory module；
The extraction module, for when the first inquiry request from client is received, being asked from the described first inquiry
Seek middle extraction target query parameter；
The processing module, the target query parameter for the extraction module to be extracted is processed into by preset rules
Second inquiry request；
First enquiry module, for inquiring about the institute that the processing module treatment is obtained in the external cache system
State the second inquiry request；
First memory module, for not inquiring institute in the external cache system when first enquiry module
When stating the second inquiry request, second inquiry request is stored into the external cache system；
Second enquiry module, for inquiring about number of targets corresponding with second inquiry request from target database
According to；
Second memory module, it is corresponding with second inquiry request for inquire about second enquiry module
Target data storage corresponding with second inquiry request is in the external cache system.
A kind of data query system, including：Inquiry server and external storage system, the inquiry server are provided with company
The interface of external cache system is connect, the inquiry server is connected by the interface with the external cache system；
The external cache system, for caching query request and target data corresponding with the inquiry request；
The inquiry server, for when the first inquiry request from client is received, from the described first inquiry
Target query parameter is extracted in request；The target query parameter is processed into the second inquiry request by preset rules, and in institute
State and inquire about second inquiry request in external cache system；Looked into when not inquiring described second in the external cache system
When asking request, second inquiry request is stored into the external cache system, and inquiry and institute from target database
State the corresponding target data of the second inquiry request；Will target data corresponding with second inquiry request and the described second inquiry
Request correspondence storage is in the external cache system.
Wherein, the inquiry server, be additionally operable to when be inquired in the external cache system it is described second inquiry please
When asking, target data corresponding with second inquiry request is obtained in the external cache system, and will be with described second
The corresponding target data of inquiry request feeds back to the client.
Wherein, the inquiry server, is additionally operable to judge target data corresponding with second inquiry request described
Whether the time cached in external cache system is later than the latest update time in the target database；Looked into when with described second
The time for asking corresponding target data to be cached in the external cache system is ask earlier than in the newest of the target database
During the renewal time, target data corresponding with second inquiry request is removed from the external cache system, from the mesh
Inquire about target data corresponding with second inquiry request again in mark database, and the target data that will be inquired again with
The second inquiry request correspondence is stored into the external cache system.
Above-mentioned technical proposal has the advantages that：
Data processing method, device and data query system that the present invention is provided, are receiving the first inquiry of client
During request, the first inquiry request can be processed into the second inquiry request that can be inquired about in outside caching system, and based on second
Inquiry request is inquired about in outside caching system, if do not inquire the second inquiry request in outside caching system, is shown
It is uncached in external cache system to have target data corresponding with the second inquiry request, now, from target database inquiry with
The corresponding target data of second inquiry request, and the target data is buffered in external cache system.As can be seen here, the present invention
The data processing method of offer, device and data query system cause that target data is no longer buffered in the machine by inquiry server
Internal memory in, but by the second inquiry request and corresponding target data correspondence be buffered in external cache system, due to outside
The buffer memory capacity of caching is generally larger or expansible, therefore, it can realize the caching to all Query Results, solve existing
Have in technology and target data is buffered in the limitation that local internal memory is brought.Further, since the inquiry in data query system
Server is connected with external cache system, i.e. each shared external cache system of inquiry server, as long as this causes outside delaying
Target data corresponding with inquiry request is cached with deposit system, any one inquiry server can be anti-to client faster
Feedback target data corresponding with inquiry request, and avoid the unnecessary wasting of resources.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this
Inventive embodiment, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 is the Organization Chart of Kylin inquiry systems in the prior art；
Fig. 2 is a schematic flow sheet of data cache method data processing method provided in an embodiment of the present invention；
Fig. 3 is another schematic flow sheet of data processing method provided in an embodiment of the present invention；
Fig. 4 is another schematic flow sheet of data processing method provided in an embodiment of the present invention；
Fig. 5 is a structural representation of data processing equipment provided in an embodiment of the present invention；
Fig. 6 is the structural representation of data query system provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
A kind of data processing method is the embodiment of the invention provides, the inquiry server in data query system is applied to,
Inquiry server is provided with the interface of connection external cache system, and is connected with external cache system by the interface, and outside is slow
Deposit system is used for caching query asks and target data corresponding with inquiry request, refers to Fig. 2, shows the data processing side
The schematic flow sheet of method, can include：
Step S201：When the first inquiry request from client is received, target is extracted from the first inquiry request
Query argument.
Step S202：Target query parameter is processed into the second inquiry request by preset rules, and in outside caching system
The second inquiry request of middle inquiry.
It should be noted that the inquiry request of the above-mentioned external cache system cache for referring to is the second inquiry request, and look into
It is target data corresponding with the second inquiry request to ask the corresponding target data of request.
In the present embodiment, external cache system can with but be not limited to memory capacity more than preset value caching system,
It can also be the expansible distributed cache system of buffer memory capacity.Preferably, external cache system is that buffer memory capacity is expansible
Distributed cache system.
Step S203：When the second inquiry request is not inquired in outside caching system, by the storage of the second inquiry request
Into external cache system, and target data corresponding with the second inquiry request is inquired about from target database.
When the second inquiry request is not inquired in outside caching system, show not to be cached with external cache system
Target data corresponding with the second inquiry request, at this time, it may be necessary to go in target database to inquire about corresponding with the second inquiry request
Target data.
Step S204：Will target data corresponding with the second inquiry request corresponding storage is slow in outside with the second inquiry request
In deposit system.
It should be noted that for the ease of subsequently for the inquiry of data, storing mesh corresponding with the second inquiry request
Mark data when, it is necessary to will target data corresponding with the second inquiry request it is corresponding with the second inquiry request.
Data processing method provided in an embodiment of the present invention, when the first inquiry request of client is received, first will
First inquiry request is processed into the second inquiry request that can be inquired about in outside caching system, and based on the second inquiry request in outside
Inquired about in caching system, if not inquiring the second inquiry request in outside caching system, shown in external cache system not
Target data corresponding with the second inquiry request is cached with, now, inquires about corresponding with the second inquiry request from target database
Target data, and the target data is buffered in external cache system.As can be seen here, data provided in an embodiment of the present invention
Processing method causes, no longer be buffered in target data in the internal memory of the machine by inquiry server, but by the second inquiry request with
And corresponding target data correspondence is buffered in external cache system, because the buffer memory capacity of external cache is generally larger or can expand
Exhibition, therefore, it can realize the caching to all Query Results, solve target data is buffered in local in the prior art
Deposit brought limitation.
In addition, in the prior art, because each inquires about server by data buffer storage in the internal memory of the machine, and each is looked into
Server is ask separate, if having cached target data in certain server, when the client request server lookup mesh
Mark data when, the inquiry server can faster to client return Query Result, but if client request another
The inquiry server of uncached target data, the inquiry server quickly cannot return to Query Result to client, because it is needed
Go to inquire about data in target database, then Query Result is returned into client, this can cause to inquire about server customer in response
Hold the speed of inquiry request sometimes fast and sometimes slow, and cause the unnecessary wasting of resources, and at data provided in an embodiment of the present invention
In reason method, because the inquiry server in data query system is connected with external cache system, i.e., each inquires about server
Shared external cache system, it is any as long as this causes to be cached with target data corresponding with inquiry request in external cache system
One inquiry server can faster to client feedback target data corresponding with inquiry request, and avoid it is unnecessary
The wasting of resources.
The data processing method that above-described embodiment is provided, giving ought not inquire the second inquiry in outside caching system
Processing procedure during request, however, sometimes, it is understood that there may be stored with the second inquiry request in external cache system
Situation, for the situation, in order to provide more fully solution, refers to Fig. 3, shows number provided in an embodiment of the present invention
According to another schematic flow sheet of processing method, the data processing method applies equally to the inquiry service in data query system
Device, inquiry server is provided with the interface of connection external cache system, and is connected with external cache system by the interface, outside
Caching system is used for caching query asks and target data corresponding with inquiry request, and the data processing method can include：
Step S301：When the first inquiry request from client is received, target is extracted from the first inquiry request
Query argument.
Step S302：Target query parameter is processed into the second inquiry request by preset rules, and in outside caching system
The second inquiry request of middle inquiry.
In the present embodiment, external cache system can be caching system of the memory capacity more than preset value, or be slow
Deposit the expansible distributed cache system of capacity, it is preferable that external cache system is the expansible distributed caching of buffer memory capacity
System.
Because the inquiry request of client cannot be directly applied for external cache system, therefore, current embodiment require that will be from
The target query parameter processing that first inquiry request is extracted is into the second inquiry request that can be inquired about in outside caching system.
Step S303a：When the second inquiry request is not inquired in outside caching system, by the storage of the second inquiry request
Into external cache system, and in the outside caching system at the storage location of target data corresponding with the second inquiry request
Write-in Target indication information.
Wherein, Target indication information is used to indicate to inquire about target corresponding with the second inquiry request just in target database
Data.Exemplary, write " isRunning " at the storage location of target data corresponding with the second inquiry request.
Step S304a：After target data corresponding with the second inquiry request is obtained, will be corresponding with the second inquiry request
Target data storage location at Target indication information updating be target data corresponding with the second inquiry request, then hold
Row step S305.
Above-mentioned steps S303a~S304a is given when the second inquiry request is not inquired in outside caching system
Processing procedure, however, sometimes, it is also possible to exist external data caching system be cached with it is corresponding with the second inquiry request
Target data situation, now, processing procedure refers to following steps：
Step S303b：When the second inquiry request is inquired in outside caching system, judge and the second inquiry request pair
Whether the storage location storage of the target data answered is Target indication information, when target data corresponding with the second inquiry request
Storage location storage be Target indication information when, perform step S304b-a, otherwise, perform step S304b-b.
Step S304b-a：Poll is waited, until Target indication information updating is number of targets corresponding with the second inquiry request
According to, target data corresponding with the second inquiry request is obtained from storage location, then perform step S305.
Step S304b-b：Target data corresponding with the second inquiry request is obtained from storage location, step is then performed
S305。
Step S305：Target data corresponding with the second inquiry request is fed back into client.
It should be noted that in embodiments of the present invention, when not inquiring the second inquiry request in outside caching system
When, the second inquiry request is stored into external cache system, and it is corresponding with the second inquiry request in outside caching system
Target indication information is write at the storage location of target data, to indicate inquiring about number of targets corresponding with the second inquiry request
According to, it is necessary to explanation, in alternatively possible implementation, by the second inquiry request store to external cache system it
Afterwards, any information, external cache system can be also not written at the storage location of target data corresponding with the second inquiry request
In store the second inquiry request, but the corresponding storage location of the second inquiry request is sky, shows to inquire about and the second inquiry
Corresponding target data is asked, after target data corresponding with the second inquiry request is inquired, directly by the target data
Store to storage location corresponding with the second inquiry request.
Accordingly, when the second inquiry request is inquired in external cache system, judge corresponding with the second inquiry request
Whether the storage location storage of target data is empty, if the storage location of target data corresponding with the second inquiry request is
Sky, then show obtaining target data corresponding with the second inquiry request, then poll wait, until with the second inquiry request pair
The storage location of the target data answered is stored with target data corresponding with the second inquiry request.
Data processing method provided in an embodiment of the present invention, when the first inquiry request of client is received, first will
First inquiry request is processed into the second inquiry request that can be inquired about in outside caching system, and please based on the second inquiry
Ask and inquired about, if not inquiring the second inquiry request in outside caching system, show uncached in external cache system
There is target data corresponding with the second inquiry request, now, mesh corresponding with the second inquiry request is inquired about from target database
Mark data, and the target data is buffered in external cache system, if inquiring the second inquiry in outside caching system
Request, shows caching in external cache system or has been cached with target data corresponding with the second inquiry request, now may be used
Target data corresponding with the second inquiry request is obtained from external cache and client is fed back to.As can be seen here, the present invention is real
The data processing method for applying example offer causes that no longer be buffered in target data in the internal memory of the machine by inquiry server, but will
Second inquiry request and corresponding target data correspondence are buffered in external cache system, because the buffer memory capacity of external cache is led to
It is often larger or expansible, therefore, it can realize the caching to all Query Results, solve number of targets in the prior art
According to being buffered in the limitation that local internal memory is brought.Further, since the inquiry server in data query system is slow with outside
Deposit system is connected, i.e. each shared external cache system of inquiry server, if this cause to be cached with external cache system with
The corresponding target data of inquiry request, any one inquiry server can be corresponding with inquiry request to client feedback faster
Target data, and avoid the unnecessary wasting of resources.
It is understood that the data in target database are probably continuous renewal, sometimes, it is understood that there may be will
One target data is cached to after external cache system, and the target data in target database is updated, and when inquiry
When server is received from client to the inquiry request of the target data, due to the buffered mesh of external cache system
Mark data, therefore, inquiry server can directly obtain the target data from external cache system and feed back to client, and actual
On target data now renewal is had occurred that in target database, that is to say, that inquiry server feedback is to client
Data be not be current newest data, but stale data.In order to the current newest renewal of client feedback
Data, refer to Fig. 4, another schematic flow sheet of data processing method provided in an embodiment of the present invention are shown, at the data
Reason method is applied to the inquiry server in data query system, and inquiry server is provided with connecing for connection external cache system
Mouthful, and be connected with external cache system by the interface, external cache system for caching query request and with inquiry request pair
The target data answered, the data processing method can include：
Step S401：When the first inquiry request from client is received, target is extracted from the first inquiry request
Query argument.
Step S402：Target query parameter is processed into the second inquiry request by preset rules, and in outside caching system
The second inquiry request of middle inquiry.
It should be noted that the inquiry request of the above-mentioned external cache system cache for referring to is the second inquiry request, and look into
It is target data corresponding with the second inquiry request to ask the corresponding target data of request.
In the present embodiment, external cache system can be caching system of the memory capacity more than preset value, or be slow
Deposit the expansible distributed cache system of capacity, it is preferable that external cache system is the expansible distributed caching of buffer memory capacity
System.
Because the inquiry request of client cannot be directly applied for external cache system, therefore, current embodiment require that will be from
The target query parameter processing that first inquiry request is extracted is into the second inquiry request that can be inquired about in outside caching system.
Step S403a：When the second inquiry request is not inquired in outside caching system, by the storage of the second inquiry request
Into external cache system, and in the outside caching system at the storage location of target data corresponding with the second inquiry request
Write-in Target indication information.
Wherein, Target indication information is used to indicate to inquire about target corresponding with the second inquiry request just in target database
Data.Exemplary, write " isRunning " at the storage location of target data corresponding with the second inquiry request.
Step S404a：Target data corresponding with the second inquiry request is inquired about from target data mesh, and is being obtained and second
After the corresponding target data of inquiry request, the target at the storage location of target data corresponding with the second inquiry request is referred to
Show that information updating is target data corresponding with the second inquiry request, then perform step S407.
Above-mentioned steps S403a~S404a is given when the second inquiry request is not inquired in outside caching system
Processing procedure, however, sometimes, it is also possible to exist external data caching system be cached with it is corresponding with the second inquiry request
Target data situation, now, processing procedure refers to following steps：
Step S403b：When the second inquiry request is inquired in outside caching system, judge and the second inquiry request pair
Whether the storage location storage of the target data answered is Target indication information, when target data corresponding with the second inquiry request
Storage location storage be Target indication information when, perform step S404b, otherwise, direct execution step S405b.
Step S404b：Wait, until Target indication information updating is target data corresponding with the second inquiry request, so
Step S405b is performed afterwards.
Step S405b：Judge the time that target data corresponding with the second inquiry request is cached in outside caching system
Whether latest update time in target database is later than, when target data corresponding with the second inquiry request is in external cache system
The time cached in system was later than in the latest update time of target database, performed step S406b-a, was asked when with the second inquiry
When asking the time that corresponding target data is cached in outside caching system earlier than latest update time in target database, hold
Row step S406b-b.
It should be noted that when the target data in target database is updated, can automatic record data it is newest more
The new time, therefore, the latest update time of target data, the number of targets in external cache system can be obtained from target database
According to when being cached also can record buffer memory time automatically, the cache-time of target data is associated with the second inquiry request, based on
Two inquiry requests can obtain the cache-time of target data from external cache system.
If if it is understood that cache-time of the target data in outside caching system is later than target data and exists
The latest update time in target database, after showing that target data is cached into external cache system, the target data exists
Target database is not updated, and the data cached in external cache system are latest data., whereas if target data exists
The latest update time of cache-time in external cache system earlier than target data in target database, show target data
After in caching to external cache system, the target data is updated in target database, i.e., delay in external cache system
The target data deposited is out of date.
Step S406b-a：Target data corresponding with the second inquiry request is obtained from external cache system, is then performed
Step S407.
Step S406b-b：Target data corresponding with the second inquiry request is removed from external cache system, again from mesh
Inquiry target data corresponding with the second inquiry request in mark database, the target data that will be inquired about again is stored to external cache
In system, step S407 is then performed (it should be noted that the target data for now feeding back to client is obtained to inquire about again
Target data).
Step S407：Target data corresponding with the second inquiry request is fed back into client.
It should be noted that in embodiments of the present invention, when not inquiring the second inquiry request in outside caching system
When, the second inquiry request is stored into external cache system, and it is corresponding with the second inquiry request in outside caching system
Target indication information is write at the storage location of target data, to indicate inquiring about number of targets corresponding with the second inquiry request
According to, it is necessary to explanation, in alternatively possible implementation, by the second inquiry request store to external cache system it
Afterwards, any information, external cache system can be also not written at the storage location of target data corresponding with the second inquiry request
In store the second inquiry request, but the corresponding storage location of the second inquiry request is sky, shows to inquire about and the second inquiry
Corresponding target data is asked, after target data corresponding with the second inquiry request is inquired, directly by the target data
Store to storage location corresponding with the second inquiry request.
Accordingly, when the second inquiry request is inquired in external cache system, judge corresponding with the second inquiry request
Whether the storage location storage of target data is empty, if the storage location of target data corresponding with the second inquiry request is
Sky, then show obtaining target data corresponding with the second inquiry request, then poll wait, until with the second inquiry request pair
The storage location of the target data answered is stored with target data corresponding with the second inquiry request.
Data processing method provided in an embodiment of the present invention, when the first inquiry request of client is received, first will
First inquiry request is processed into the second inquiry request that can be inquired about in outside caching system, and based on the second inquiry request
Inquired about, if not inquiring the second inquiry request in outside caching system, shown uncached in external cache system have
Target data corresponding with the second inquiry request, now, inquires about target corresponding with the second inquiry request from target database
Data, and the target data is buffered in external cache system, please if inquiring the second inquiry in outside caching system
When asking, show caching in external cache system or be cached with target data corresponding with the second inquiry request, now may be used
To obtain the target data from external cache system, in order to corresponding with the second inquiry request newest to client feedback
Target data, the embodiment of the present invention can judge that target data corresponding with the second inquiry request is cached in outside caching system
Whether the time is later than the latest update time in target database, if it is, obtained from external cache please with the second inquiry
Seek corresponding target data and feed back to client, otherwise, show that the target data cached in external cache system is out of date, this
When removed, corresponding with the second inquiry request target data is obtained from target database again is cached, and will be weighed
The new target data for obtaining feeds back to client.As can be seen here, data processing method provided in an embodiment of the present invention causes inquiry
No longer be buffered in target data in the internal memory of the machine by server, but by the second inquiry request and corresponding target data pair
External cache system should be buffered in, because the buffer memory capacity of external cache is generally larger or expansible, therefore, it can be realized
To the caching of all Query Results, solve and target data is buffered in the limitation that local internal memory is brought in the prior art.
Further, since the inquiry server in data query system is connected with external cache system, i.e., each inquiry server is shared
External cache system, as long as this causes to be cached with target data corresponding with inquiry request in external cache system, any one
Inquiry server faster to client feedback target data corresponding with inquiry request, and can avoid unnecessary resource
Waste.Furthermore, data processing method provided in an embodiment of the present invention can automatically remove the stale data in external cache system,
And obtain current latest data from target database again and cached, and the latest data of reacquisition is fed back into client
End, so, on the one hand realizes the automatic removing of stale data, on the other hand ensure that when client request data, can
To the current newest data of its feedback.
On the basis of the data processing method that the embodiments of the present invention are provided, now enumerate an instantiation and said
It is bright：
Assuming that data query system is kylin inquiry systems, external cache system is Redis distributed cache systems,
Redis distributed cache systems are data cached with key/value storage organizations, and kylin inquiry systems include that multiple queries take
Business device, each inquiry server is provided with the interface of connection Redis distributed cache systems, and each inquiry server passes through should
Interface is connected with Redis distributed cache systems：
(1) because the storage organization of Redis distributed cache systems is key/value types, and visitor in kylin inquiry systems
The inquiry request that family end sends can not be directly applied to inquiry data cached in Redis distributed cache systems, be based on
This, first be processed into for the first inquiry request from client by inquiry server needs can be in Redis distributed cache systems
Second inquiry request of inquiry, by key of the second inquiry request storage in key/value storage organizations, and deposits in value
Storage target data corresponding with the second inquiry request.
Specifically, when the first inquiry request from client is received：
First, extracted from the first inquiry request target query parameter sql, offset, limit, acceptPartial and
Project, wherein, sql is the sql sentences of inquiry, and offset is the side-play amount of relative record first trip in sql, and limit is limitation
Record strip number, whether, to allow piecemeal, the scope that project is inquired about for this is in which project for acceptPartial.
Then, in order that target query parameter can be unifiedly stored in the key of Redis, and this key unique mark is made
This inquiry, reaches identical inquiry and corresponds to same key so as to find the corresponding same value values purposes of the key, together
When, it is contemplated that json can very easily give expression to the corresponding relation of attribute and value, and also can be with the shape of uniform character string
Formula is stored in the key of Redis, and can be combined together for all target query parameter groups in the form of json by the present embodiment,
And all of target query parameter is arranged and deposited according to the order of sql, offset, limit, acceptPartia, project
Storage, the value values that so both can guarantee that the corresponding redis cachings of request of same inquiry are identicals, can guarantee that it is again
Uniquely, the concrete structure of json is as follows：
{"sql":SqlStr,"offset":OffsetNum,"limit":LimitNum,
"acceptPartial":FTFlag,"project":ProjectName}
Wherein, the sql parameters in the first inquiry request of SqlStr correspondences；In OffsetNum the first inquiry requests of correspondence
Offset parameters；Limit parameters in LimitNum the first inquiry requests of correspondence；In FTFlag the first inquiry requests of correspondence
AcceptPartial parameters；Project parameters in ProjectName the first inquiry requests of correspondence.
It should be noted that process the first inquiry request during, in order to ensure inquiry request uniqueness, it is necessary to
SqlStr is standardized, unnecessary space is removed, it is exemplary, SqlStr parameters " select a from table " and
" select a from table " corresponding Query Result is substantially consistent, if be not standardized to it, can be caused
Two identical inquiry requests are stored in Redis distributed cache systems, can so cause to repeat to inquire about, in order to avoid above-mentioned feelings
The generation of condition, will remove unnecessary space in SqlStr parameters, it is to avoid identical two inquiries of meaning are become into two and is delayed
Record is deposited, to improve Buffer Utilization and search efficiency.
(2) after the first inquiry request to be processed into the second inquiry request (i.e. json query strings), inquiry Redis distributions
Key in formula caching system, if not inquiring the second inquiry request in each key, is transferred to step (3), otherwise, is transferred to step
Suddenly (4).
(3) the second inquiry request is added in key, the corresponding value of the key are temporarily entered as " isRunning ", table
It is bright to inquire about target data corresponding with the second inquiry request in key, the inquiry and the in the target database (such as hbase)
Two inquiry requests correspondence target data, and after the target data is inquired, by value corresponding with key
" isRunning " is updated to the target data, and the target data is fed back into client.
(4) whether using the key of second inquiry request that is stored with as target key, the corresponding value values of target key are judged
It is " isRunning ", if not, being currently cached with showing Redis distributed cache systems corresponding with the second inquiry request
Target data, be now directly transferred to step (5), if it is, poll wait, until the corresponding value values of target key are no longer
For " isRunning " but be updated to Query Result, then go to step (5).
(5) judge whether target data corresponding with the second inquiry request in target key is expired, specifically：From target
Database obtains the latest update time of target data, and the caching of target data is obtained from Redis distributed cache systems
Time, judge whether the cache-time of target data is later than the latest update time of target data：
If the cache-time of target data is later than the latest update time of target data, show target data cache to
After in Redis distributed cache systems, the target data is not updated in target database, Redis distributed cachings
The data cached in system are latest data, and now, acquisition target data feeds back to client from target key corresponding value
End；
If the cache-time of target data is earlier than the latest update time of target data, show target data cache to
After in Redis distributed cache systems, the target data is updated in target database, i.e. Redis distributed cachings
The target data cached in system is out of date, now, the target data is removed from Redis distributed cache systems, i.e.,
The corresponding value values of target key are removed, number of targets corresponding with the second inquiry request is obtained from target database again
According to, then the target data of reacquisition is stored into the corresponding value of target key, and by the number of targets of the reacquisition
According to feeding back to client.
Corresponding with above-mentioned data processing method, the embodiment of the present invention additionally provides a kind of data processing equipment, the data
Processing unit is applied to the inquiry server in data query system, and inquiry server is provided with connecing for connection external cache system
Mouthful, and be connected with external cache system by interface, external cache system is used for caching query asks and corresponding with inquiry request
Target data, refer to Fig. 5, show a structural representation of the data processing equipment, can include：Extraction module 501,
Processing module 502, the first enquiry module 503, the first memory module 504, the second enquiry module 505 and the second memory module 506.
Wherein：
Extraction module 501, for when the first inquiry request from client is received, being carried from the first inquiry request
Take target query parameter.
Processing module 502, the target query parameter for extraction module 501 to be extracted is processed into second and looks into by preset rules
Ask request.
First enquiry module 503, the obtain second inquiry is processed for the query processing module 502 in outside caching system
Request.
It should be noted that the inquiry request of the above-mentioned external cache system cache for referring to is the second inquiry request, and look into
It is target data corresponding with the second inquiry request to ask the corresponding target data of request.
In the present embodiment, external cache system can be caching system of the memory capacity more than preset value, or be slow
Deposit the expansible distributed cache system of capacity, it is preferable that external cache system is the expansible distributed caching of buffer memory capacity
System.
First memory module 504, for not inquiring the second inquiry in outside caching system when the first enquiry module 503
During request, the second inquiry request is stored into external cache system.
Second enquiry module 505, for inquiring about target data corresponding with the second inquiry request from target database.
Second memory module 506, for the number of targets corresponding with the second inquiry request for inquiring about the second enquiry module 505
According to storage corresponding with the second inquiry request in outside caching system.
Data processing equipment provided in an embodiment of the present invention so that target data is no longer buffered in the machine by inquiry server
Internal memory in, but by the second inquiry request and corresponding target data correspondence be buffered in external cache system, due to outside
The buffer memory capacity of caching is generally larger or expansible, therefore, it can realize the caching to all Query Results, solve existing
Have in technology and target data is buffered in the limitation that local internal memory is brought.Further, since the inquiry in data query system
Server is connected with external cache system, i.e. each shared external cache system of inquiry server, as long as this causes outside delaying
Target data corresponding with inquiry request is cached with deposit system, any one inquiry server can be anti-to client faster
Feedback target data corresponding with inquiry request, and avoid the unnecessary wasting of resources.
The data processing equipment that above-described embodiment is provided can also include：Acquisition module and feedback module.
First acquisition module, for when the second inquiry request is inquired in outside caching system, in external cache system
Target data corresponding with the second inquiry request is obtained in system.
Feedback module, for target data corresponding with the second inquiry request to be fed back into client.
The data processing equipment that above-described embodiment is provided can also include：First judge module.
First judge module, for judging that target data corresponding with the second inquiry request is cached in outside caching system
Time whether be later than latest update time in target database.
Acquisition module, specifically for what is cached in outside caching system when target data corresponding with the second inquiry request
Time was later than in the latest update time of target database, obtained corresponding with the second inquiry request from external cache system
Target data.
The data processing equipment that above-described embodiment is provided can also include：Data dump module and the 3rd enquiry module.
Data dump module, for what is cached in outside caching system when target data corresponding with the second inquiry request
Time, earlier than in the latest update time of target database, removes corresponding with the second inquiry request from external cache system
Target data.
3rd enquiry module, for inquiring about target data corresponding with the second inquiry request again from target database,
And the target data that will be inquired again is corresponding with the second inquiry request stores into external cache system.
The data processing equipment that above-described embodiment is provided can also include：3rd memory module.
3rd memory module, in outside caching system, the storage of target data corresponding with the second inquiry request
Target indication information is write at position, wherein, Target indication information is used to indicate inquiring about corresponding with the second inquiry request
Target data.
Then the second memory module, specifically for after target data corresponding with the second inquiry request is obtained, will be with the
Target indication information updating at the storage location of the corresponding target data of two inquiry requests is corresponding with the second inquiry request
Target data.
The data processing equipment that above-described embodiment is provided can also include：Second judge module.
Second judge module, the storage location for judging corresponding with the second inquiry request target data store whether
It is Target indication information.
Then the first judge module, specifically for what is stored when the storage location of target data corresponding with the second inquiry request
When being Target indication information, wait, until Target indication information updating is target data corresponding with the second inquiry request, judge
Whether the time that target data corresponding with the second inquiry request is cached in outside caching system is later than in target database
The latest update time；When target data corresponding with the second inquiry request storage location store be not Target indication information
When, judge whether the time that target data corresponding with the second inquiry request is cached in outside caching system is later than in number of targets
According to the latest update time in storehouse.
The embodiment of the present invention additionally provides a kind of data query system, refers to Fig. 6, shows the data query system
One structural representation, can include：Inquiry server 601 and external storage system 602, inquiry server 601 are provided with connection
The interface of external cache system 602, inquiry server is connected by interface with external cache system 602.It should be noted that looking into
Asking server 601 can be for one can also be multiple, and shown in Fig. 6 is only an example.
External cache system 602, for caching query request and target data corresponding with inquiry request.
In the present embodiment, external cache system can be caching system of the memory capacity more than preset value, or be slow
Deposit the expansible distributed cache system of capacity, it is preferable that external cache system is the expansible distributed caching of buffer memory capacity
System.
Inquiry server 601, for when the first inquiry request from client is received, from the first inquiry request
Extract target query parameter；Target query parameter is processed into the second inquiry request by preset rules, and in outside caching system
The second inquiry request of middle inquiry；When the second inquiry request is not inquired in outside caching system, the second inquiry request is deposited
In storage to external cache system, and inquiry and the second inquiry request from target database (data server shown in Fig. 6)
Corresponding target data；Will target data corresponding with the second inquiry request with the second inquiry request it is corresponding storage in external cache
In system.
Data query system provided in an embodiment of the present invention can also include load equalizer 603 and client end.Wherein
Load equalizer 603 is used for balanced the first inquiry request from client, and the first inquiry request is sent into inquiry server
601。
In the data query system that the present invention is provided, target data is no longer buffered in inquiry server the internal memory of the machine
In, but the second inquiry request and corresponding target data correspondence are buffered in external cache system, due to external cache
Buffer memory capacity is generally larger or expansible, therefore, it can realize the caching to all Query Results, solve prior art
It is middle that target data is buffered in the limitation that local internal memory is brought.Further, since the inquiry server in data query system
It is connected with external cache system, i.e. each shared external cache system of inquiry server, as long as this causes external cache system
In be cached with target data corresponding with inquiry request, any one inquiry server can faster to client feedback with look into
The corresponding target data of request is ask, and avoids the unnecessary wasting of resources.
In the data query system that above-described embodiment is provided, server is inquired about, be additionally operable to work as and looked into outside caching system
Ask during to the second inquiry request, obtain corresponding with the second inquiry request target data in outside caching system, and will be with the
The corresponding target data of two inquiry requests feeds back to client.
In the data query system that above-described embodiment is provided, server is inquired about, be additionally operable to judge and the second inquiry request pair
Whether the time that the target data answered is cached in outside caching system is later than the latest update time in target database.When with
The time that the corresponding target data of second inquiry request is cached in the external cache system is later than in the target database
The latest update time when, inquiry server target data corresponding with the second inquiry request is obtained from external cache system；
The time cached in outside caching system when target data corresponding with the second inquiry request earlier than in target database most
During the new renewal time, inquiry server removes target data corresponding with the second inquiry request from external cache system, from mesh
Again target data corresponding with the second inquiry request, and the target data that will be inquired again and second are inquired about in mark database
Inquiry request correspondence is stored into external cache system.
In the data query system that above-described embodiment is provided, server is inquired about, be additionally operable to be stored by the second inquiry request
After into external cache system, Target indication information is write at the storage location of target data corresponding with inquiry request,
Wherein, Target indication information is used to indicate inquiring about target data corresponding with the second inquiry request, is looked into second obtaining
After the corresponding target data of inquiry request, by the Target indication at the storage location of target data corresponding with the second inquiry request
Information updating is target data corresponding with the second inquiry request.
In the data query system that above-described embodiment is provided, server is inquired about, be additionally operable to judge and the second inquiry request pair
Whether the storage location storage of the target data answered is Target indication information, when target data corresponding with the second inquiry request
Storage location storage be the Target indication information when, wait, until Target indication information updating be with second inquiry please
Seek corresponding target data, judge that target data corresponding with second inquiry request caches in outside caching system when
Between whether be later than latest update time in target database；When the storage location of target data corresponding with the second inquiry request
Storage be not Target indication information when, judge that corresponding with the second inquiry request target data is cached in outside caching system
Time whether be later than latest update time in target database.
Each embodiment is described by the way of progressive in this specification, and what each embodiment was stressed is and other
The difference of embodiment, between each embodiment identical similar portion mutually referring to.
In several embodiments provided herein, it should be understood that disclosed method, device and equipment, can be with
Realize by another way.For example, device embodiment described above is only schematical, for example, the unit
Divide, only a kind of division of logic function there can be other dividing mode when actually realizing, for example multiple units or component
Can combine or be desirably integrated into another system, or some features can be ignored, or do not perform.It is another, it is shown or
The coupling each other for discussing or direct-coupling or communication connection can be by some communication interfaces, between device or unit
Connect coupling or communicate to connect, can be electrical, mechanical or other forms.
The unit that is illustrated as separating component can be or may not be it is physically separate, it is aobvious as unit
The part for showing can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple
On NE.Some or all of unit therein can be according to the actual needs selected to realize the mesh of this embodiment scheme
's.In addition, during each functional unit in each embodiment of the invention can be integrated in a processing unit, or each
Unit is individually physically present, it is also possible to which two or more units are integrated in a unit.
If the function is to realize in the form of SFU software functional unit and as independent production marketing or when using, can be with
Storage is in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other words
The part contributed to prior art or the part of the technical scheme can be embodied in the form of software product, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used to so that a computer equipment (can be individual
People's computer, server, or network equipment etc.) perform all or part of step of each embodiment methods described of the invention.
And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), magnetic disc or CD etc. are various can be with the medium of store program codes.
The foregoing description of the disclosed embodiments, enables professional and technical personnel in the field to realize or uses the present invention.
Various modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, the present invention
The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one
The scope most wide for causing.
Claims (10)
1. a kind of data processing method, it is characterised in that be applied to the inquiry server in data query system, the inquiry clothes
Business device is provided with the interface of connection external cache system, and is connected with the external cache system by the interface, described outer
Portion's caching system is used for caching query asks and target data corresponding with the inquiry request；
The data processing method includes：
When the first inquiry request from client is received, target query parameter is extracted from first inquiry request；
The target query parameter is processed into the second inquiry request by preset rules, and is inquired about in the external cache system
Second inquiry request；
When second inquiry request is not inquired in the external cache system, by second inquiry request store to
In the external cache system, and target data corresponding with second inquiry request is inquired about from target database；
Will target data corresponding with second inquiry request with second inquiry request it is corresponding storage delay in the outside
In deposit system.
2. data processing method according to claim 1, it is characterised in that the data processing method also includes：
When second inquiry request is inquired in the external cache system, in the external cache system obtain with
The corresponding target data of second inquiry request；
Target data corresponding with second inquiry request is fed back into the client.
3. data processing method according to claim 2, it is characterised in that obtained and institute in the external cache system
Before stating the corresponding target data of the second inquiry request, the data processing method also includes：
Judge whether the time that target data corresponding with second inquiry request is cached in the external cache system is late
In the latest update time in the target database；
When the time that target data corresponding with second inquiry request is cached in the external cache system is later than in institute
When stating the latest update time of target database, being obtained from the external cache system described in execution please with the described second inquiry
Seek corresponding target data.
4. data processing method according to claim 3, it is characterised in that the data processing method also includes：
The time cached in the external cache system when target data corresponding with second inquiry request is earlier than in institute
When stating the latest update time of target database, removed from the external cache system corresponding with second inquiry request
Target data；
Inquire about target data corresponding with second inquiry request again from the target database, and will inquire again
Target data corresponding with second inquiry request store into the external cache system.
5. the data processing method according to claim 3 or 4, it is characterised in that by second inquiry request storage
After to the external cache system, the data processing method also includes：
In the external cache system, target is write at the storage location of target data corresponding with second inquiry request
Configured information, the Target indication information is used to indicate inquiring about target data corresponding with second inquiry request；
It is described will target data corresponding with second inquiry request with second inquiry request corresponding storage described outer
In portion's caching system, including：
After target data corresponding with second inquiry request is obtained from target database, will be with the described second inquiry
It is corresponding with second inquiry request to ask the Target indication information updating at the storage location of corresponding target data
Target data.
6. data processing method according to claim 5, it is characterised in that the data processing method also includes：
Whether the storage location storage for judging target data corresponding with second inquiry request is Target indication letter
Breath；
When target data corresponding with second inquiry request storage location store be the Target indication information when, etc.
Treat, until the Target indication information updating is target data corresponding with second inquiry request, perform the judgement with
Whether the time that the corresponding target data of second inquiry request is cached in the external cache system is later than in the mesh
Mark the latest update time of database；
When target data corresponding with second inquiry request storage location store be not the Target indication information when,
Performing the time for judging that target data corresponding with second inquiry request is cached in the external cache system is
The no latest update time being later than in the target database.
7. a kind of data processing equipment, it is characterised in that be applied to the inquiry server in data query system, the inquiry clothes
Business device is provided with the interface of connection external cache system, and is connected with the external cache system by the interface, described outer
Portion's caching system is used for caching query asks and target data corresponding with the inquiry request；
The data processing equipment includes：Extraction module, processing module, the first enquiry module, the first memory module, the second inquiry
Module and the second memory module；
The extraction module, for when the first inquiry request from client is received, from first inquiry request
Extract target query parameter；
The processing module, the target query parameter for the extraction module to be extracted is processed into second by preset rules
Inquiry request；
First enquiry module, for inquire about that processing module treatment obtains in the external cache system described the
Two inquiry requests；
First memory module, for not inquiring described in the external cache system when first enquiry module
During two inquiry requests, second inquiry request is stored into the external cache system；
Second enquiry module, for inquiring about target data corresponding with second inquiry request from target database；
Second memory module, for the target corresponding with second inquiry request for inquiring about second enquiry module
Data storage corresponding with second inquiry request is in the external cache system.
8. a kind of data query system, it is characterised in that including：Inquiry server and external storage system, the inquiry service
Device is provided with the interface of connection external cache system, and the inquiry server is connected by the interface with the external cache system
Connect；
The external cache system, for caching query request and target data corresponding with the inquiry request；
The inquiry server, for when the first inquiry request from client is received, from first inquiry request
Middle extraction target query parameter；The target query parameter is processed into the second inquiry request by preset rules, and described outer
Second inquiry request is inquired about in portion's caching system；Please when second inquiry is not inquired in the external cache system
When asking, second inquiry request is stored into the external cache system, and inquired about from target database and described
The corresponding target data of two inquiry requests；Will target data corresponding with second inquiry request and second inquiry request
Correspondence storage is in the external cache system.
9. data query system according to claim 8, it is characterised in that the inquiry server, is additionally operable to when in institute
Stating when inquire second inquiry request in external cache system, obtained in the external cache system and looked into described second
The corresponding target data of request is ask, and target data corresponding with second inquiry request is fed back into the client.
10. data query system according to claim 9, it is characterised in that the inquiry server, be additionally operable to judge with
Whether the time that the corresponding target data of second inquiry request is cached in the external cache system is later than in the mesh
Mark the latest update time of database；When target data corresponding with second inquiry request is in the external cache system
The time of caching earlier than in the latest update time of the target database, removed from the external cache system with it is described
The corresponding target data of second inquiry request, inquires about corresponding with second inquiry request again from the target database
Target data, and the target data that will be inquired again is corresponding with second inquiry request stores to the external cache system
In.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611147798.8A CN106777085A (en) | 2016-12-13 | 2016-12-13 | A kind of data processing method, device and data query system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611147798.8A CN106777085A (en) | 2016-12-13 | 2016-12-13 | A kind of data processing method, device and data query system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106777085A true CN106777085A (en) | 2017-05-31 |
Family
ID=58880876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611147798.8A Pending CN106777085A (en) | 2016-12-13 | 2016-12-13 | A kind of data processing method, device and data query system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106777085A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154984A (en) * | 2017-07-03 | 2017-09-12 | 罗征 | The distributed control method of shops Saas systems on Novel wire |
| CN107562803A (en) * | 2017-08-08 | 2018-01-09 | 上海数据交易中心有限公司 | Data supply system and method, terminal |
| CN108459914A (en) * | 2017-12-26 | 2018-08-28 | 深圳神州数码云科数据技术有限公司 | A kind of middleware command execution method and device |
| CN109446448A (en) * | 2018-09-10 | 2019-03-08 | 平安科技（深圳）有限公司 | Data processing method and system |
| CN109597915A (en) * | 2018-09-18 | 2019-04-09 | 北京微播视界科技有限公司 | Access request treating method and apparatus |
| CN110807040A (en) * | 2019-10-30 | 2020-02-18 | 北京达佳互联信息技术有限公司 | Method, device, equipment and storage medium for managing data |
| CN112187908A (en) * | 2020-09-23 | 2021-01-05 | 易小博(武汉)科技有限公司 | Data processing method, system and interface service platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101576918A (en) * | 2009-06-19 | 2009-11-11 | 用友软件股份有限公司 | Data buffering system with load balancing function |
| CN101765096A (en) * | 2008-12-26 | 2010-06-30 | 中兴通讯股份有限公司 | Method, device and system for querying order relations |
| CN102117309A (en) * | 2010-01-06 | 2011-07-06 | 卓望数码技术（深圳）有限公司 | Data caching system and data query method |
| CN102902730A (en) * | 2012-09-10 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and device for reading data based on data cache |
| CN105183394A (en) * | 2015-09-21 | 2015-12-23 | 北京奇虎科技有限公司 | Data storage processing method and device |
-
2016
- 2016-12-13 CN CN201611147798.8A patent/CN106777085A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101765096A (en) * | 2008-12-26 | 2010-06-30 | 中兴通讯股份有限公司 | Method, device and system for querying order relations |
| CN101576918A (en) * | 2009-06-19 | 2009-11-11 | 用友软件股份有限公司 | Data buffering system with load balancing function |
| CN102117309A (en) * | 2010-01-06 | 2011-07-06 | 卓望数码技术（深圳）有限公司 | Data caching system and data query method |
| CN102902730A (en) * | 2012-09-10 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and device for reading data based on data cache |
| CN105183394A (en) * | 2015-09-21 | 2015-12-23 | 北京奇虎科技有限公司 | Data storage processing method and device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154984A (en) * | 2017-07-03 | 2017-09-12 | 罗征 | The distributed control method of shops Saas systems on Novel wire |
| CN107562803A (en) * | 2017-08-08 | 2018-01-09 | 上海数据交易中心有限公司 | Data supply system and method, terminal |
| CN108459914A (en) * | 2017-12-26 | 2018-08-28 | 深圳神州数码云科数据技术有限公司 | A kind of middleware command execution method and device |
| CN108459914B (en) * | 2017-12-26 | 2021-01-01 | 深圳神州数码云科数据技术有限公司 | Middleware command execution method and device |
| CN109446448A (en) * | 2018-09-10 | 2019-03-08 | 平安科技（深圳）有限公司 | Data processing method and system |
| CN109597915A (en) * | 2018-09-18 | 2019-04-09 | 北京微播视界科技有限公司 | Access request treating method and apparatus |
| CN109597915B (en) * | 2018-09-18 | 2022-03-01 | 北京微播视界科技有限公司 | Access request processing method and device |
| CN110807040A (en) * | 2019-10-30 | 2020-02-18 | 北京达佳互联信息技术有限公司 | Method, device, equipment and storage medium for managing data |
| CN112187908A (en) * | 2020-09-23 | 2021-01-05 | 易小博(武汉)科技有限公司 | Data processing method, system and interface service platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106777085A (en) | A kind of data processing method, device and data query system | |
| US8818942B2 (en) | Database system with multiple layer distribution | |
| CN109977129A (en) | Multi-stage data caching method and equipment | |
| CN105426321B (en) | Using RDMA close friend's caching method of remote location information | |
| CN105224255B (en) | A kind of storage file management method and device | |
| CN105701219B (en) | A kind of implementation method of distributed caching | |
| CN107577678A (en) | The method of processing data storehouse affairs, client and server | |
| CN104765661B (en) | The multinode hot spare method of Metadata Service node in a kind of cloud storage service | |
| CN104537013B (en) | A kind of method and apparatus of processing data | |
| CN105677580A (en) | Method and device for accessing cache | |
| CN100437590C (en) | Method for prefetching object | |
| CN101094129A (en) | Method for accessing domain name, and client terminal | |
| CN108111325B (en) | Resource allocation method and device | |
| CN106095533A (en) | Method of server expansion and device | |
| CN110069419A (en) | Multilevel cache system and its access control method, equipment and storage medium | |
| US20150142845A1 (en) | Smart database caching | |
| CN103312624A (en) | Message queue service system and method | |
| CN106021468A (en) | Updating method and system for distributed caches and local caches | |
| CN112000287A (en) | IO request processing device, method, equipment and readable storage medium | |
| CN107483631A (en) | A kind of system and method that mobile internet service access is realized using configuration file control caching | |
| CN108780458A (en) | A kind of page cache processing method, device and server | |
| CN106649150A (en) | Cache management method and device | |
| KR20210040864A (en) | File directory traversal method, apparatus, device, and medium | |
| CN109086462A (en) | The management method of metadata in a kind of distributed file system | |
| CN105320676A (en) | Customer data query service method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170531 |