CN106021335A - A database accessing method and device - Google Patents
A database accessing method and device Download PDFInfo
- Publication number
- CN106021335A CN106021335A CN201610299223.1A CN201610299223A CN106021335A CN 106021335 A CN106021335 A CN 106021335A CN 201610299223 A CN201610299223 A CN 201610299223A CN 106021335 A CN106021335 A CN 106021335A
- Authority
- CN
- China
- Prior art keywords
- request
- data
- daily record
- write
- data base
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a database accessing method and device. The method comprises the steps of receiving a request for accessing a database from a client; performing type analysis on the request to determine whether the request is a data writing request; if the request is a data writing request, recording an writing operation corresponding to the request to a local log and writing the data corresponding to the request to a cache layer of the database. According to the embodiments of the invention, the data corresponding to a data writing request is directly written into a cache layer of a database and a response message is returned to a client, so that the efficient of subsequent reading of the data is increased.
Description
Technical field
The present invention relates to field of computer technology, a kind of method and device accessing data base.
Background technology
Along with the development of society, substantial amounts of data need to store, and the data to storage are used according to difference
The demand at family applies certain data access method to process, and data base is exactly the one adapting to this demand
Technology.In simple terms, data base is that data can persist with one, and the mode that can be operated
The data acquisition system preserved.
In actual applications, user can be written and read operation to the data in data base, as increased, deleting,
Amendment, inquiry etc. operate.In correlation technique, as a example by the leveldb LSM Tree data base as representative,
When the dsc data in data base is read by user, data base receives what subscription client sent
After read operation request, can go from local file search, backward subscription client return response message.
Such reading manner, its reading performance is relatively low, can not meet and needs a large amount of situation asking dsc data.Thus,
This technical problem urgently to be resolved hurrily.
Summary of the invention
In view of the above problems, it is proposed that the present invention is to provide one to overcome the problems referred to above or at least in part
Solve the method accessing data base of the problems referred to above and corresponding device.
According to an aspect of of the present present invention, it is provided that a kind of method accessing data base, including:
Receive the request accessing data base from client；
Described request is carried out type analysis, to determine that whether described request is as data write request；And
When described request is data write request, by corresponding for described request write operation record to local day
Will, and corresponding for described request data are write the cache layer of described data base.
Alternatively, described method also includes:
When the read requests that described request is data, described cache layer is searched described request and counts accordingly
According to, and the described data found are returned to described client.
Alternatively, described by the most local for corresponding for described request write operation record daily record, and by described request
The step of the cache layer that corresponding data write described data base farther includes:
Call first thread, and utilize described first thread by corresponding for described request write operation record to institute
State local daily record and corresponding for described request data are write described cache layer.
Alternatively, described method also includes:
Call the second thread and utilize local daily record described in described second thread scans to determine described local day
Whether there is daily record increment in will, wherein said second thread is asynchronous with described first thread；And
If finding described local daily record exists daily record increment, then the described corresponding data of daily record increment are write
In described data base.
Alternatively, described utilize described in described second thread scans local daily record to determine in described local daily record
The step that whether there is daily record increment farther includes:
Utilize described in described second thread scans local daily record, determine whether described local daily record exists not by
The log recording of labelling；And
If, it is determined that described not labeled log recording is described daily record increment.
Alternatively, described the described corresponding data of daily record increment are write in described data base after, described side
Method also includes:
The described corresponding log recording of daily record increment is marked by described local daily record.
Alternatively, described method also includes:
Delete log recording labeled in described local daily record.
Alternatively, described method also includes:
After described data base is broken down and restarted, scan described local daily record, determine described local day
Whether will exists not labeled log recording；And
The most described not labeled corresponding data of log recording are write described data base, and in institute
State in local daily record and described not labeled log recording is marked.
Alternatively, described method also includes:
When the read requests that described request is data, if not finding described request phase in described cache layer
The data answered, then forward the request to described data base；And
Search the corresponding data of described request in the database, the described request found is counted accordingly
According to returning to described client, and in described cache layer, cache the corresponding data of described request.
Alternatively, described cache layer is distributed cache systems memcached.
According to another aspect of the present invention, additionally provide a kind of device accessing data base, including:
Request receiver module, is suitable to receive the request accessing data base from client；
Requirement analysis module, is suitable to described request is carried out type analysis, to determine whether described request is several
According to write request；And
First writing module, is suitable to when described requirement analysis module determines the write request that described request is data
Time, by corresponding for described request write operation record to local daily record, and corresponding for described request data are write
Enter the cache layer of described data base.
Alternatively, described device also includes:
Read module, is suitable to when described requirement analysis module determines the read requests that described request is data,
In described cache layer, search the corresponding data of described request, and the described data found are returned to described
Client.
Alternatively, described first writing module is further adapted for:
Call first thread, and utilize described first thread by corresponding for described request write operation record to institute
State local daily record and corresponding for described request data are write described cache layer.
Alternatively, described device also includes:
Second writing module, is suitable to call the second thread and utilize local daily record described in described second thread scans
To determine in described local daily record whether there is daily record increment, wherein said second thread and described first thread
Asynchronous；And if find described local daily record exists daily record increment, then described daily record increment is counted accordingly
According to writing in described data base.
Alternatively, described second writing module is further adapted for:
Utilize described in described second thread scans local daily record, determine whether described local daily record exists not by
The log recording of labelling；And
If, it is determined that described not labeled log recording is described daily record increment.
Alternatively, described device also includes:
Mark module, is suitable to described for the write of described daily record increment corresponding data at described second writing module
After in data base, in described local daily record, the described corresponding log recording of daily record increment is marked.
Alternatively, described device also includes:
Removing module, is suitable to delete log recording labeled in described local daily record.
Alternatively, described device also includes:
Fault processing module, is suitable to after described data base is broken down and restarted, and scans described local day
Will, determines in described local daily record whether there is not labeled log recording；And
The most described not labeled corresponding data of log recording are write described data base, and in institute
State in local daily record and described not labeled log recording is marked.
Alternatively, described read module is further adapted for:
When described requirement analysis module determines the read requests that described request is data, if at described cache layer
In do not find the corresponding data of described request, then forward the request to described data base；And
Search the corresponding data of described request in the database, the described request found is counted accordingly
According to returning to described client, and in described cache layer, cache the corresponding data of described request.
Alternatively, described cache layer is distributed cache systems memcached.
In embodiments of the present invention, when receiving the request accessing data base from client, please to this
Ask and carry out type analysis to determine that whether this request is as data write request.When determining that this request is data writes
During request, write the data into the most local daily record of the corresponding write operation record of request, and write the data into
Ask the cache layer of corresponding data write into Databasce.It follows that data are directly write by the embodiment of the present invention
Entering the cache layer asking corresponding data write into Databasce, and return response message to client, it can carry
Rise the follow-up efficiency to described digital independent.Further, the embodiment of the present invention writes data into and asks to write accordingly
Enter the most local daily record of operation note, such that it is able to utilize local daily record asynchronously by corresponding for write request data
Write into Databasce, it is ensured that the persistence of data and integrity in data base.It addition, break down data base
After restarting, it is also possible to utilize local daily record that data base is carried out data recovery, it is achieved the mesh of fault restoration
's.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technology of the present invention
Means, and can being practiced according to the content of description, and in order to allow above and other objects of the present invention,
Feature and advantage can become apparent, below especially exemplified by the detailed description of the invention of the present invention.
According to below in conjunction with the accompanying drawing detailed description to the specific embodiment of the invention, those skilled in the art will
More understand the above-mentioned of the present invention and other purposes, advantage and feature.
Accompanying drawing explanation
By reading the detailed description of hereafter preferred implementation, various other advantage and benefit for ability
Territory those of ordinary skill will be clear from understanding.Accompanying drawing is only used for illustrating the purpose of preferred implementation, and also
It is not considered as limitation of the present invention.And in whole accompanying drawing, it is denoted by the same reference numerals identical
Parts.In the accompanying drawings:
Fig. 1 shows the schematic flow sheet of the method accessing data base according to an embodiment of the invention；
Fig. 2 shows the schematic flow sheet of the method accessing data base in accordance with another embodiment of the present invention；
Fig. 3 shows the structural representation of the device accessing data base according to an embodiment of the invention；With
And
Fig. 4 shows the structural representation of the device accessing data base in accordance with another embodiment of the present invention.
Detailed description of the invention
It is more fully described the exemplary embodiment of the disclosure below with reference to accompanying drawings.Although accompanying drawing shows
The exemplary embodiment of the disclosure, it being understood, however, that may be realized in various forms the disclosure and should be by
Embodiments set forth here is limited.On the contrary, it is provided that these embodiments are able to be best understood from this
Open, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
For solving above-mentioned technical problem, embodiments provide a kind of method accessing data base.Fig. 1
Show the schematic flow sheet of the method accessing data base according to an embodiment of the invention.As it is shown in figure 1,
The method at least can include step S102, step S104 and step S106:
Step S102, receives the request accessing data base from client；
Step S104, carries out type analysis to this request, to determine that whether this request is as data write request；
Step S106, when this request is data write request, by corresponding for this request write operation record extremely
Local daily record, and by the cache layer of corresponding for this request data write into Databasce.
In this step, local daily record refers to data base's journal file of hard-disc storage in local physical machine,
It is used for database of record and is accessed for operation.
In embodiments of the present invention, when receiving the request accessing data base from client, please to this
Ask and carry out type analysis to determine that whether this request is as data write request.When determining that this request is data writes
During request, write the data into the most local daily record of the corresponding write operation record of request, and write the data into
Ask the cache layer of corresponding data write into Databasce.It follows that data are directly write by the embodiment of the present invention
Enter the cache layer asking corresponding data write into Databasce, and return response message to client, and without inciting somebody to action
Data write into Databasce, it is possible to increase the follow-up efficiency to described digital independent.Further, the embodiment of the present invention
Write data into the most local daily record of the corresponding write operation record of request, such that it is able to utilize local daily record asynchronous
Ground is by corresponding for write request data write into Databasce, it is ensured that the persistence of data and integrity in data base.
It addition, after data base is broken down and restarted, it is also possible to utilize local daily record that data base is carried out data
Recover, it is achieved the purpose of fault restoration.
In the alternative embodiment of the present invention, in order to improve the safety of database access, in above step
S102 receives after the request accessing data base of client, can test the legitimacy of this request
Card, carries out follow-up step S104 and step S106 after being verified, if checking is not passed through, then enters
Row intercept process, and send alarm prompt.Here legitimacy is according to the safety certification strategy pre-set
Measuring, safety certification strategy can include the legitimacy of authority, the effectiveness of request or attack inspection
Deng, the invention is not restricted to this.Such as, safety certification strategy could be arranged to the visitor according to which IP address
Family end is unallowed to the inquiry of data base；Client in addition to certain IP address be can not delete,
Amendment data base in data, etc. strategy.
In the alternative embodiment of the present invention, when step S104 carries out type analysis to this request, permissible
Parsing the statement accessing data base from this request, analyzing this statement is which type of operation, thus really
Whether this request fixed is data write request.Here type such as increases, deletes, revises, inquiry etc., when
When statement is the statement increasing operation, it is determined that this request is data write request；While statement is deletion action
Statement time, it is determined that this request is data write request；When while statement is the statement of amendment operation, the most really
This request fixed is data write request；When while statement is the statement of inquiry operation, it is determined that this request is data
Read requests, etc..
In the alternative embodiment of the present invention, the cache layer mentioned in above step S106 can be in data
Cache layer newly-built outside storehouse.Owing to data base itself does not possess cache layer, the embodiment of the present invention selects counting
It is used for storing data according to cache layer newly-built outside storehouse temporarily.Such as, by corresponding for write request write operation
After the most local daily record of record, by cache layer newly-built for the write of corresponding for write request data, and return to client
Return response message, thus improve the follow-up efficiency to described digital independent.Here it is possible to according to business demand
Choosing the cache layer of respective stored capacity, the type of cache layer can be such as distributed cache systems
Memcached or the redis etc. deposited based on content.
In the alternative embodiment of the present invention, above step S106 is by corresponding for this request write operation record
To local daily record, and by the cache layer of corresponding for this request data write into Databasce, the embodiment of the present invention provides
A kind of optional scheme, in this scenario, can call first thread, and utilizes the first thread will request
Corresponding write operation record is to local daily record and corresponding data will be asked to write cache layer.It is to say,
Corresponding write operation record will be asked to local daily record by first thread, subsequently corresponding for request data be write
Enter cache layer, backward client return represent completed data base write response message.
Further, in order to ensure persistence and the integrity of data in data base, the embodiment of the present invention is permissible
Utilize local daily record asynchronously by corresponding for write request data write into Databasce, i.e. to call the second thread also
Utilize second thread scans this locality daily record to determine in local daily record whether there is daily record increment, wherein the second line
Journey is asynchronous with first thread, if finding to there is daily record increment in local daily record, is then counted accordingly by daily record increment
According in write into Databasce.Here, local daily record can be scanned by the second thread in real time, it is also possible to sets
Surely scan period of specifying, local daily record was scanned by (such as 0.5 second or 1 second etc.).
In the alternative embodiment of the present invention, when determining daily record increment, embodiments provide one
Optional scheme, i.e. utilize second thread scans this locality daily record, determine whether local daily record exists not by
, if there is not labeled log recording in the log recording of labelling, it is determined that not labeled log recording is
Daily record increment.Further, after in corresponding for daily record increment data write into Databasce, in local daily record
In log recording corresponding to daily record increment be marked, to represent that the corresponding data of daily record increment are written into number
According in storehouse, the problem that can avoid being repeatedly written.Such as, by corresponding for daily record increment data write into Databasce
After in, the corresponding log recording of increment can be kept a diary for " operating " or " in local daily record acceptance of the bid
Record " etc..
In the alternative embodiment of the present invention, in order to save locally stored space, can delete in local daily record
Labeled log recording, such as, is configured local daily record, can delete at predetermined time intervals
Labeled log recording.It addition, local daily record can be according to size cutting, when in certain cutting region
Log recording is the most all labeled, then can delete the log recording in this cutting region.
In the alternative embodiment of the present invention, cause data base that event occurs if as the machine of delaying or other factors
Barrier so that newly write on the most not persistence in data base of the data in cache layer, then restart data base in next time
After, it is possible to use local daily record carries out data recovery to data base, it is achieved the purpose of fault restoration.Specifically,
After data base is broken down and restarted, enable scanning thread, and utilize scanning thread scans this locality daily record,
Determine and whether local daily record exists not labeled log recording, the most then by not labeled daily record note
Record corresponding data write into Databasce, and in local daily record, not labeled log recording is marked.
Such as, newly write on the data " b=2 " in cache layer not in data base persistence (" b=2 " writes accordingly
Enter operation to have recorded to local daily record), data base's catastrophic failure, after data base restarts, enable and sweep
Retouch thread, and utilize scanning thread scans this locality daily record, find local daily record exists not labeled daily record
The corresponding write operation of record, i.e. " b=2 ", then by " b=2 " write into Databasce, and in local daily record
Not labeled log recording (i.e. " b=2 " corresponding write operation) is marked, as being labeled as "
Operation " or " recording " etc..
Further, after data are recovered, data base normally works, and also has new data to write (this in cache layer
In, can use first thread mentioned above that corresponding write operation record will be asked to local daily record and to incite somebody to action
Corresponding data are asked to write cache layer), now can continue with scanning thread scans this locality daily record, with
Determine in local daily record and whether there is daily record increment, if finding local daily record exists daily record increment, then by day
In will increment corresponding data write into Databasce.Here scanning thread and the second thread effect mentioned above
Identical, scanning thread is asynchronous with first thread.
In the alternative embodiment of the present invention, owing to data being cached at cache layer, thus when from
During the read requests that the request of the access data base of client is data, then can directly search in cache layer
The corresponding data of this read requests, and the data found are returned to client, such that it is able to reduce logarithm
According to the read access in storehouse, improve the efficiency of digital independent.
In the alternative embodiment of the present invention, if database failure or other factors cause cache layer data portion
Divide or all lose so that do not find the corresponding data of read requests in cache layer, then the present invention implements
Read requests can be forwarded to data base by example, searches the corresponding data of read requests, will look in data base
The corresponding data of read requests found return to client, and it is corresponding to cache read requests in cache layer
Data.
The realization of the method that access data base of the present invention will be discussed in detail below by a specific embodiment
Journey.In this embodiment, choose the cache layer of respective stored capacity according to business demand, newly-built for data base
Cache layer, the type of cache layer can be such as distributed cache systems memcached or based on interior
The redis etc. deposited.
Fig. 2 shows the schematic flow sheet of the method accessing data base in accordance with another embodiment of the present invention.
As in figure 2 it is shown, the method at least can include step S202, step S204, step S206, step S208
And step S210.
Step S202, for the newly-built cache layer of data base, receives the request accessing data base from client.
Step S204, carries out type analysis to this request, determines whether this request is data write request, if
It is then to continue executing with step S206；If it is not, then continue executing with step S208.
Step S206, calls first thread, and utilizes first thread will ask corresponding write operation record extremely
Local daily record also will ask corresponding data to write cache layer.
In this step, local daily record refers to data base's journal file of hard-disc storage in local physical machine,
It is used for database of record and is accessed for operation.
Step S208, when the read requests that the request of the access data base from client is data, slow
Deposit and layer is searched the corresponding data of this read requests, and the data found are returned to client.
In this step, if database failure or other factors cause cache layer data division or all lose
Lose so that do not find the corresponding data of read requests in cache layer, then the embodiment of the present invention can will be read
The request of taking is forwarded to data base, searches the corresponding data of read requests, the reading that will find in data base
Ask corresponding data to return to client, and in cache layer, cache the corresponding data of read requests.
Step S210, calls the second thread and utilizes second thread scans this locality daily record to determine in local daily record
Whether there is daily record increment, wherein the second thread is asynchronous with first thread, if finding to there is day in local daily record
Will increment, then by corresponding for daily record increment data write into Databasce.
When determining daily record increment, it is possible to use second thread scans this locality daily record, determine in local daily record and be
, if there is not labeled log recording in the log recording that no existence is not labeled, it is determined that not labeled
Log recording is daily record increment.Further, after in corresponding for daily record increment data write into Databasce,
In local daily record, log recording corresponding to daily record increment is marked, to represent that daily record increment counts accordingly
According in written data storehouse, the problem that can avoid being repeatedly written.Such as, by corresponding for daily record increment data
After in write into Databasce, the corresponding log recording of increment can be kept a diary for " to grasp in local daily record acceptance of the bid
Make " or " recording " etc..It addition, in order to save locally stored space, can delete in local daily record
Labeled log recording.
In embodiments of the present invention, data base is caused to be broken down if as the machine of delaying or other factors so that
Newly write on the most not persistence in data base of the data in cache layer, then after restarting data base next time, permissible
Utilize local daily record that data base is carried out data recovery, it is achieved the purpose of fault restoration.Specifically, in data
Breaking down after restarting in storehouse, enables scanning thread, and utilize scanning thread scans this locality daily record, determine
Whether local daily record exists not labeled log recording, the most then by not labeled log recording phase
The data write into Databasce answered, and in local daily record, not labeled log recording is marked.
Further, after data are recovered, data base normally works, and also has new data to write (this in cache layer
In, can use first thread mentioned above that corresponding write operation record will be asked to local daily record and to incite somebody to action
Corresponding data are asked to write cache layer), now can continue with scanning thread scans this locality daily record, with
Determine in local daily record and whether there is daily record increment, if finding local daily record exists daily record increment, then by day
In will increment corresponding data write into Databasce.Here scanning thread and the second thread effect mentioned above
Identical, scanning thread is asynchronous with first thread.
It should be noted that in actual applications, above-mentioned all optional embodiments can use the side of combination
Formula combination in any, forms the alternative embodiment of the present invention, and this is no longer going to repeat them.
The method accessing data base provided based on each embodiment above, based on same inventive concept, this
Bright embodiment additionally provides a kind of device accessing data base.
Fig. 3 shows the structural representation of the device accessing data base according to an embodiment of the invention.As
Shown in Fig. 3, this device 300 at least can include ask receiver module 310, requirement analysis module 320 with
And first writing module 330.
Now introduce each composition of the device 300 accessing data base of the embodiment of the present invention or the function of device and
Annexation between each several part:
Request receiver module 310, is suitable to receive the request accessing data base from client；
Requirement analysis module 320, is coupled with request receiver module 310, is suitable to that request is carried out type and divides
Analysis, to determine that whether request is as data write request；And
First writing module 330, is coupled with requirement analysis module 320, is suitable to when requirement analysis module is true
When fixed request is the write request of data, the most local daily record of corresponding write operation record will be asked, and please
Seek the cache layer of corresponding data write into Databasce.
In an embodiment of the present invention, as shown in Figure 4, the device that Fig. 3 shows can also include:
Read module 340, is coupled with requirement analysis module the 320, first writing module 330, is suitable to work as
When requirement analysis module 320 determines the read requests that request is data, in cache layer, search request is corresponding
Data, and the data found are returned to client.
In an embodiment of the present invention, the first writing module 330 is further adapted for:
Call first thread, and utilize first thread will ask corresponding write operation record to local daily record also
Corresponding data will be asked to write cache layer.
In an embodiment of the present invention, as shown in Figure 4, the device that Fig. 3 shows can also include:
Second writing module 350, is coupled with the first writing module 330, is suitable to call the second thread profit
With second thread scans this locality daily record to determine in local daily record whether there is daily record increment, wherein the second thread
Asynchronous with first thread；And if finding local daily record exists daily record increment, then by corresponding for daily record increment
In data write into Databasce.
In an embodiment of the present invention, the second writing module 350 is further adapted for:
Utilize second thread scans this locality daily record, determine and whether local daily record exists not labeled daily record note
Record；And
If, it is determined that not labeled log recording is daily record increment.
In an embodiment of the present invention, as shown in Figure 4, the device that Fig. 3 shows can also include:
Mark module 360, is coupled with the second writing module 350, and being suitable to will at the second writing module 350
After in daily record increment corresponding data write into Databasce, daily record corresponding to daily record increment in local daily record
Record is marked.
In an embodiment of the present invention, as shown in Figure 4, the device that Fig. 3 shows can also include:
Removing module 370, is coupled with mark module 360, is suitable to delete in local daily record labeled
Log recording.
In an embodiment of the present invention, as shown in Figure 4, the device that Fig. 3 shows can also include:
Fault processing module 380, is coupled with the first writing module 330, mark module 360, is suitable to
Whether data base is broken down after restarting, the local daily record of scanning, determine to exist in local daily record and do not marked
The log recording of note；And
The most then by not labeled log recording corresponding data write into Databasce, and in local daily record
Not labeled log recording is marked.
In an embodiment of the present invention, read module 340 is further adapted for:
When requirement analysis module determines the read requests that request is data, please if not finding in cache layer
Seek corresponding data, then forward the request to data base；
Corresponding for the request found data are returned to client by the corresponding data of search request in data base
End, and the corresponding data of cache request in cache layer.
In an embodiment of the present invention, cache layer is distributed cache systems memcached.
According to any one preferred embodiment above-mentioned or the combination of multiple preferred embodiment, embodiment of the present invention energy
Enough reach following beneficial effect:
In embodiments of the present invention, when receiving the request accessing data base from client, please to this
Ask and carry out type analysis to determine that whether this request is as data write request.When determining that this request is data writes
During request, write the data into the most local daily record of the corresponding write operation record of request, and write the data into
Ask the cache layer of corresponding data write into Databasce.It follows that data are directly write by the embodiment of the present invention
Entering the cache layer asking corresponding data write into Databasce, and return response message to client, it can carry
The high follow-up efficiency to described digital independent.Further, the embodiment of the present invention writes data into and asks to write accordingly
Enter the most local daily record of operation note, such that it is able to utilize local daily record asynchronously by corresponding for write request data
Write into Databasce, it is ensured that the persistence of data and integrity in data base.It addition, break down data base
After restarting, it is also possible to utilize local daily record that data base is carried out data recovery, it is achieved the mesh of fault restoration
's.
In description mentioned herein, illustrate a large amount of detail.It is to be appreciated, however, that this
Bright embodiment can be put into practice in the case of not having these details.In some instances, the most in detail
Known method, structure and technology are shown, in order to do not obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure help to understand in each inventive aspect one or
Multiple, above in the description of the exemplary embodiment of the present invention, each feature of the present invention is sometimes by one
Rise and be grouped in single embodiment, figure or descriptions thereof.But, should be by the method for the disclosure
It is construed to reflect an intention that i.e. the present invention for required protection requires than institute in each claim clearly
The more feature of feature recorded.More precisely, as the following claims reflect, send out
Bright aspect is all features less than single embodiment disclosed above.Therefore, it then follows detailed description of the invention
Claims be thus expressly incorporated in this detailed description of the invention, the conduct of the most each claim itself
The independent embodiment of the present invention.
Those skilled in the art are appreciated that and can carry out adaptive to the module in the equipment in embodiment
Change to answering property and they are arranged in one or more equipment different from this embodiment.Can be reality
Execute the module in example or unit or assembly is combined into a module or unit or assembly, and in addition can be it
Be divided into multiple submodule or subelement or sub-component.Except in such feature and/or process or unit
Outside at least some excludes each other, can use any combination that (this specification being included, adjoint right is wanted
Ask, make a summary and accompanying drawing) disclosed in all features and so disclosed any method or equipment all
Process or unit are combined.Unless expressly stated otherwise, this specification (include adjoint claim,
Summary and accompanying drawing) disclosed in each feature can by provide identical, equivalent or the alternative features of similar purpose
Replace.
Although additionally, it will be appreciated by those of skill in the art that embodiments more described herein include other
Some feature included in embodiment rather than further feature, but the combination meaning of the feature of different embodiment
Taste and is within the scope of the present invention and is formed different embodiments.Such as, in detail in the claims,
One of arbitrarily can mode using in any combination of embodiment required for protection.
The all parts embodiment of the present invention can realize with hardware, or with at one or more processor
The software module of upper operation realizes, or realizes with combinations thereof.It will be understood by those of skill in the art that
Microprocessor or digital signal processor (DSP) can be used in practice to realize according to the present invention real
The some or all functions of the some or all parts in the device of the access data base executing example.The present invention
It is also implemented as part or all the equipment for performing method as described herein or device
Program (such as, computer program and computer program).Such program realizing the present invention is permissible
Store on a computer-readable medium, or can be to have the form of one or more signal.Such letter
Number can download from internet website and to obtain, or provide on carrier signal, or with any other shape
Formula provides.
The present invention will be described rather than limits the invention to it should be noted above-described embodiment, and
And those skilled in the art can design replacement enforcement without departing from the scope of the appended claims
Example.In the claims, any reference marks that should not will be located between bracket is configured to claim
Limit.Word " comprises " and does not excludes the presence of the element or step not arranged in the claims.Be positioned at element it
Front word "a" or "an" does not excludes the presence of multiple such element.The present invention can be by means of bag
Include the hardware of some different elements and realize by means of properly programmed computer.Some listing
In the unit claim of device, several in these devices can be to be come specifically by same hardware branch
Embody.Word first, second and third use do not indicate that any order.Can be by these word solutions
It is interpreted as title.
So far, although those skilled in the art will appreciate that and the most detailed illustrate and describing the present invention's
Multiple exemplary embodiments, but, without departing from the spirit and scope of the present invention, still can be according to this
Disclosure of invention directly determines or derives other variations or modifications of many meeting the principle of the invention.Cause
This, the scope of the present invention is it is understood that and regard as covering other variations or modifications all these.
The one side of the embodiment of the present invention discloses A1, a kind of method accessing data base, including:
Receive the request accessing data base from client；
Described request is carried out type analysis, to determine that whether described request is as data write request；And
When described request is data write request, by corresponding for described request write operation record to local day
Will, and corresponding for described request data are write the cache layer of described data base.
A2, according to the method described in A1, wherein, also include:
When the read requests that described request is data, described cache layer is searched described request and counts accordingly
According to, and the described data found are returned to described client.
A3, according to the method described in A1 or A2, wherein, described by corresponding for described request write operation
Record is to local daily record, and the step of the cache layer that corresponding for described request data are write described data base is entered
One step includes:
Call first thread, and utilize described first thread by corresponding for described request write operation record to institute
State local daily record and corresponding for described request data are write described cache layer.
A4, according to the method according to any one of A1-A3, wherein, also include:
Call the second thread and utilize local daily record described in described second thread scans to determine described local day
Whether there is daily record increment in will, wherein said second thread is asynchronous with described first thread；And
If finding described local daily record exists daily record increment, then the described corresponding data of daily record increment are write
In described data base.
A5, according to the method according to any one of A1-A4, wherein, described utilize described second thread to sweep
Retouch described local daily record to determine that the step that whether there is daily record increment in described local daily record farther includes:
Utilize described in described second thread scans local daily record, determine whether described local daily record exists not by
The log recording of labelling；And
If, it is determined that described not labeled log recording is described daily record increment.
A6, according to the method according to any one of A1-A5, wherein, described by corresponding for described daily record increment
Data write in described data base after, described method also includes:
The described corresponding log recording of daily record increment is marked by described local daily record.
A7, according to the method according to any one of A1-A6, wherein, also include:
Delete log recording labeled in described local daily record.
A8, according to the method according to any one of A1-A7, wherein, also include:
After described data base is broken down and restarted, scan described local daily record, determine described local day
Whether will exists not labeled log recording；And
The most described not labeled corresponding data of log recording are write described data base, and in institute
State in local daily record and described not labeled log recording is marked.
A9, according to the method according to any one of A1-A8, wherein, also include:
When the read requests that described request is data, if not finding described request phase in described cache layer
The data answered, then forward the request to described data base；And
Search the corresponding data of described request in the database, the described request found is counted accordingly
According to returning to described client, and in described cache layer, cache the corresponding data of described request.
A10, according to the method according to any one of A1-A9, wherein, described cache layer is distributed height
Speed caching system memcached.
The another aspect of the embodiment of the present invention also discloses B11, a kind of device accessing data base, including:
Request receiver module, is suitable to receive the request accessing data base from client；
Requirement analysis module, is suitable to described request is carried out type analysis, to determine whether described request is several
According to write request；And
First writing module, is suitable to when described requirement analysis module determines the write request that described request is data
Time, by corresponding for described request write operation record to local daily record, and corresponding for described request data are write
Enter the cache layer of described data base.
B12, according to the device described in B11, wherein, also include:
Read module, is suitable to when described requirement analysis module determines the read requests that described request is data,
In described cache layer, search the corresponding data of described request, and the described data found are returned to described
Client.
B13, according to the device described in B11 or B12, wherein, described first writing module is further adapted for:
Call first thread, and utilize described first thread by corresponding for described request write operation record to institute
State local daily record and corresponding for described request data are write described cache layer.
B14, according to the device according to any one of B11-B13, wherein, also include:
Second writing module, is suitable to call the second thread and utilize local daily record described in described second thread scans
To determine in described local daily record whether there is daily record increment, wherein said second thread and described first thread
Asynchronous；And if find described local daily record exists daily record increment, then described daily record increment is counted accordingly
According to writing in described data base.
B15, according to the device according to any one of B11-B14, wherein, described second writing module is the suitableeest
In:
Utilize described in described second thread scans local daily record, determine whether described local daily record exists not by
The log recording of labelling；And
If, it is determined that described not labeled log recording is described daily record increment.
B16, according to the device according to any one of B11-B15, wherein, also include:
Mark module, is suitable to described for the write of described daily record increment corresponding data at described second writing module
After in data base, in described local daily record, the described corresponding log recording of daily record increment is marked.
B17, according to the device according to any one of B11-B16, wherein, also include:
Removing module, is suitable to delete log recording labeled in described local daily record.
B18, according to the device according to any one of B11-B17, wherein, also include:
Fault processing module, is suitable to after described data base is broken down and restarted, and scans described local day
Will, determines in described local daily record whether there is not labeled log recording；And
The most described not labeled corresponding data of log recording are write described data base, and in institute
State in local daily record and described not labeled log recording is marked.
B19, according to the device according to any one of B11-B18, wherein, described read module is further adapted for:
When described requirement analysis module determines the read requests that described request is data, if at described cache layer
In do not find the corresponding data of described request, then forward the request to described data base；And
Search the corresponding data of described request in the database, the described request found is counted accordingly
According to returning to described client, and in described cache layer, cache the corresponding data of described request.
B20, according to the device according to any one of B11-B19, wherein, described cache layer is distributed
Cache systems memcached.
Claims (10)
1. the method accessing data base, including:
Receive the request accessing data base from client；
Described request is carried out type analysis, to determine that whether described request is as data write request；And
When described request is data write request, by corresponding for described request write operation record to local day
Will, and corresponding for described request data are write the cache layer of described data base.
Method the most according to claim 1, wherein, also includes:
When the read requests that described request is data, described cache layer is searched described request and counts accordingly
According to, and the described data found are returned to described client.
Method the most according to claim 1 and 2, wherein, described writes described request accordingly
Operation note is to local daily record, and corresponding for described request data write the step of the cache layer of described data base
Suddenly farther include:
Call first thread, and utilize described first thread by corresponding for described request write operation record to institute
State local daily record and corresponding for described request data are write described cache layer.
4. according to the method according to any one of claim 1-3, wherein, also include:
Call the second thread and utilize local daily record described in described second thread scans to determine described local day
Whether there is daily record increment in will, wherein said second thread is asynchronous with described first thread；And
If finding described local daily record exists daily record increment, then the described corresponding data of daily record increment are write
In described data base.
5. according to the method according to any one of claim 1-4, wherein, described utilize described second thread
Scan described local daily record to determine that the step that whether there is daily record increment in described local daily record is wrapped further
Include:
Utilize described in described second thread scans local daily record, determine whether described local daily record exists not by
The log recording of labelling；And
If, it is determined that described not labeled log recording is described daily record increment.
6. according to the method according to any one of claim 1-5, wherein, described by described daily record increment phase
After the data answered write in described data base, described method also includes:
The described corresponding log recording of daily record increment is marked by described local daily record.
7. according to the method according to any one of claim 1-6, wherein, also include:
Delete log recording labeled in described local daily record.
8. according to the method according to any one of claim 1-7, wherein, also include:
After described data base is broken down and restarted, scan described local daily record, determine described local day
Whether will exists not labeled log recording；And
The most described not labeled corresponding data of log recording are write described data base, and in institute
State in local daily record and described not labeled log recording is marked.
9. according to the method according to any one of claim 1-8, wherein, also include:
When the read requests that described request is data, if not finding described request phase in described cache layer
The data answered, then forward the request to described data base；And
Search the corresponding data of described request in the database, the described request found is counted accordingly
According to returning to described client, and in described cache layer, cache the corresponding data of described request.
10. access the device of data base, including:
Request receiver module, is suitable to receive the request accessing data base from client；
Requirement analysis module, is suitable to described request is carried out type analysis, to determine whether described request is several
According to write request；And
First writing module, is suitable to when described requirement analysis module determines the write request that described request is data
Time, by corresponding for described request write operation record to local daily record, and corresponding for described request data are write
Enter the cache layer of described data base.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610299223.1A CN106021335A (en) | 2016-05-06 | 2016-05-06 | A database accessing method and device |
| PCT/CN2017/080290 WO2017190580A1 (en) | 2016-05-06 | 2017-04-12 | Method and device for accessing database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610299223.1A CN106021335A (en) | 2016-05-06 | 2016-05-06 | A database accessing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106021335A true CN106021335A (en) | 2016-10-12 |
Family
ID=57082231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610299223.1A Pending CN106021335A (en) | 2016-05-06 | 2016-05-06 | A database accessing method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106021335A (en) |
| WO (1) | WO2017190580A1 (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713450A (en) * | 2016-12-21 | 2017-05-24 | 广州优视网络科技有限公司 | Downloading acceleration method and apparatus based on read-write separation mode |
| WO2017190580A1 (en) * | 2016-05-06 | 2017-11-09 | 北京奇虎科技有限公司 | Method and device for accessing database |
| CN107341267A (en) * | 2017-07-24 | 2017-11-10 | 郑州云海信息技术有限公司 | A kind of distributed file system access method and platform |
| CN108733704A (en) * | 2017-04-20 | 2018-11-02 | 北京京东尚科信息技术有限公司 | Multiple database data processing method, device and storage medium and electronic equipment |
| CN109213432A (en) * | 2017-07-04 | 2019-01-15 | 华为技术有限公司 | Utilize the log-structured storage device and method thereof for merging tree and writing data into |
| CN110019345A (en) * | 2017-12-28 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Data processing method, device, system and medium |
| CN110019132A (en) * | 2017-12-20 | 2019-07-16 | 北京潘达互娱科技有限公司 | Thumb up date storage method and equipment |
| CN110908838A (en) * | 2019-11-19 | 2020-03-24 | 杭州安恒信息技术股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| WO2023116827A1 (en) * | 2021-12-23 | 2023-06-29 | 深圳市世强元件网络有限公司 | High-concurrency data storage method and system |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107895039B (en) * | 2017-11-29 | 2020-11-24 | 华中科技大学 | Method for constructing log database of campus network authentication system |
| CN108829730A (en) * | 2018-05-11 | 2018-11-16 | 中国平安人寿保险股份有限公司 | The method, apparatus and its readable storage medium storing program for executing that a kind of pair of daily record data is stored |
| CN109800260A (en) * | 2018-12-14 | 2019-05-24 | 深圳壹账通智能科技有限公司 | High concurrent date storage method, device, computer equipment and storage medium |
| CN111008233B (en) * | 2019-11-24 | 2023-02-28 | 浪潮电子信息产业股份有限公司 | Method, device and equipment for accessing KV database |
| CN112231274B (en) * | 2020-10-16 | 2024-04-05 | 京东科技控股股份有限公司 | Log summarizing method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1858738A (en) * | 2006-02-15 | 2006-11-08 | 华为技术有限公司 | Method and device for access data bank |
| CN104317926A (en) * | 2014-10-31 | 2015-01-28 | 北京思特奇信息技术股份有限公司 | Persistent data storage method, persistent data query method, persistent data storage device and persistent data query system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810056A (en) * | 2012-11-06 | 2014-05-21 | 南京壹进制信息技术有限公司 | Method for obtaining consistent backup on Linux platform |
| CN104462194A (en) * | 2014-10-28 | 2015-03-25 | 北京国双科技有限公司 | Service data processing method, device and server |
| CN104834724B (en) * | 2015-05-12 | 2018-12-18 | 广东睿江云计算股份有限公司 | A kind of database synchronization method and device |
| CN106021335A (en) * | 2016-05-06 | 2016-10-12 | 北京奇虎科技有限公司 | A database accessing method and device |
-
2016
- 2016-05-06 CN CN201610299223.1A patent/CN106021335A/en active Pending
-
2017
- 2017-04-12 WO PCT/CN2017/080290 patent/WO2017190580A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1858738A (en) * | 2006-02-15 | 2006-11-08 | 华为技术有限公司 | Method and device for access data bank |
| CN104317926A (en) * | 2014-10-31 | 2015-01-28 | 北京思特奇信息技术股份有限公司 | Persistent data storage method, persistent data query method, persistent data storage device and persistent data query system |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017190580A1 (en) * | 2016-05-06 | 2017-11-09 | 北京奇虎科技有限公司 | Method and device for accessing database |
| CN106713450A (en) * | 2016-12-21 | 2017-05-24 | 广州优视网络科技有限公司 | Downloading acceleration method and apparatus based on read-write separation mode |
| CN108733704A (en) * | 2017-04-20 | 2018-11-02 | 北京京东尚科信息技术有限公司 | Multiple database data processing method, device and storage medium and electronic equipment |
| CN108733704B (en) * | 2017-04-20 | 2020-11-20 | 北京京东尚科信息技术有限公司 | Multi-database data processing method and device, storage medium and electronic equipment |
| CN109213432A (en) * | 2017-07-04 | 2019-01-15 | 华为技术有限公司 | Utilize the log-structured storage device and method thereof for merging tree and writing data into |
| CN109213432B (en) * | 2017-07-04 | 2021-10-26 | 华为技术有限公司 | Storage device for writing data using log structured merge tree and method thereof |
| CN107341267A (en) * | 2017-07-24 | 2017-11-10 | 郑州云海信息技术有限公司 | A kind of distributed file system access method and platform |
| CN110019132A (en) * | 2017-12-20 | 2019-07-16 | 北京潘达互娱科技有限公司 | Thumb up date storage method and equipment |
| CN110019345A (en) * | 2017-12-28 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Data processing method, device, system and medium |
| CN110908838A (en) * | 2019-11-19 | 2020-03-24 | 杭州安恒信息技术股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN110908838B (en) * | 2019-11-19 | 2022-09-02 | 杭州安恒信息技术股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| WO2023116827A1 (en) * | 2021-12-23 | 2023-06-29 | 深圳市世强元件网络有限公司 | High-concurrency data storage method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2017190580A1 (en) | 2017-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106021335A (en) | A database accessing method and device | |
| CN103957239B (en) | DNS cache information processing method, equipment and system | |
| CN105049260B (en) | Blog management method and device | |
| CN110188306A (en) | A kind of method and device of web form Auto-writing | |
| CN113687974B (en) | Client log processing method and device and computer equipment | |
| CN110888837B (en) | Object storage small file merging method and device | |
| US20170220298A1 (en) | Methods and apparatus for storing data to a solid state storage device based on data classification | |
| CN111953556B (en) | Website automatic dial testing method and device, computer equipment and readable storage medium | |
| CN107256233B (en) | Data storage method and device | |
| CN104239353B (en) | WEB classification control and log audit method | |
| WO2018006721A1 (en) | Method and apparatus for storing log file | |
| CN102968456B (en) | A kind of raster data reading and processing method and device | |
| CN105095423A (en) | Write-in method and device for count data | |
| CN110781372A (en) | Method and device for optimizing website, computer equipment and storage medium | |
| CN111782621B (en) | Business application log processing method and device | |
| CN109992469B (en) | Method and device for merging logs | |
| CN115840731A (en) | File processing method, computing device and computer storage medium | |
| CN111831691B (en) | Data reading and writing method and device, electronic equipment and storage medium | |
| CN113641681B (en) | Space self-adaptive mass data query method | |
| US10719497B2 (en) | Utilization of optimized ordered metadata structure for container-based large-scale distributed storage | |
| CN104915425A (en) | Method and device for retrieving file content | |
| CN104933105A (en) | Analysis method and device for database access request | |
| CN103336671A (en) | Method and equipment for acquiring data from network | |
| CN108021562B (en) | Disk storage method and device applied to distributed file system and distributed file system | |
| CN113542326B (en) | Data caching method and device of distributed system, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161012 |
|
| RJ01 | Rejection of invention patent application after publication |