CN106897433A - A kind of data capture method and device - Google Patents
A kind of data capture method and device Download PDFInfo
- Publication number
- CN106897433A CN106897433A CN201710110206.3A CN201710110206A CN106897433A CN 106897433 A CN106897433 A CN 106897433A CN 201710110206 A CN201710110206 A CN 201710110206A CN 106897433 A CN106897433 A CN 106897433A
- Authority
- CN
- China
- Prior art keywords
- data
- caching
- database
- request
- target data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Abstract
A kind of data capture method and device are the embodiment of the invention provides, method includes：Access request is obtained, and determines the corresponding target data of the access request；Judge the total data that whether there is in the target data in caching；Wherein, be stored with partial data in database in the caching；If it is, obtaining the target data from the caching；If not, the first data in determining to be present in the caching in the target data, and the second data in the caching are not present in, and first data are obtained from the caching, second data are obtained from the database.Using the embodiment of the present invention, the speed for obtaining data is improve.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of data capture method and device.
Background technology
With the fast development of Internet technology, various information are propagated by internet, and people can lead to
The information crossed needed for internet is obtained.In actual applications, Internet firm provides network service using website, and such user can
With by accessing website, the information required for obtaining oneself.
Specifically, when website gets access request every time, it can determine number to be obtained according to access request
According to, and obtain data to be obtained from database.The access request obtained when website is relatively more, concurrent request amount than it is larger when
Wait, will cause to access the increase of database number of times, so as to increase the pressure of database, cause database access speed slow, obtain
The time of evidence of fetching is long.
The content of the invention
The purpose of the embodiment of the present invention is to provide a kind of data capture method and device, to improve the acquisition speed of data
Degree.Concrete technical scheme is as follows：
In a first aspect, in order to achieve the above object, the embodiment of the invention discloses a kind of data capture method, methods described
Including：
Access request is obtained, and determines the corresponding target data of the access request；
Judge the total data that whether there is in the target data in caching；Wherein, be stored with data in the caching
Partial data in storehouse；
If it is, obtaining the target data from the caching；
If not, the first data in determining to be present in the caching in the target data, and it is not present in institute
State the second data in caching；
First data are obtained from the caching, second data are obtained from the database.
Optionally, it is described that first data are obtained from the caching, second number is obtained from the database
After, methods described also includes：
In being cached described in the second data Cun Chudao.
Optionally, the data for being stored in the caching are the data in the caching term of validity, and the caching term of validity is pin
Data different in the database are pre-set.
Optionally, methods described also includes：
The change obtained for data is asked；Wherein, it is described change request include it is following at least one：Data modification please
Ask, data addition is asked, data removal request and data merge request；
According to the change request, corresponding data in the database are changed；
Determine to have changed in the database identification information of data, according to the identification information, in updating the caching
Data.
Optionally, it is described to obtain the target data from caching, including：
The identification information of the target data is matched with the identification information of the paged data for pre-setting；
Determine whether the target data is paged data according to matching result；
If it is, determining the corresponding page number of the target data；From the corresponding data storage location of the page number described in caching
The target data is obtained, wherein, the caching is stored in same storage in advance by the paged data on the same page number
In region in different storage locations.
Second aspect, in order to achieve the above object, the embodiment of the invention also discloses a kind of data acquisition facility, the dress
Put including：
First determining module, for obtaining access request, and determines the corresponding target data of the access request；
Judge module, for judging in caching with the presence or absence of the total data in the target data；Wherein, the caching
In be stored with partial data in database；
First acquisition module, for being in the case of being, to be obtained from the caching in the judged result of the judge module
Take the target data；
Second determining module, in the case of being no in the judged result of the judge module, in the target data
Middle determination is present in the first data in the caching, and is not present in the second data in the caching；
Second acquisition module, for obtaining first data from the caching, obtains described from the database
Second data.
Optionally, described device also includes：
Memory module, for it will be cached described in the second data Cun Chudao in.
Optionally, the data for being stored in the caching are the data in the caching term of validity, and the caching term of validity is pin
Data different in the database are pre-set.
Optionally, described device also includes：
3rd acquisition module, the change for obtaining for data is asked；Wherein, it is described change request include it is following at least
One：Data modification request, data addition request, data removal request and data merge request；
Change module, for according to the change request, changing corresponding data in the database；
Update module, the identification information for determining to have changed in the database data, according to the identification information, more
Data in the new caching.
Optionally, first acquisition module, including：
Matched sub-block, for by the identification information of the identification information of the target data and the paged data for pre-setting
Matched；
Determination sub-module, for determining whether the target data is paged data according to matching result；
Acquisition submodule, for being in the case of being, to determine the number of targets in the determination result of the determination sub-module
According to the corresponding page number；The target data is obtained from the corresponding data storage location of the page number described in caching, wherein, the caching
In advance by the paged data on the same page number, in the different storage locations in the same storage region of storage.
A kind of data capture method provided in an embodiment of the present invention and device, can in the buffer obtain data, from caching
The speed that the middle speed ratio for obtaining data obtains data from database is fast such that it is able to reduce the time for obtaining data.Certainly,
Implement any product of the invention or method it is not absolutely required to while reaching all the above advantage.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the first schematic flow sheet of data capture method provided in an embodiment of the present invention；
Fig. 2 is second schematic flow sheet of data capture method provided in an embodiment of the present invention；
Fig. 3 is the third schematic flow sheet of data capture method provided in an embodiment of the present invention；
Fig. 4 is the 4th kind of schematic flow sheet of data capture method provided in an embodiment of the present invention；
Fig. 5 is the first structural representation of data acquisition facility provided in an embodiment of the present invention；
Fig. 6 is second structural representation of data acquisition facility provided in an embodiment of the present invention；
Fig. 7 is the third structural representation of data acquisition facility provided in an embodiment of the present invention；
Fig. 8 is the 4th kind of structural representation of data acquisition facility provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
In order to solve the problems of the prior art, a kind of data capture method and device are the embodiment of the invention provides.Under
Kept man of a noblewoman is first described in detail to a kind of data capture method.
Fig. 1 is the first schematic flow sheet of data capture method provided in an embodiment of the present invention, and method includes：
S101：Access request is obtained, and determines the corresponding target data of the access request.
It should be noted that data capture method provided in an embodiment of the present invention can apply to PC (Personal
Computer, personal computer) equipment in internet, it is also possible to it is applied to the equipment in mobile Internet.It is mentioned here
PC internets are traditional internet, and the method that computer network is linked together mutually can be referred to as " network interconnection ", at this
On the basis of develop the global global internetwork of covering and claim internet.Mobile Internet mentioned here refers to internet
Technology, platform, business model and the application movable general name that is combined and put into practice with mobile communication technology.Equipment mentioned here
Can be for processing the server of access request in website.
In embodiments of the present invention, access request is to access the request of the webpage in website.Access is got in server
After request, it may be determined that the corresponding target data of access request.Specifically, can be to be visited according to what is included in access request
The identification information of webpage.According to the identification information of webpage to be visited, it is possible to determine to need the data of loading on webpage to be visited,
So that it is determined that target data.
Exemplary, what access request was accessed is the homepage of the finance and economics video website of mobile terminal, and the homepage of the website is divided into：
Net cast, live broadcast in both illustration and text room, scalp plan and famous expert dynamic etc. column.After server receives above-mentioned access request, can be with
According to the identification information of the homepage included in access request, the corresponding data of homepage are determined.Further, when server gets
For each column access request when, the need for being pre-set in each column load data then be target data.Wherein,
The corresponding data of homepage can be understood as being needed in homepage the data of displaying, and these data are predetermined.Certainly, homepage
In can also include part timing update data, specifically, timing update data can be according to the access of conventional data
The sequence of frequency is determined, or is determined by other method, this is not defined in embodiments of the present invention.
In the net cast column of finance and economics video website, there is newest net cast list and previous direct playing and playback video
List, can also play video, and the drop-down column can show more contents.
In the column of live broadcast in both illustration and text room, expert's studio's list is shown.Corresponding expert studio is clicked on, this can be entered special
The live broadcast in both illustration and text room of family carries out interactive live.Dragging expert's studio's list can show the studio of more multi-expert.Wherein, it is each special
Studio of family sequencing in lists can be ranked up according to number of subscribers and obtain.
The dynamic list of famous expert user's issue that with good grounds popularity algorithm is filtered out in famous expert's dynamic column, click can be looked into
Dynamic full text details are seen, the interactive operation such as can also be commented on and be forwarded.The list is slided, more dynamics can be loaded.
Scalp the list of schedules of scalping comprising expert financial planner in plan column, display is scalped plan prospective earnings and current
The information such as income.Click can check plan details of scalping, and can also carry out subscription operation etc..
If what access request was accessed is the request for accessing some column in homepage, target data is the column pair
The data answered.The corresponding label of column is clicked in homepage, then for the server in finance and economics video website, is exactly obtained
Access request.To be loaded content of the server according to the corresponding column of label clicked on, it may be determined that target data.Certainly, exist
Clicked in each column, dragged, being slided etc. to obtain the operation of more information, then to the server in finance and economics video website
For, exactly obtain access request.In sum, user carries out the operation of acquisition information to finance and economics video website, be exactly to
Access request have sent to the server in finance and economics video website.
S102：Judge the total data that whether there is in the target data in caching；Wherein, it is stored with the caching
Partial data in database, if it is, step S103 is performed, if not, performing step S104.
It should be noted that caching mentioned here can be Redis or Memcache.Redis be one increase income make
Write with ANSI C languages, support network, can be based on internal memory also can persistence log type, Key-Value (key-value) data
Storehouse.Wherein, ANSI is a kind of character code, to make computer supported more multilingual, usually using 1 of 0x00~0x7f scopes
Byte represents 1 English character.Memcache is a freedom, open source code, high-performance, distributed distributed memory pair
As caching system.
In embodiments of the present invention, partial data that can in the buffer in data storage storehouse.Specifically, being stored in caching
Data be caching the term of validity in data.For example, at the beginning, can data storage storehouse in the buffer total data, when
After the time that any data is stored in the buffer exceedes its corresponding caching term of validity, just the data are deleted.Stored in caching
Data be not more than data in database.
Wherein, the caching term of validity can be directed to what different data in database pre-set.It should be noted that slow
It can be determined according to data attribute to deposit the term of validity.For example, can be determined according to the clicking rate of data.If data
Clicking rate is fewer, then cache the term of validity shorter；If the clicking rate of a data is relatively more, the caching term of validity compares
It is long.
After server determines access request corresponding target data, it may determine that in caching with the presence or absence of target data
In total data.For example, the identification information of each data that server can include according to target data, is in judgement caching
The no total data existed in target data.
S103：The target data is obtained from the caching.
When caching includes the total data in target data, illustrate that the target data in the caching term of validity, is serviced
Device can obtain target data from caching.Determined by caching and database property in itself, data are obtained in the buffer
Speed ratio obtained in database data speed it is fast.And data are obtained in the buffer, the visit to database can be reduced
Number of times is asked, database pressure is reduced, the access speed of website is improved so that the quantity for concurrently accessing that website can bear is carried significantly
Height, the machine of delaying that prevention is caused because concurrent access number is big.
Because the speed that the speed ratio for obtaining data in the buffer obtains data in database is fast, operation can be significantly improved
Fluency, efficiently solve the phenomenons such as interim card, improve Consumer's Experience.
S104：The first data in determining to be present in the caching in the target data, and be not present in described
The second data in caching.
In embodiments of the present invention, in the absence of the total data of target data in caching, then illustrate target data all not
In the caching term of validity, or partial data in target data is not in the caching term of validity.The number that can be searched in the buffer
According to being exactly the first data said herein, it is impossible to which the data for finding are exactly the second data.Certainly, the first data and the second number are determined
According to method be not limited in above-mentioned example.
It should be noted that when the total data in target data not in the buffer, then do not include appointing in the first data
What data, the second data are the total data in target data.
S105：First data are obtained from the caching, second data are obtained from the database.
Server determine be present in caching in the first data, and be not present in caching in the second data after, it can
To obtain the first data from caching, the second data are obtained from database, to get whole target datas.
Using the embodiment of the present invention, data can be in the buffer obtained, the speed ratio of data is obtained from caching from data
The speed that data are obtained in storehouse is fast such that it is able to reduce the time for obtaining data.
Second schematic flow sheet of the data capture method that Fig. 2 is provided by the embodiment of the present invention.There is provided based on Fig. 1
Embodiment, a kind of data capture method that the embodiment of the present invention is provided also includes after S105：
S106, in being cached described in the second data Cun Chudao.
Not in the buffer, in order to reduce access times during follow-up acquisition data to database, server can for second data
In the second data Cun Chudao to be cached.Next time, in the caching term of validity of the second data, can be directly obtained from caching
Two data.Data obtaining time is which reduced, the access speed of website is improve, and then, improve Consumer's Experience.
Using the embodiment of the present invention, store data in caching, next time in the caching term of validity of the data, directly
Obtain the data in the buffer, from caching obtain data speed ratio is obtained from database data speed soon, so as to
It is enough to reduce the time for obtaining data.
In another specific embodiment of the invention, referring to Fig. 3, there is provided the third flow of data capture method is shown
It is intended to, compared with embodiment illustrated in fig. 1, in the present embodiment, the target data (S103) is obtained from caching, including：
S1031：The identification information of the target data is matched with the identification information of the paged data for pre-setting.
In embodiments of the present invention, each paged data can be in advance directed to, corresponding identification information is set.For example, the page number 1
Identification information be identification information A, the corresponding identification information of the page number 2 is identification information B, and the corresponding identification information of the page number 3 is mark
Knowledge information C.
The identification information of target data can be included in access request.Wherein, no matter in target data comprising one or
Multiple data, the identification information only one of which of target data.When in target data comprising multiple data, the mark of target data
Information is the overall identification information of all data in target data.
For example, if the identification information of target data is identification information C, the match is successful；If the mark of target data
Information is identification information M, then it fails to match.
S1032：Determine whether the target data is paged data according to matching result, if it is, performing step
S1033。
If matching result is successfully, can determine that target data is paged data；If matching result is failure,
Can determine that target data is not paged data.
S1033：Determine the corresponding page number of the target data；Obtained from the corresponding data storage location of the page number described in caching
The target data is taken, wherein, the caching is stored in same memory block in advance by the paged data on the same page number
In domain in different storage locations.
In embodiments of the present invention, more frequently request is the access request of loading more data in access request.It is right
In this kind of access request, it is necessary to obtain substantial amounts of data, but after getting substantial amounts of data, user might not all need
Or all to check.Therefore, it can add paging and cache, when target data acquisition is carried out, acquisition is paged data,
Waste and the load too high for frequently asking to cause this avoid resource.
Determining the corresponding page number of target data can be：By the identification information of target data and the identification information of paged data
Matched, the corresponding page number of the paged data for matching is the corresponding page number of target data.Certainly, it is not limited in above-mentioned true
The method of the corresponding page number of the data that set the goal.
In embodiments of the present invention, server can be by the data storage on the same page number in caching in same storage
Different storage locations, can so quickly determine the storage location of paged data in region, can improve the speed for obtaining data
Degree.
Likewise, in embodiments of the present invention, server can also be by the data storage on the same page number in database
The different storage locations in same storage region.Server when target data is paged data, and does not exist when data are obtained
When in caching, target data can be obtained in the corresponding storage location of the page number of target data in database.Work as target data
It is paged data, and partial data is not in the buffer, can obtain number of targets by the corresponding data storage location of the page number from caching
Partial data in, in the database in the corresponding storage location of the page number of target data, obtains in target data in addition
A part of data.
Using the embodiment of the present invention, when target data acquisition is carried out, acquisition is paged data, this avoid resource
Waste and frequently ask the load too high that causes.
In an optional embodiment of the embodiment of the present invention, referring to Fig. 4, there is provided the 4th kind of stream of data capture method
Journey schematic diagram, the method is further comprising the steps of：
S107：The change obtained for data is asked；Wherein, it is described change request include it is following at least one：Data are repaiied
Change request, data addition request, data removal request and data and merge request.
In embodiments of the present invention, the data in database can be modified at any time, change mentioned here includes
The modification of data, the addition of data, the deletion of data, the merging of data.
S108：According to the change request, corresponding data in the database are changed.
Server is after change request is got, it is determined that the identification information of the data in change request.When change request is
During data modification request, the identification information of data is the identification information of data to be modified, according to the identification information of data, it is determined that treating
Data to be modified are modified by the data of modification.When request is changed for data addition request, the identification information of data is to treat
The identification information of interpolation data, according to the identification information of data, the storage of the data to be added included from data addition request
Position, obtains data storage to be added to database.When change request is for data removal request, the identification information of data is
The identification information of data to be deleted, according to the identification information of data, determines data to be deleted from database, and deletion is determined
Data.When change request merges request for data, the identification information of data is the identification information of data to be combined, according to number
According to identification information, determine data to be combined from database, merge determined by data.
S109：Determine to have changed in the database identification information of data, according to the identification information, update described slow
Data in depositing.
After being modified to the data in database according to change request, in order to ensure the data in database with caching
In data synchronization, in addition it is also necessary to update caching in data.Certainly, before the data in updating caching, in addition it is also necessary to judge
Whether in the buffer data to be changed, if if, just update the data in caching in database.
When change request is for data modification request, according to the Data Identification before the identification information and change for having changed data
Information, determines the data before modification in the buffer, and the data before modification are replaced with the data changed.When change request is number
During according to addition request, the data of addition are determined according to identification information, the data of addition are obtained from database, and store in caching
In.When change request is for data removal request, data to be deleted are removed from the cache.When change request for data merge
During request, the data before merging are determined in the buffer, the data before merging are replaced with the data for having merged.
Using the embodiment of the present invention, to the data change in database after, according to database change data mark
Information, the data in change caching, it is ensured that the synchronization of data, while also ensure that the correct of the data for obtaining in the buffer
Property.
Corresponding with the embodiment of the method shown in Fig. 1, Fig. 5 is the first of data acquisition facility provided in an embodiment of the present invention
Structural representation is planted, the device includes that the first determining module 201, judge module 202, the first acquisition module 203, second determine mould
The acquisition module 205 of block 204 and second, wherein,
First determining module 201, for obtaining access request, and determines the corresponding target data of the access request；
Judge module 202, for judging in caching with the presence or absence of the total data in the target data；Wherein, it is described
Be stored with partial data in database in caching；
First acquisition module 203, for being in the case of being, from the caching in the judged result of the judge module
Obtain the target data；
Second determining module 204, in the case of being no in the judged result of the judge module, in the number of targets
It is present in the first data in the caching according to middle determination, and is not present in the second data in the caching；
Second acquisition module 205, for obtaining first data from the caching, obtains institute from the database
State the second data.
Using the embodiment of the present invention, data can be in the buffer obtained, the speed ratio of data is obtained from caching from data
The speed that data are obtained in storehouse is fast such that it is able to reduce the time for obtaining data.
Specifically, the data stored in the caching are the data in the caching term of validity, the caching term of validity is pin
Data different in the database are pre-set.
Corresponding with the embodiment of the method shown in Fig. 2, Fig. 6 is the second of data acquisition facility provided in an embodiment of the present invention
Structural representation is planted, the device also includes memory module 206, wherein,
Memory module 206, for it will be cached described in the second data Cun Chudao in.
Using the embodiment of the present invention, store data in caching, next time in the caching term of validity of the data, directly
Obtain the data in the buffer, from caching obtain data speed ratio is obtained from database data speed soon, so as to
It is enough to reduce the time for obtaining data
Corresponding with the embodiment of the method shown in Fig. 3, Fig. 7 is the 3 of data acquisition facility provided in an embodiment of the present invention
Structural representation is planted, the first acquisition module 203 includes matched sub-block 2031, determination sub-module 2032 and acquisition submodule
2033, wherein,
Matched sub-block 2031, for by the mark of the identification information of the target data and the paged data for pre-setting
Information is matched；
Determination sub-module 2032, for determining whether the target data is paged data according to matching result；
Acquisition submodule 2033, for being in the case of being, to determine the mesh in the determination result of the determination sub-module
The corresponding page number of mark data；The target data is obtained from the corresponding data storage location of the page number described in caching, wherein, it is described
Caching in advance by the paged data on the same page number, in the different storage locations in the same storage region of storage.
Using the embodiment of the present invention, when target data acquisition is carried out, acquisition is paged data, this avoid resource
Waste and frequently ask the load too high that causes.
Corresponding with the embodiment of the method that Fig. 4 shows, Fig. 8 is the 4th kind of data acquisition facility provided in an embodiment of the present invention
Structural representation, device also includes：3rd acquisition module 207, change module 208 and update module 209, wherein,
3rd acquisition module 207, the change for obtaining for data is asked；Wherein, the change request includes following
At least one：Data modification request, data addition request, data removal request and data merge request；
Change module 208, for according to the change request, changing corresponding data in the database；
Update module 209, the identification information for determining to have changed in the database data, according to the mark letter
Breath, updates the data in the caching.
Using the embodiment of the present invention, to the data change in database after, according to database change data mark
Information, the data in change caching, it is ensured that the synchronization of data, while also ensure that the correct of the data for obtaining in the buffer
Property.
It should be noted that herein, such as first and second or the like relational terms are used merely to a reality
Body or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or deposited between operating
In any this actual relation or order.And, term " including ", "comprising" or its any other variant be intended to
Nonexcludability is included, so that process, method, article or equipment including a series of key elements not only will including those
Element, but also other key elements including being not expressly set out, or also include being this process, method, article or equipment
Intrinsic key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that
Also there is other identical element in process, method, article or equipment including the key element.
Each embodiment in this specification is described by the way of correlation, identical similar portion between each embodiment
Divide mutually referring to what each embodiment was stressed is the difference with other embodiment.Especially for system reality
Apply for example, because it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
Presently preferred embodiments of the present invention is the foregoing is only, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent substitution and improvements made within the spirit and principles in the present invention etc., are all contained in protection scope of the present invention
It is interior.
Claims (10)
1. a kind of data capture method, it is characterised in that methods described includes：
Access request is obtained, and determines the corresponding target data of the access request；
Judge the total data that whether there is in the target data in caching；Wherein, it is stored with database in the caching
Partial data；
If it is, obtaining the target data from the caching；
If not, the first data in determining to be present in the caching in the target data, and be not present in described slow
The second data in depositing；
First data are obtained from the caching, second data are obtained from the database.
2. method according to claim 1, it is characterised in that described that first data are obtained from the caching, from
Obtained in the database after second data, methods described also includes：
In being cached described in the second data Cun Chudao.
3. method according to claim 1 and 2, it is characterised in that the data stored in the caching are effective in caching
Data in phase, the caching term of validity is directed to what different data in the database pre-set.
4. method according to claim 1 and 2, it is characterised in that methods described also includes：
The change obtained for data is asked；Wherein, it is described change request include it is following at least one：Data modification request, number
Merge request according to addition request, data removal request and data；
According to the change request, corresponding data in the database are changed；
Determine to have changed in the database identification information of data, according to the identification information, update the number in the caching
According to.
5. method according to claim 1, it is characterised in that described to obtain the target data from caching, including：
The identification information of the target data is matched with the identification information of the paged data for pre-setting；
Determine whether the target data is paged data according to matching result；
If it is, determining the corresponding page number of the target data；Obtained from the corresponding data storage location of the page number described in caching
The target data, wherein, the caching is stored in same storage region in advance by the paged data on the same page number
In interior different storage location.
6. a kind of data acquisition facility, it is characterised in that described device includes：
First determining module, for obtaining access request, and determines the corresponding target data of the access request；
Judge module, for judging in caching with the presence or absence of the total data in the target data；Wherein, deposited in the caching
Contain partial data in database；
First acquisition module, for being in the case of being, institute to be obtained from the caching in the judged result of the judge module
State target data；
Second determining module, in the case of being no in the judged result of the judge module, in the target data really
Surely it is present in the first data in the caching, and is not present in the second data in the caching；
Second acquisition module, for obtaining first data from the caching, obtains described second from the database
Data.
7. device according to claim 6, it is characterised in that described device also includes：
Memory module, for it will be cached described in the second data Cun Chudao in.
8. the device according to claim 6 or 7, it is characterised in that the data stored in the caching are effective in caching
Data in phase, the caching term of validity is directed to what different data in the database pre-set.
9. the device according to claim 6 or 7, it is characterised in that described device also includes：
3rd acquisition module, the change for obtaining for data is asked；Wherein, the change request includes following at least one
：Data modification request, data addition request, data removal request and data merge request；
Change module, for according to the change request, changing corresponding data in the database；
Update module, the identification information for determining to have changed in the database data, according to the identification information, updates institute
State the data in caching.
10. device according to claim 6, it is characterised in that first acquisition module, including：
Matched sub-block, for the identification information of the target data to be carried out with the identification information of the paged data for pre-setting
Matching；
Determination sub-module, for determining whether the target data is paged data according to matching result；
Acquisition submodule, for being in the case of being, to determine the target data pair in the determination result of the determination sub-module
The page number answered；The target data is obtained from the corresponding data storage location of the page number described in caching, wherein, the caching is advance
By the paged data on the same page number, in the different storage locations in the same storage region of storage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710110206.3A CN106897433A (en) | 2017-02-27 | 2017-02-27 | A kind of data capture method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710110206.3A CN106897433A (en) | 2017-02-27 | 2017-02-27 | A kind of data capture method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106897433A true CN106897433A (en) | 2017-06-27 |
Family
ID=59184336
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710110206.3A Pending CN106897433A (en) | 2017-02-27 | 2017-02-27 | A kind of data capture method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106897433A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357857A (en) * | 2017-06-29 | 2017-11-17 | 深圳市金立通信设备有限公司 | A kind of method and service node device for updating cache information |
| CN107808343A (en) * | 2017-11-07 | 2018-03-16 | 福建中金在线信息科技有限公司 | Data processing method, terminal and server |
| CN108132996A (en) * | 2017-12-20 | 2018-06-08 | 福建中金在线信息科技有限公司 | A kind of data processing method and transmitting terminal |
| CN109213875A (en) * | 2018-07-24 | 2019-01-15 | 努比亚技术有限公司 | File management method, mobile terminal and computer readable storage medium |
| CN110019362A (en) * | 2017-11-08 | 2019-07-16 | 中移(苏州)软件技术有限公司 | A kind of method and device accessing database |
| CN110377622A (en) * | 2019-06-19 | 2019-10-25 | 深圳新度博望科技有限公司 | Data capture method, data retrieval method and request responding method |
| CN110866202A (en) * | 2019-11-15 | 2020-03-06 | 成都知道创宇信息技术有限公司 | Front-end paging method and device and readable storage medium |
| CN112883065A (en) * | 2021-03-03 | 2021-06-01 | 中煤航测遥感集团有限公司 | Data storage method and device and computer equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102214192A (en) * | 2010-04-12 | 2011-10-12 | 腾讯科技（深圳）有限公司 | Method for realizing data curve chart display on Web page and server |
| CN105138587A (en) * | 2015-07-31 | 2015-12-09 | 小米科技有限责任公司 | Data access method, apparatus and system |
| CN105183764A (en) * | 2015-07-29 | 2015-12-23 | 广州神马移动信息科技有限公司 | Data paging method and apparatus |
| CN105608159A (en) * | 2015-12-18 | 2016-05-25 | 北京奇虎科技有限公司 | Data caching method and device |
| CN106066877A (en) * | 2016-05-30 | 2016-11-02 | 北京皮尔布莱尼软件有限公司 | A kind of method and system of asynchronous refresh data |
| CN106202485A (en) * | 2016-07-18 | 2016-12-07 | 乐视控股（北京）有限公司 | Data manipulation method and system |
-
2017
- 2017-02-27 CN CN201710110206.3A patent/CN106897433A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102214192A (en) * | 2010-04-12 | 2011-10-12 | 腾讯科技（深圳）有限公司 | Method for realizing data curve chart display on Web page and server |
| CN105183764A (en) * | 2015-07-29 | 2015-12-23 | 广州神马移动信息科技有限公司 | Data paging method and apparatus |
| CN105138587A (en) * | 2015-07-31 | 2015-12-09 | 小米科技有限责任公司 | Data access method, apparatus and system |
| CN105608159A (en) * | 2015-12-18 | 2016-05-25 | 北京奇虎科技有限公司 | Data caching method and device |
| CN106066877A (en) * | 2016-05-30 | 2016-11-02 | 北京皮尔布莱尼软件有限公司 | A kind of method and system of asynchronous refresh data |
| CN106202485A (en) * | 2016-07-18 | 2016-12-07 | 乐视控股（北京）有限公司 | Data manipulation method and system |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357857A (en) * | 2017-06-29 | 2017-11-17 | 深圳市金立通信设备有限公司 | A kind of method and service node device for updating cache information |
| CN107808343A (en) * | 2017-11-07 | 2018-03-16 | 福建中金在线信息科技有限公司 | Data processing method, terminal and server |
| CN110019362A (en) * | 2017-11-08 | 2019-07-16 | 中移(苏州)软件技术有限公司 | A kind of method and device accessing database |
| CN110019362B (en) * | 2017-11-08 | 2021-08-13 | 中移(苏州)软件技术有限公司 | Method and device for accessing database |
| CN108132996A (en) * | 2017-12-20 | 2018-06-08 | 福建中金在线信息科技有限公司 | A kind of data processing method and transmitting terminal |
| CN109213875A (en) * | 2018-07-24 | 2019-01-15 | 努比亚技术有限公司 | File management method, mobile terminal and computer readable storage medium |
| CN110377622A (en) * | 2019-06-19 | 2019-10-25 | 深圳新度博望科技有限公司 | Data capture method, data retrieval method and request responding method |
| CN110866202A (en) * | 2019-11-15 | 2020-03-06 | 成都知道创宇信息技术有限公司 | Front-end paging method and device and readable storage medium |
| CN112883065A (en) * | 2021-03-03 | 2021-06-01 | 中煤航测遥感集团有限公司 | Data storage method and device and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106897433A (en) | A kind of data capture method and device | |
| US9892426B2 (en) | Search assistant system and method | |
| US20160328485A1 (en) | Managing web browser cache for offline browsing | |
| CN100559768C (en) | A kind of information distribution method and system | |
| CN104065565B (en) | The method of PUSH message, server, client terminal device and system | |
| US9535999B1 (en) | Trending search magazines | |
| KR100672277B1 (en) | Personalized Search Method Using Cookie Information And System For Enabling The Method | |
| US20170168959A1 (en) | Managing and organizing web browser cache | |
| US20180336175A1 (en) | Method and System for Semantically Generating and Digitally Publishing Articles | |
| CN104052809B (en) | A kind of flow-dividing control method and apparatus of website test | |
| EP2941724A1 (en) | Method and apparatus for generating webpage content | |
| CN102737037A (en) | Webpage pre-reading method, device and browser | |
| CN104978368A (en) | Method and device used for providing recommendation information | |
| US10558721B2 (en) | Search tool enhancement using dynamic tagging | |
| CN102750081A (en) | Information processing apparatus, information processing method, and program | |
| US20150066902A1 (en) | Modifying Search Results Based on Selection of One of the Search Results | |
| US9852229B2 (en) | System, method and computer program product for navigating content on a single page | |
| CN106294402A (en) | The data search method of a kind of heterogeneous data source and device thereof | |
| US20100174719A1 (en) | System, method, and program product for personalization of an open network search engine | |
| CN113779416B (en) | Information recommendation method and device, electronic equipment and computer-readable storage medium | |
| US20200073925A1 (en) | Method and system for generating a website from collected content | |
| CN112559913B (en) | Data processing method, device, computing equipment and readable storage medium | |
| US20180365302A1 (en) | System for interactively searching for and displaying information on a webpage | |
| US10235459B1 (en) | Creating entries in at least one of a personal cache and a personal index | |
| WO2021189766A1 (en) | Data visualization method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170627 |