CN106549983B - Database access method, terminal and server - Google Patents
Database access method, terminal and server Download PDFInfo
- Publication number
- CN106549983B CN106549983B CN201510591099.1A CN201510591099A CN106549983B CN 106549983 B CN106549983 B CN 106549983B CN 201510591099 A CN201510591099 A CN 201510591099A CN 106549983 B CN106549983 B CN 106549983B
- Authority
- CN
- China
- Prior art keywords
- meta table
- updating
- update
- identification information
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a database access method, a terminal and a server, wherein the terminal locally stores a meta table, and the method comprises the following steps: the terminal starts a timer to start timing; the terminal judges whether a task for accessing the database is obtained or not in the process of starting timing by the timer to obtain a first judgment result; when the first judgment result shows that the task of accessing the database is not obtained when the timer is timed to end, the terminal sends an access request to the server, wherein the access request is used for requesting first identification information in the meta table, and the first identification information is used for showing whether the terminal needs to update the local meta table; the terminal receives an access response sent by the server, wherein the access response carries first identification information; and when the first identification information indicates that the terminal needs to update the local meta table, the terminal updates the local meta table.
Description
Technical Field
The present invention relates to a distributed storage technology, and in particular, to a database access method, a terminal, and a server.
Background
Big data (big data) is a direction which is widely concerned in the current academic research field and the commercial application field, and refers to massive, high-growth rate and diversified information assets which can have stronger decision-making power, insight discovery power and flow optimization capability under a new processing mode. The strategic significance of big data technology is not to grasp huge data information, but to specialize the data containing significance. That is, the value of the big data is not in the data itself, but the information which can be generated after the data is processed and has the function of guidance and decision making,
big data is generated along with cloud computing, in the cloud era, massive user data are turned to a cloud end from the local, a mechanism providing cloud service grasps massive data, and the big data technology is an effective tool for mining the value hidden behind the data. Technically, the relation between big data and cloud computing is as inseparable as the front and back of a coin. The large data cannot be processed by a single computer necessarily, and a distributed architecture must be adopted. The method is characterized in that distributed data mining is carried out on mass data, but the method is based on distributed processing of cloud computing, a distributed database, cloud storage, virtualization technology and the like. Meanwhile, target information mined by the big data technology can provide more intelligent decision support for cloud computing. The two forms an intelligent cloud service system together.
A distributed storage system is a system in which data is stored in a distributed manner in a plurality of independent devices connected to each other. The traditional network storage system adopts a centralized storage server to store all data, the storage server becomes the bottleneck of the system performance, meanwhile, the hidden dangers of safety and reliability exist, and the requirement of large-scale storage application cannot be met. The distributed network storage system is a necessary storage means in a big data environment, adopts an expandable system structure, utilizes a plurality of storage servers to share storage load, utilizes a position server to position storage information, not only improves the reliability, the availability and the access efficiency of the system, but also is easy to expand.
The distributed storage system adopts a non-relational database system, has a relatively loose mechanism and is mainly used for storing unstructured and semi-structured data. Distributed large data storage databases that have been widely used today include HadoopDatabase (Hbase), Google Bigtable, NoSQL, and so on. Hbase is the technology with the highest application rate in the current large data platform, and particularly shows excellent performance for unstructured data such as text, social media subscription and video.
Distributed storage databases (Hbase) are distributed, column-oriented storage systems that provide real-time read-write and random access to large data sets. Fig. 1-1 is a schematic diagram illustrating a composition structure of a distributed storage database Hbase in the related art, and as shown in fig. 1-1, a table of the database is composed of different regions (regions), each Region containing a subset of all rows of the table. A Region server (Region server) is responsible for managing one or more regions. The database consists of one Master node (Master) coordinating one or more zone servers. The main node is responsible for guiding initial installation, distributing the region to the region server and recovering the failure of the region server, and the load is light. The area server is responsible for managing 0 to a plurality of areas and responding to the read-write request of the client. The realization of the database depends on a manager (Zookeeper) to coordinate management, wherein the manager is responsible for selecting one node as a main node and the rest nodes as regional servers.
The Region (Region) is the minimum storage unit of distributed storage, when the table initially writes data, the table has only one Region, when the capacity of the Region begins to become larger as the data increases, and when it reaches a defined threshold size, the master node splits the Region into two regions of substantially the same size. The master node server maintains two data tables, a root table and a meta table, to ensure that the area with the target data is located. The root table records only one, is stored in a single area, and records the position of the meta table; there may be a plurality of meta tables, which record areas where user information is located. Fig. 1-2 are schematic flow diagrams of physical storage in the related art, and as shown in fig. 1-2, a client needs to perform the following 4 steps when accessing target data:
in step 121, the client connects to the manager and looks up the location of the Root table (-Root).
In step 122, the client acquires the position of the access target in the Meta table (-Meta) through the root table.
Step 123, the client searches the meta table to obtain the node and the position of the target information in the user table.
And step 124, the client directly interacts with the regional server to perform the operations of adding, deleting, modifying and checking.
As can be seen from fig. 1-2, the client needs to access the location information of the target at least three times (manager, root table, meta table), which results in a slow access speed, in order to achieve fast access to the distributed database, the client caches the remote root table and meta table information locally, and only needs one connection to locate the target data when accessing the database each time, such as the access manner ① shown in fig. 1-3, which is directly located to the area by the client, which undoubtedly greatly shortens the access time of the client, and can achieve fast reading and writing of the distributed database, but this local caching manner also brings the problem of asynchronous caching,
as shown in FIGS. 1-3, in an access manner ②, when a client accesses a target area, the client first reads the local cache, finds that the target area cannot be found after multiple attempts, then the client accesses the target data through a manager, a root table and a meta table for three times again, and then updates the information of the data in the local cache, so that the access time of the client is greatly increased.
Disclosure of Invention
In view of this, embodiments of the present invention provide a database access method, a terminal and a server to solve at least one problem in the prior art, so as to enable the terminal to access the database at a high speed.
The technical scheme of the embodiment of the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a method for accessing a database, where a terminal locally stores a meta table, and the method includes:
starting a timer to start timing;
judging whether a task for accessing the database is obtained or not in the process of starting timing by the timer to obtain a first judgment result;
when the first judgment result shows that the task for accessing the database is not obtained when the timer is timed to end, sending an access request to a server, wherein the access request is used for requesting first identification information in a meta table;
receiving an access response sent by the server, wherein the access response carries first identification information;
and when the first identification information indicates that the terminal needs to update the local meta table, updating the local meta table.
In a second aspect, an embodiment of the present invention provides a method for accessing a database, where the method includes:
when the information in the meta table changes, the server updates the parameter values in the meta table attributes;
judging whether the parameter value meets a preset condition or not to obtain a second judgment result;
and when the second judgment result shows that the parameter value meets the preset condition, updating the first identification information of the element table into first information, wherein the first information shows that the terminal needs to update the locally stored element table.
In a third aspect, an embodiment of the present invention provides a terminal, where a meta table is locally stored in the terminal, and the terminal includes an opening unit, a first determining unit, a first sending unit, a first receiving unit, and a first updating unit, where:
the starting unit is used for starting a timer to start timing;
the first judging unit is used for judging whether a task for accessing the database is obtained in the process of starting timing of the timer to obtain a first judging result;
the first sending unit is configured to send an access request to a server when the first determination result indicates that a task for accessing the database is not obtained when the timer is timed to end, where the access request is used to request first identification information in a meta table;
the first receiving unit is configured to receive an access response sent by the server, where the access response carries first identification information;
and the first updating unit is used for updating the local meta table when the first identification information indicates that the terminal needs to update the local meta table.
In a fourth aspect, an embodiment of the present invention provides a server, where the server includes a second updating unit, a second determining unit, and a third updating unit, where:
the second updating unit is used for updating the parameter values in the attribute of the meta table when the information in the meta table changes;
the second judging unit is used for judging whether the parameter value meets a preset condition or not to obtain a second judging result;
the third updating unit is configured to update the first identifier information of the meta table to first information when the second determination result indicates that the parameter value meets a preset condition, where the first information indicates that the terminal needs to update the locally stored meta table.
The embodiment of the invention provides a database access method, a terminal and a server, wherein the terminal locally stores a meta table, and the method comprises the following steps: the terminal starts a timer to start timing; the terminal judges whether a task for accessing the database is obtained or not in the process of starting timing by the timer to obtain a first judgment result; when the first judgment result shows that the task of accessing the database is not obtained when the timer is timed to end, the terminal sends an access request to the server, wherein the access request is used for requesting first identification information in the meta table, and the first identification information is used for showing whether the terminal needs to update the local meta table; the terminal receives an access response sent by the server, wherein the access response carries first identification information; when the first identification information indicates that the terminal needs to update the local meta table, the terminal updates the local meta table; in this way, the speed at which the terminal can access the database is enabled.
Drawings
FIG. 1-1 is a schematic diagram illustrating a structure of a distributed database Hbase according to the related art;
FIGS. 1-2 are schematic diagrams illustrating a flow chart of a related art physical storage;
FIGS. 1-3 are schematic diagrams illustrating a data access flow using a cache mechanism in the related art;
fig. 1-4 are schematic diagrams illustrating a flow chart of an implementation of a database access method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating an implementation of a method for accessing a second database according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an implementation flow of a database access method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a component of an access system of a database according to an embodiment of the present invention.
Detailed Description
In the following embodiments of the present invention, two mechanisms will be provided, the first is a new meta table change maintenance mechanism, and the second is a client low-load cache update mechanism. The new meta table change maintenance mechanism is to set a new parameter attribute and record the change of the meta table, and since the meta table itself records all the area information, the change of the meta table reflects the change of the entire database. The client low-load cache updating mechanism is used for enabling a local cache of a client to be updated in time under the condition that the extra load of a database is reduced as much as possible, so that the temporary updating when the client accesses the database is avoided. It is seen from the background that the temporary update when a client accesses a database should be avoided, just because the temporary update when the client accesses the database causes the access time to become longer.
The following introduces a meta-table change maintenance mechanism and a client low-load cache update mechanism, wherein:
the mechanism for maintaining the change of the meta table is to add the following 4 attributes to the original meta table, as shown in table 1:
table 1 added attribute field in meta table
| Parameter(s) | Description of the invention |
| Number of times of changeCount update | Number of regions in meta table that change |
| Percent of changePercent update | Percentage of change in area in meta table |
| average time update time | Average time required to reach target percent change |
| update flag bit of updateFlag | Updating the flag bit |
| Threshold value | Threshold value |
The meanings and effects of the individual attributes in Table 1: the changeCount parameter represents the number of the changed areas in the current meta table, and the areas in the database respond to the attribute every time the division and combination actions are performed. The changePercent parameter represents the percentage of the current changed area quantity in the total area quantity, when the percentage reaches a threshold value threshold, the current database is indicated to be changed greatly, the client side needs to synchronously update the local cache of the client side, and at the moment, the updateFlag bit is operated, so that the client side can monitor the change information in time. The averageTime parameter records the average time when the changePercent reaches the threshold value in all rounds, and the client uses the time as the reference time for updating the local cache.
The client low-load cache updating mechanism comprises the following steps:
the working process of the client can be divided into two situations, firstly, when the client obtains an access task, data reading and writing are carried out according to the trial and error mode shown in figures 1-3, and local updating of local cache is realized; secondly, when the client does not obtain an access task in the running state (the client is idle), a timer is set, the distributed database is automatically accessed after a time period averageTime elapses, an updateFlag attribute is firstly inquired, if the attribute is true, which indicates that the database is greatly changed (a change threshold value can be defined by a user), the client needs to completely update the local cache, and meanwhile, the averageTime is updated to obtain the time of the next automatic access. Therefore, the access frequency of the client can be controlled by setting the threshold, and automatic access and update can be triggered only when the database changes to a certain degree, and the access amount of the database does not cause great load pressure on the database, so that a low-load cache update mechanism of the client is realized.
By adopting the low-load automatic updating method, the local cache of the client is automatically updated, the longer access delay with a certain probability brought by the original trial-and-error updating mode is avoided, and the pressure on the database caused by the frequent access of the client is also avoided. The method is suitable for a distributed database system with a rapidly increased data volume, and has important application significance in the field of large data storage and management.
The technical solution of the present invention is further elaborated below with reference to the drawings and the specific embodiments.
Example one
In order to solve the problem of the background art, an embodiment of the present invention provides a method for accessing a database, where the method is applied to a terminal, where a meta table and a root table are locally stored in the terminal, and a function implemented by the method may be implemented by a processor in the terminal calling a program code, where the program code may be stored in a computer storage medium.
Fig. 1 to 4 are schematic diagrams illustrating a flow of implementing a database access method according to an embodiment of the present invention, as shown in fig. 1 to 4, the method includes:
step S101, a terminal starts a timer to start timing;
here, the timer is preset, and the timer is set in relation to averageTime of the server (which may be a master node), and as a preferred embodiment, the timer may be set to averageTime; of course, it is also possible to modify averageTime according to a certain rule, and then use the modified value as the timing time, where the modification may be to add or subtract a numerical value to averageTime, or to multiply averageTime by a coefficient (between 0 and 1).
Step S102, the terminal judges whether a task of accessing a database is obtained in the process of starting timing of the timer, and a first judgment result is obtained;
here, the task is an access task to data stored in a database;
step S103, when the first judgment result shows that the task of accessing the database is not obtained when the timer is timed to end, the terminal sends an access request to the server;
here, the access request is used for requesting first identification information in a meta table, and the first identification information is used for indicating whether the terminal needs to update the local meta table; the first identification information has at least two kinds of information (namely, first information and second information), wherein the first information can be used for indicating that the terminal needs to update the local meta table, and the second information can be used for indicating that the terminal does not need to update the local meta table. The first identification information may be set by a person skilled in the art, for example, the first information may be 0 or 1 or Y (indicating Yes) or n (no), and details of how to set the first identification information are not described herein.
In the specific implementation process, as a preferred embodiment, the first identification information may be represented by a flag bit, for example, the updateFlag mentioned in the above embodiment, when the flag bit of the first identification information may be represented by 1 bit (bit), for example, when the updateFlag is 1, the first identification information is a true value, and the terminal needs to update a local meta table; when updateFlag is 0, it indicates that the second information is a false value, and the terminal does not need to update the local meta table.
Step S104, the terminal receives an access response sent by the server, wherein the access response carries first identification information;
step S105, when the first identification information indicates that the terminal needs to update the local meta table, the terminal updates the local meta table.
In the embodiment of the present invention, the method further includes:
step S106, when the first judgment result shows that the task for accessing the database is obtained when the timer is timed to finish, the terminal reads a locally stored root table and a locally stored meta table;
step S107, the terminal queries a root table and a meta table according to the task to obtain an access target;
and step S108, the terminal processes the user data in the server according to the access target.
Here, the processing includes at least deletion, addition, and modification of user data, and the like.
In the embodiment of the invention, a terminal locally stores a meta table and a root table, and the method comprises the following steps: the terminal starts a timer to start timing; the terminal judges whether a task for accessing the database is obtained or not in the process of starting timing by the timer to obtain a first judgment result; when the first judgment result shows that the task of accessing the database is not obtained when the timer is timed to end, the terminal sends an access request to the server, wherein the access request is used for requesting first identification information in the meta table, and the first identification information is used for showing whether the terminal needs to update the local meta table; the terminal receives an access response sent by the server, wherein the access response carries first identification information; when the first identification information indicates that the terminal needs to update the local meta table, the terminal updates the local meta table; in this way, the speed at which the terminal can access the database is enabled.
Example two
Based on the foregoing embodiment, an embodiment of the present invention provides a database access method, where the method is applied to a terminal, where the terminal locally stores a meta table and a root table, and fig. 2 is a schematic diagram of an implementation flow of the database access method according to the embodiment of the present invention, and as shown in fig. 2, the method includes:
step S201, when the information in the meta table changes, the server updates the parameter value in the attribute of the meta table;
here, the parameters in the meta table attribute include parameters newly added in the above embodiment, such as changeCount, changePercent, averageTime, updateFlag, and Threshold, in addition to the parameters in the prior art.
Here, the server in the embodiment of the present invention may be a master node.
Step S202, the server judges whether the parameter value meets a preset condition or not to obtain a second judgment result;
here, taking changeCount and changePercent as examples, two ways of implementing step S202 are provided.
In a first manner, the parameter in the meta table attribute includes an update frequency (changeCount), and correspondingly, the determining, by the server, whether the parameter value satisfies a preset condition includes:
the server judges whether the updating times is larger than a preset first threshold value or not, determines that the parameter value meets a preset condition when the updating times is larger than the first threshold value, and determines that the parameter value does not meet the preset condition when the updating times is smaller than or equal to the first threshold value.
In a second manner, the parameter in the meta table attribute includes an update percentage (changePercent), and correspondingly, the determining, by the server, whether the parameter value satisfies a preset condition includes:
the server judges whether the updating percentage is larger than a preset second threshold value, determines that the parameter value meets a preset condition when the updating percentage is larger than the second threshold value, and determines that the parameter value does not meet the preset condition when the updating percentage is smaller than or equal to the second threshold value.
Here, assuming that another percentage parameter exists in the meta table attribute, where the percentage parameter is used to indicate a percentage of unchanged areas in the meta table, the server determines whether the parameter value satisfies a preset condition, including: the server judges whether the updating percentage is smaller than a preset third threshold value, determines that the parameter value meets a preset condition when the updating percentage is smaller than the second threshold value, and determines that the parameter value does not meet the preset condition when the updating percentage is larger than or equal to the second threshold value.
Step S203, when the second determination result indicates that the parameter value satisfies a preset condition, the server updates the first identification information of the meta table to first information;
here, the first information indicates that the terminal needs to update the locally stored meta table.
Step S204, the terminal starts a timer to start timing;
step S205, the terminal judges whether a task for accessing the database is obtained in the process of starting timing by the timer, and a first judgment result is obtained;
here, the task is an access task to data stored in a database;
step S206, when the first judgment result shows that the task of accessing the database is not obtained when the timer is timed to end, the terminal sends an access request to the server;
here, the steps S204 to S206 correspond to the steps S101 to S103 in the first embodiment, respectively, and therefore, a person skilled in the art can understand the steps S204 to S206 with reference to the first embodiment, and details are not described herein for brevity.
Step S207, the server receives an access request sent by the terminal;
here, the access request is used for requesting first identification information in a meta table, and the first identification information is used for indicating whether the terminal needs to update the local meta table;
step S208, the server searches a meta table according to the access request to obtain first identification information of the meta table, and the first identification information in the meta table is carried in an access response;
step S209, the server sends the access response to the terminal.
Step S210, the terminal receives the access response sent by the server;
here, the access response carries first identification information;
step S211, when the first identification information indicates that the terminal needs to update the local meta table, the terminal updates the local meta table.
Here, the steps S209 to S211 correspond to the steps S104 to S105 in the first embodiment, respectively, so that those skilled in the art can understand the steps S209 to S211 with reference to the first embodiment, and details are not described herein for brevity.
Step S211, the terminal sends an updating request to a server;
here, the update request is used for requesting a meta table, and the first identification information of the requested meta table indicates that a local meta table needs to be updated; the access request and the updating request carry identification information of a meta table; correspondingly, the access response carries first identification information corresponding to the identification information of the element table, and the update response carries the element table corresponding to the identification information of the element table.
Step S212, the server receives an updating request sent by the terminal, and acquires a meta table according to the identification information of the meta table in the updating request;
step S213, the server carries the obtained meta table in an update response and sends the update response to the terminal;
step S214, the terminal receives the update response sent by the server;
here, the update response carries an updated meta table corresponding to the first identification information;
step S215, the terminal updates the updated meta table sent by the server to the local.
In the embodiment of the present invention, the method further includes:
step S216, when the first judgment result shows that the task for accessing the database is obtained when the timer is timed to end, the terminal reads a locally stored root table and a locally stored meta table;
step S217, the terminal queries the root table and the element table according to the task to obtain an access target;
and step S218, the terminal processes the user data in the server according to the access target.
Here, the processing includes at least deletion, addition, and modification of user data, and the like.
In this embodiment of the present invention, the update response further carries an update time, and the method further includes:
step S220, the terminal sets the timing time of the timer according to the updating time.
In the embodiment of the present invention, the method further includes: and the server updates the update time and/or the current time in the meta table attribute.
EXAMPLE III
Based on the foregoing embodiments, an embodiment of the present invention provides a database access method, and fig. 3 is a schematic flow chart illustrating an implementation of a database access method according to a third embodiment of the present invention, as shown in fig. 3, the method includes:
step S301, the client starts to operate;
here, the client is installed on the terminal, and the terminal operates the client.
Step S302, starting timing;
specifically, a timer is started and starts timing.
Step S303, during the timer timing process, determine whether to obtain an access task? If yes, go to step S304, otherwise, go to step S310;
step S304, reading a local cache;
here, the local cache is a root table and a meta table;
step S305, accessing a target;
specifically, a root table and a meta table are searched according to an access task to obtain a target address, and then a client directly accesses the target address;
step S306, determine whether it is correct? If yes, go to step S308, otherwise, go to step S307;
in step S307, whether n attempts are successful? If yes, the process proceeds to step S308, otherwise, the process proceeds to step S309;
step S308, ending the task;
step S309, updating the target cache, and then returning to step S305;
here, the target cache is a meta table of the local cache.
Step S310, waiting for timing completion;
specifically, waiting for the timer to finish timing;
step S311, accessing a database;
step S312, determine whether updateFlag is True? If no, the process proceeds to step S302, and if yes, the process proceeds to step S313;
step S313, updating all caches;
step S314, the timing time of the timer is set according to averageTime.
Example four
Based on the foregoing method embodiment, an embodiment of the present invention further provides a database access system, where the system includes a terminal and a server, where units such as an opening unit, a first determining unit, a first sending unit, a first receiving unit, and a first updating unit included in the terminal, and modules included in the units can all be implemented by a processor in the terminal; the second updating unit, the second judging unit, the third updating unit, the second receiving unit, the carrying unit, the second sending unit and other units included in the server can be all realized through a processor in the server.
The functions implemented by the processor may also be implemented by specific logic circuits; in the course of a particular embodiment, the processor may be a Central Processing Unit (CPU), a Microprocessor (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like.
Fig. 4 is a schematic structural diagram of a component of a four-database access system according to an embodiment of the present invention, and as shown in fig. 4, the system includes a terminal 410 and a server 420, where:
the terminal 410 includes an opening unit 411, a first determining unit 412, a first sending unit 413, a first receiving unit 414, and a first updating unit 415, wherein:
the starting unit 411 is used for starting a timer to start timing;
the first judging unit 412 is configured to judge whether a task for accessing the database is obtained in a process of starting timing by the timer, so as to obtain a first judgment result;
the first sending unit 413 is configured to, when the first determination result indicates that the task of accessing the database is not obtained when the timer is ended, send an access request to the server by the terminal, where the access request is used to request first identification information in the meta table, and the first identification information is used to indicate whether the terminal needs to update the local meta table;
the first receiving unit 414 is configured to receive an access response sent by the server, where the access response carries first identification information;
the first updating unit 415 is configured to update the local meta table by the terminal when the first identification information indicates that the terminal needs to update the local meta table.
The server 420 includes a second updating unit 421, a second determining unit 422, a third updating unit 423, a second receiving unit 424, a carrying unit 425, and a second sending unit 426, where:
the second updating unit 421, configured to update a parameter value in a meta table attribute when information in the meta table changes;
the second judging unit 422 is configured to judge whether the parameter value meets a preset condition, so as to obtain a second judgment result;
the third updating unit 423 is configured to update the first identifier information of the meta table to first information when the second determination result indicates that the parameter value meets a preset condition, where the first information indicates that the terminal needs to update a locally stored meta table;
the second receiving unit 424 is configured to receive an access request sent by a terminal, where the access request is used to request first identification information in a meta table, and the first identification information is used to indicate whether the terminal needs to update a local meta table;
the carrying unit 425 is configured to search a meta table according to the access request, obtain first identification information of the meta table, and carry the first identification information in the meta table in an access response;
the second sending unit 426 is configured to send the access response to the terminal.
In this embodiment of the present invention, the first updating unit includes a first sending module, a first receiving module, and an updating module, where:
the first sending module is configured to send an update request to the server when the first identifier indicates that the terminal needs to update the local meta table, where the update request is used to request the meta table, and the first identifier of the requested meta table indicates that the local meta table needs to be updated;
the first receiving module is configured to receive an update response sent by the server, where the update response carries an updated meta table corresponding to the first identifier information;
and the updating module is used for updating the updated meta table sent by the server to the local.
In the embodiment of the present invention, the terminal further includes a reading unit, an inquiring unit, and a processing unit, where:
the reading unit is used for reading the locally stored root table and the locally stored element table by the terminal when the first judgment result shows that the task for accessing the database is obtained when the timer is timed to finish;
the query unit is used for querying the root table and the element table according to the task to obtain an access target;
and the processing unit is used for processing the user data in the server according to the access target.
In the embodiment of the present invention, the update response further carries an update time, and the terminal further includes a setting unit, configured to set the timing time of the timer according to the update time.
In this embodiment of the present invention, the parameter in the meta table attribute includes an update time, and correspondingly, the second determining unit is configured to determine whether the update time is greater than a preset first threshold, so as to obtain a second determination result.
In an embodiment of the present invention, the parameter in the meta table attribute includes an update percentage, and correspondingly, the second determining unit is configured to determine whether the update percentage is greater than a preset second threshold, and determine that the parameter value satisfies a preset condition when the update percentage is greater than the second threshold.
In this embodiment of the present invention, the server further includes a fourth updating unit, configured to update the update time and/or the current time in the meta table attribute.
As previously mentioned, distributed big data storage databases that have been widely used today include HadoopDatabase (Hbase), Google Bigtable, NoSQL, and so on. Hbase is the technology with the highest application rate in the current big data platform, but the product scheme of Hbase has the following defects and shortcomings: 1) and the target data is positioned through three times of accesses, so that the access delay is large. 2) The client updates the local cache in a trial and error mode, and the efficiency is low. 3) There is no relatively complete local cache synchronization mechanism, and the access cost becomes high under the condition of rapid change of data volume. 4) The problem of how to reduce the load on the database when caching the updated data is not solved.
The technical scheme provided by the embodiment of the invention realizes the transformation of several aspects: 1) the method provides a solution for optimizing the local cache on the basis of the original three-time access positioning mechanism, and the synchronization efficiency of the local cache and the remote database is improved by means of the cache automatic updating mechanism, so that reliable local cache data are provided for each access, data can be read and written through one access under most conditions, the access amount is reduced to the maximum extent, the access time is shortened, and the database load is reduced. 2) And adding a new attribute responsible for the cache updating of the client on the basis of the meta table. Through the attributes, the client can quickly and accurately know the current change situation of the database, so that an optimization decision can be made. 3) A low-load cache updating mechanism is designed at a client, the local cache is automatically updated by using the idle time of the client, the updating mechanism considers the operation condition of the database, dynamically adjusts the updating time according to the data growth rate of the database, and meanwhile, can adapt to the special requirements of a user to update the local cache, and has higher flexibility.
Based on the advantages, the scheme provided by the embodiment of the invention can effectively improve the reading and writing speed of the distributed database, is suitable for the management and maintenance of the database in a big data environment, and has high application value.
Here, it should be noted that: the above description of the system embodiment is similar to the above description of the method embodiment, and has similar beneficial effects to the method embodiment, and therefore, the description thereof is omitted. For technical details that are not disclosed in the embodiment of the system of the present invention, please refer to the description of the embodiment of the method of the present invention for understanding, and therefore, for brevity, will not be described again.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention. The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (15)
1. An access method for a database, wherein a terminal locally stores a meta table, and the meta table is used for recording an area of user data in the database, the method comprising:
starting a timer to start timing;
judging whether a task for accessing the database is obtained or not in the process of starting timing of the timer to obtain a first judgment result;
when the first judgment result shows that the task for accessing the database is not obtained when the timer is timed to end, sending an access request to a server, wherein the access request is used for requesting first identification information in a meta table;
receiving an access response sent by the server, wherein the access response carries first identification information;
when the first identification information indicates that the local meta table needs to be updated, sending an update request to the server, wherein the update request is used for requesting the meta table, and the first identification information of the requested meta table indicates that the local meta table needs to be updated;
receiving an update response sent by the server, wherein the update response carries an updated meta table corresponding to the first identification information and update time, and the update time is average time required for achieving a condition of updating a local meta table and is used for representing a data growth rate of the database;
updating the updated meta table sent by the server to the local;
and resetting the timing time of the timer according to the updating time.
2. The method of claim 1, further comprising:
when the first judgment result shows that the task for accessing the database is obtained when the timer is timed to finish, reading a root table and a meta table which are locally stored;
inquiring the root table and the element table according to the task to obtain an access target;
and processing the user data in the server according to the access target.
3. The method according to claim 1, wherein the access request and the update request carry identification information of a meta table; correspondingly, the access response carries first identification information corresponding to the identification information of the element table, and the update response carries the element table corresponding to the identification information of the element table.
4. A method for accessing a database, the method comprising:
when the information in the meta table changes, updating the parameter values in the meta table attributes; the meta table is used for recording the area of user data in the database; the parameter value is used for recording the change condition of the element table;
judging whether the parameter value meets a preset condition or not to obtain a second judgment result;
when the second judgment result shows that the parameter value meets the preset condition, updating the first identification information of the meta table into first information, wherein the first information shows that the terminal needs to update the locally stored meta table;
the method further comprises the following steps:
receiving an update request sent by the terminal, wherein the update request is used for requesting a meta table;
acquiring a meta table and an updating time according to first identification information of the meta table in the updating request, wherein the updating time is the average time required by the condition of updating the local meta table and is used for the data growth rate of the database;
and carrying the obtained meta table and the updating time in an updating response, and sending the updating response to the terminal.
5. The method according to claim 4, wherein the parameters in the meta table attribute include the number of updates, and correspondingly, the determining whether the parameter values satisfy a preset condition includes:
and judging whether the updating times are greater than a preset first threshold value or not, and determining that the parameter value meets a preset condition when the updating times are greater than the first threshold value.
6. The method of claim 4, wherein the parameters in the meta table attribute include an update percentage, and correspondingly, the determining whether the parameter values satisfy a preset condition includes:
and judging whether the updating percentage is larger than a preset second threshold value, and determining that the parameter value meets a preset condition when the updating percentage is larger than the second threshold value.
7. The method according to any one of claims 4 to 6, further comprising:
receiving an access request sent by the terminal, wherein the access request is used for requesting first identification information in a meta table, and the first identification information is used for indicating whether the terminal needs to update a local meta table;
searching a meta table according to the access request to obtain first identification information of the meta table, and carrying the first identification information in the meta table in an access response;
and sending the access response to the terminal.
8. The method according to any one of claims 4 to 6, further comprising: and updating the updating time and/or the current time in the meta table attribute.
9. A terminal is characterized in that a terminal locally stores a meta table, the meta table is used for recording an area of user data in a database, the terminal comprises an opening unit, a first judging unit, a first sending unit, a first receiving unit and a first updating unit, wherein:
the starting unit is used for starting a timer to start timing;
the first judging unit is used for judging whether a task for accessing the database is obtained in the process of starting timing of the timer to obtain a first judging result;
the first sending unit is configured to send an access request to a server when the first determination result indicates that a task for accessing the database is not obtained when the timer is timed to end, where the access request is used to request first identification information in a meta table;
the first receiving unit is configured to receive an access response sent by the server, where the access response carries first identification information;
the first updating unit is used for sending an updating request to the server when the first identification information indicates that the terminal needs to update the local meta table, wherein the updating request is used for requesting the meta table, and the first identification information of the requested meta table indicates that the local meta table needs to be updated; receiving an update response sent by the server, wherein the update response carries an updated meta table corresponding to the first identification information and update time, and the update time is average time required for achieving a condition of updating a local meta table and is used for representing a data growth rate of the database; updating the updated meta table sent by the server to the local; and resetting the timing time of the timer according to the updating time.
10. The terminal according to claim 9, characterized in that the terminal further comprises a reading unit, a querying unit and a processing unit, wherein:
the reading unit is used for reading a locally stored root table and a locally stored meta table when the first judgment result shows that the task for accessing the database is obtained when the timer is timed to finish;
the query unit is used for querying the root table and the element table according to the task to obtain an access target;
and the processing unit is used for processing the user data in the server according to the access target.
11. A server is characterized by comprising a second updating unit, a second judging unit, a third updating unit, a second receiving unit, a carrying unit and a second sending unit, wherein:
the second updating unit is used for updating the parameter values in the attribute of the meta table when the information in the meta table changes; the meta table is used for recording the area of user data in the database; the parameter value is used for recording the change condition of the element table;
the second judging unit is used for judging whether the parameter value meets a preset condition or not to obtain a second judging result;
the third updating unit is configured to update the first identifier information of the meta table to first information when the second determination result indicates that the parameter value meets a preset condition, where the first information indicates that the terminal needs to update the locally stored meta table;
the second receiving unit is configured to receive an update request sent by the terminal, where the update request is used to request a meta table, and obtain the meta table and an update time according to first identification information of the meta table in the update request, where the update time is an average time required for reaching a condition of updating a local meta table and is used for a data growth rate of the database;
the carrying unit is used for carrying the acquired meta table and the updating time in the updating response;
and the second sending unit is used for sending the updating response to the terminal.
12. The server according to claim 11, wherein the parameter in the meta table attribute includes an update time, and correspondingly, the second determining unit is configured to determine whether the update time is greater than a preset first threshold, so as to obtain a second determination result.
13. The server according to claim 11, wherein the parameter in the meta table attribute includes an update percentage, and correspondingly, the second determining unit is configured to determine whether the update percentage is greater than a preset second threshold, and determine that the parameter value satisfies a preset condition when the update percentage is greater than the second threshold.
14. The server according to any one of claims 12 to 13,
the second receiving unit is further configured to receive an access request sent by the terminal, where the access request is used to request first identification information in a meta table, and the first identification information is used to indicate whether the terminal needs to update a local meta table;
the carrying unit is further configured to search a meta table according to the access request, obtain first identification information of the meta table, and carry the first identification information in the meta table in an access response;
the second sending unit is further configured to send the access response to the terminal.
15. The server according to any one of claims 11 to 13, wherein the server further comprises a fourth updating unit configured to update the update time and/or the current time in the meta table attribute.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510591099.1A CN106549983B (en) | 2015-09-16 | 2015-09-16 | Database access method, terminal and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510591099.1A CN106549983B (en) | 2015-09-16 | 2015-09-16 | Database access method, terminal and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106549983A CN106549983A (en) | 2017-03-29 |
| CN106549983B true CN106549983B (en) | 2020-03-31 |
Family
ID=58361821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510591099.1A Active CN106549983B (en) | 2015-09-16 | 2015-09-16 | Database access method, terminal and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106549983B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108512930B (en) * | 2018-04-11 | 2021-02-09 | 广州视源电子科技股份有限公司 | Shared file management method, device, server and storage medium |
| CN113495847B (en) * | 2020-04-04 | 2023-08-18 | 厦门网宿有限公司 | Storage space recycling method, system and computer storage medium |
| CN111638948B (en) * | 2020-06-03 | 2023-04-07 | 重庆银行股份有限公司 | Multi-channel high-availability big data real-time decision making system and decision making method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101296507A (en) * | 2007-04-29 | 2008-10-29 | 华为技术有限公司 | Method and system and server for implementing list information synchronization |
| CN101645802A (en) * | 2008-08-04 | 2010-02-10 | 华为技术有限公司 | Method and device for controlling contents |
| CN102546664A (en) * | 2012-02-27 | 2012-07-04 | 中国科学院计算技术研究所 | User and authority management method and system for distributed file system |
| CN103294675A (en) * | 2012-02-23 | 2013-09-11 | 上海盛霄云计算技术有限公司 | Method and device for updating data in distributed storage system |
| CN103441997A (en) * | 2013-08-20 | 2013-12-11 | 华为技术有限公司 | Content sharing method, device and system |
| CN104281506A (en) * | 2014-07-10 | 2015-01-14 | 中国科学院计算技术研究所 | Data maintenance method and system for file system |
| CN104750740A (en) * | 2013-12-30 | 2015-07-01 | 北京新媒传信科技有限公司 | Data renewing method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8082273B2 (en) * | 2007-11-19 | 2011-12-20 | Teradata Us, Inc. | Dynamic control and regulation of critical database resources using a virtual memory table interface |
| CN102724297A (en) * | 2012-05-25 | 2012-10-10 | 上海传漾网络科技有限公司 | Method for increasing access speed in browser/server (BS) architecture advertisement management system |
| CN103916373A (en) * | 2013-01-07 | 2014-07-09 | 腾讯科技（深圳）有限公司 | Data updating method, server, client side and system |
| CN104426947A (en) * | 2013-08-27 | 2015-03-18 | 中兴通讯股份有限公司 | Terminal intelligent updating method and system, terminal and data updating server |
| CN104331494B (en) * | 2014-11-17 | 2018-12-11 | 北京国双科技有限公司 | A kind of method and system of more new data |
-
2015
- 2015-09-16 CN CN201510591099.1A patent/CN106549983B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101296507A (en) * | 2007-04-29 | 2008-10-29 | 华为技术有限公司 | Method and system and server for implementing list information synchronization |
| CN101645802A (en) * | 2008-08-04 | 2010-02-10 | 华为技术有限公司 | Method and device for controlling contents |
| CN103294675A (en) * | 2012-02-23 | 2013-09-11 | 上海盛霄云计算技术有限公司 | Method and device for updating data in distributed storage system |
| CN102546664A (en) * | 2012-02-27 | 2012-07-04 | 中国科学院计算技术研究所 | User and authority management method and system for distributed file system |
| CN103441997A (en) * | 2013-08-20 | 2013-12-11 | 华为技术有限公司 | Content sharing method, device and system |
| CN104750740A (en) * | 2013-12-30 | 2015-07-01 | 北京新媒传信科技有限公司 | Data renewing method and device |
| CN104281506A (en) * | 2014-07-10 | 2015-01-14 | 中国科学院计算技术研究所 | Data maintenance method and system for file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106549983A (en) | 2017-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109947668B (en) | Method and device for storing data | |
| US11561930B2 (en) | Independent evictions from datastore accelerator fleet nodes | |
| CN101655892A (en) | Mobile terminal and access control method | |
| CN103236989B (en) | Buffer control method in a kind of content distributing network, equipment and system | |
| CN104794228A (en) | Search result providing method and device | |
| CN110427386B (en) | Data processing method, device and computer storage medium | |
| WO2019057193A1 (en) | Data deletion method and distributed storage system | |
| CN105159845A (en) | Memory reading method | |
| CN104182435A (en) | System and method for searching information based on data missing mark | |
| US11507277B2 (en) | Key value store using progress verification | |
| CN114116613A (en) | Metadata query method, equipment and storage medium based on distributed file system | |
| CN106549983B (en) | Database access method, terminal and server | |
| CN106790422A (en) | A kind of data buffer storage cluster and data retrieval method for WEB application | |
| CN111737168A (en) | Cache system, cache processing method, device, equipment and medium | |
| CN104035925A (en) | Data storage method and device and storage system | |
| CN108154024B (en) | Data retrieval method and device and electronic equipment | |
| CN106484694B (en) | Full-text search method and system based on distributed data base | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN111597259A (en) | Data storage system, method, device, electronic equipment and storage medium | |
| US10146833B1 (en) | Write-back techniques at datastore accelerators | |
| CN114048186A (en) | Data migration method and system based on mass data | |
| CN116541427B (en) | Data query method, device, equipment and storage medium | |
| WO2016078388A1 (en) | Data aging method and apparatus | |
| WO2022062777A1 (en) | Data management method, data management apparatus, and storage medium | |
| CN108959468A (en) | A kind of monitoring method of data base directory, storage medium and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |