CN111506592A

CN111506592A - Method and device for upgrading database

Info

Publication number: CN111506592A
Application number: CN202010315231.7A
Authority: CN
Inventors: 陈创; 陈云卿
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Cloud Computing Changsha Co Ltd
Priority date: 2020-04-21
Filing date: 2020-04-21
Publication date: 2020-08-07
Anticipated expiration: 2040-04-21
Also published as: CN111506592B

Abstract

The invention provides a method and a device for upgrading a database, electronic equipment and a computer-readable storage medium; the method comprises the following steps: acquiring new version data of the meta-information; writing the new version data into a data record corresponding to the meta-information in the database, wherein the data record comprises old version data corresponding to the meta-information; during the writing of the new version data into the data record, responding to data operation aiming at the meta-information according to the old version data in the data record, and synchronously updating the result of the data operation to the new version data in the data record; and when the writing of the new version data is finished, responding to the data operation aiming at the meta information according to the new version data in the data record. The invention can ensure that the service is provided uninterruptedly during the updating period of the database.

Description

Method and device for upgrading database

Technical Field

The present invention relates to the field of database technologies, and in particular, to a method and an apparatus for upgrading a database, an electronic device, and a computer-readable storage medium.

Background

The Database (Database) is a data carrier for various services in the internet, the types of the internet services are more and more varied, and the data volume of each service is larger and larger, so that the size of the Database is larger and larger, and difficulty is brought to smooth upgrade of the Database.

In the related art, when data in a database is updated, in order to ensure consistency of the data, a write operation on the data in the database needs to be suspended, and accordingly, the database stops providing services to the outside. However, since the number of user groups to which the database is oriented is large and the time zones of the areas where the user groups are located are various, once the database stops providing services to the outside during the updating period, the use experience of a large number of users is affected.

Disclosure of Invention

Embodiments of the present invention provide a method and an apparatus for upgrading a database, an electronic device, and a computer-readable storage medium, which can ensure that a service is provided uninterruptedly during an update period of the database.

The technical scheme of the embodiment of the invention is realized as follows:

the embodiment of the invention provides a method for upgrading a database, which comprises the following steps:

acquiring new version data of the meta-information;

writing the new version data into a data record corresponding to the meta-information in the database, wherein the data record comprises old version data corresponding to the meta-information;

during writing of the new version data to the data record, responding to data operations directed to the meta information in accordance with the old version data in the data record, and

synchronously updating the result of the data operation to the new version data in the data record;

and when the writing of the new version data is finished, responding to the data operation aiming at the meta information according to the new version data in the data record.

The embodiment of the invention provides an upgrading device of a database, which comprises:

the control management service module is used for acquiring new version data of the meta-information; writing the new version data into a data record corresponding to the meta-information in the database, wherein the data record comprises old version data corresponding to the meta-information;

a retrieval service module for responding to data operations for the meta-information according to the old version data in the data record during writing the new version data to the data record;

the database agent module is used for synchronously updating the result of the data operation to the new version data in the data record;

and the retrieval service module is also used for responding to the data operation aiming at the meta-information according to the new version data in the data record when the writing of the new version data is completed.

In the above scheme, the meta information includes the number of pages in the library and the object identifier; the old version data is extracted from the source data corresponding to the object identification according to an old data model; the control management service module is further configured to traverse each data record in the database to perform the following operations: searching an object identifier in the data record in a storage page corresponding to the page number in the database of the data record; searching corresponding source data in the database according to the searched object identifier; and extracting new version data from the searched source data according to the new data model.

In the above scheme, the control management service module is further configured to search, in a data record of the database corresponding to the meta information, a standby data directory for storing the new version data; and writing the new version data into the standby data directory.

In the above solution, the type of the data operation for the meta information includes: a data increment operation; data deletion operation; data modification operation; data query operation; the retrieval service module is further configured to perform a data operation of a corresponding type on the old version data of the meta-information according to a type to which the data operation for the meta-information belongs, so as to obtain a corresponding result.

In the above scheme, the database agent module is further configured to, when the type of the data operation for the meta information is a data adding operation, a data deleting operation, or a data modifying operation, synchronously update a result of the corresponding data operation to the new version data in the data record.

In the above solution, the data record includes a status bit, where the status bit is used to indicate a data version of the data record used for responding to the data operation; the retrieval service module is further configured to uniformly update a status bit in each data record in the database to indicate the new version data to respond to the data operation; and when the updating of the status bit of any data record in the database fails, the status bit of each data record is rolled back to the position before updating.

In the above scheme, the control management service module is further configured to, in response to a rollback request for the meta information, stop writing the new version data into the data record corresponding to the meta information in the database, and delete the new version data already written into the data record; responding to data operations directed to the meta-information in accordance with the old version data in the data record.

In the foregoing solution, the retrieval service module is configured to, during writing the new version data into the data record, synchronize, when receiving a data operation for the meta information, the data record in the database corresponding to the meta information into a cache of a retrieval service, and synchronize a result of the data operation into a data record stored in the cache of the retrieval service.

In the above scheme, the retrieval service module is further configured to delete the new version data in the cache of the retrieval service when receiving a rollback request for the meta information; and deleting the old version data corresponding to the meta information in the cache of the retrieval service when the database completes the writing of the new version data corresponding to the meta information.

In the above scheme, the retrieval service module is further configured to, when a hit rate of the cache of the retrieval service is lower than a hit rate threshold or a version of data stored in the cache of the retrieval service is lower than a version threshold, respond to a data operation for the meta information, load a data record corresponding to the meta information to the cache of the retrieval service, and synchronize a result of the data operation in the loaded data record; responding to a new data operation for the meta-information according to the data record stored in the cache of the retrieval service.

An embodiment of the present invention provides an electronic device, including:

a memory for storing executable instructions;

and the processor is used for realizing the upgrading method of the database provided by the embodiment of the invention when the executable instructions stored in the memory are executed.

The embodiment of the invention provides a computer-readable storage medium, which stores executable instructions and is used for causing a processor to execute the executable instructions so as to realize the database upgrading method provided by the embodiment of the invention.

The embodiment of the invention has the following beneficial effects:

by maintaining the data of the new version and the old version, the data operation can still be responded according to the data of the old version during the upgrading period, so that the uninterrupted service provision during the updating period of the database can be ensured; and when the updating is completed and the data operation is responded by the new version data, the data operation result is synchronously written into the new version data, so that the effectiveness of the data operation is ensured, and seamless smooth upgrading is realized.

Drawings

FIG. 1 is a schematic diagram of an architecture of a system for upgrading a database according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a server 500 according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of an upgrading method for a database according to an embodiment of the present invention;

fig. 4 is a schematic flowchart of an upgrading method for a database according to an embodiment of the present invention;

fig. 5 is a schematic flowchart of an upgrading method for a database according to an embodiment of the present invention;

fig. 6 is a schematic flowchart of an upgrading method for a database according to an embodiment of the present invention;

fig. 7 is an application diagram of an upgrade method for a database according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of an architecture of a system for upgrading a database according to an embodiment of the present invention;

fig. 9A, 9B and 9C are schematic views of storage structures of feature data provided by an embodiment of the present invention;

fig. 10 is a flowchart illustrating a method for upgrading a database according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.

1) In short, the database can be regarded as an electronic storage system, and a user can add, query, update, delete and modify data in a file. A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.

2) The non-stop service means that the external retrieval service and the adding, deleting and modifying service are uninterrupted. During database updates, the user does not experience a noticeable condition of service unavailability.

3) Data update, or data upgrade, is to upgrade features in a database from version data supported by an old data model to version data supported by a new data model, generally incompatible changes of data.

4) Data Record (Data Record) refers to a complete set of related information corresponding to a line of information in a Data source, and includes meta information and Data.

5) The data directory (Index), which is a special data structure that holds a small portion of the data of a data set, holds the values of member variables (fields) or multiple fields in a specified class.

6) Rollback (Rollback), refers to the deletion of updates performed by one or more partially completed transactions. To ensure that the integrity of the database is restored after an application, database, or system error, rollback is required to restore the program or data to the last correct state.

7) Persistence, which is the mechanism by which program data is transitioned between a persistent state and a transient state. Colloquially, transient data (e.g., data in memory that cannot be permanently stored) is persisted as persistent data (e.g., persisted to a database that can be stored for a long time).

8) In response to the condition or state on which the performed operation depends, one or more of the performed operations may be in real-time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.

9) A Block chain (Blockchain) is a storage structure for encrypted, chained transactions formed from blocks (blocks).

10) A Blockchain Network (Blockchain Network) incorporates new blocks into a set of nodes of a Blockchain in a consensus manner.

11) Ledger (L edge), which is a collective term for blockchains (also called ledger data) that record transactions in the form of files in a file system, and state databases that are synchronized with blockchains that record transactions in blockchains in the form of different types of Key (Key) Value (Value) pairs for supporting fast queries of transaction data in blockchains.

12) Intelligent Contracts (Smart Contracts), also known as chain codes (chaincodes) or application codes, are programs deployed in nodes of a blockchain network, and the nodes execute the intelligent Contracts called in received transactions to perform operations of updating or querying key-value data of a state database.

13) Consensus (Consensus), a process in a blockchain network, is used to agree on a transaction in a block between the nodes involved, the agreed block to be appended to the end of the blockchain and used to update the state database.

Embodiments of the present invention provide a method and an apparatus for upgrading a database, an electronic device, and a computer-readable storage medium, which can ensure that a service is provided uninterruptedly during an update period of the database. An exemplary application of the method for upgrading a database according to the embodiment of the present invention is described below, and the electronic device according to the embodiment of the present invention may be implemented as a server.

The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, big data and artificial intelligence platform.

In the following, an exemplary application of the electronic device provided by the embodiment of the present invention implemented as a server will be described. Referring to fig. 1, fig. 1 is a schematic architecture diagram of an upgrade system for a database according to an embodiment of the present invention. The upgrading system of the database comprises: the control management service 100, the search service 200, the database agent 300, and the database 400 will be described separately.

The control management service 100 is configured to acquire new version data of the meta information, and write the new version data into a data record of the corresponding meta information in the database 400 through the database agent 300; and is also used for responding to the data adding and deleting operations (including data adding operation, data deleting operation or data modifying operation) of the user according to the old version data during the writing of the new version data, and synchronously updating the results of the data adding and deleting operations of the user to the new version data in the database 400 and the cache of the retrieval service 200.

The retrieval service 200 includes a cache for storing data for responding to a data query operation of a user according to an old version data stored in the database 400 or its own cache during the control management service 100 writes a new version data into a data record of corresponding meta information in the database 400; and is also used for responding to the data query operation of the user according to the new version data stored in the database 400 or the cache of the control management service 100 when the control management service completes writing the new version data.

The database agent 300 serves as an intermediary between the control management service 100 and the database 400, and between the search service 300 and the database 400.

The database 400 includes data records, wherein the data records include meta information, and old version data and new version data corresponding to the meta information.

The interaction process among the control management service 100, the search service 200, the database agent 300, and the database 400 in the database upgrade system will be described in detail below with reference to flowcharts, and will not be described herein again.

It should be noted that the control management service 100, the search service 200, the database agent 300, and the database 400 in the above-mentioned database upgrading system may be integrated in the same server; or may be integrated into multiple (i.e., at least 2) servers to form a server cluster, depending on the functionality implemented.

Next, the configuration of the server will be described by taking an example in which the control management service 100, the search service 200, the database agent 300, and the database 400 in fig. 1 are integrated into the same server. Referring to fig. 2, fig. 2 is a schematic structural diagram of a server 500 according to an embodiment of the present invention, where the server 500 shown in fig. 2 includes: at least one processor 510, memory 530, and at least one network interface 520. The various components in server 500 are coupled together by a bus system 540. It is understood that the bus system 540 is used to enable communications among the components. The bus system 540 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 540 in fig. 2.

The Processor 510 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.

Memory 530 includes volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The non-volatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). Memory 530 described with respect to embodiments of the present invention is intended to comprise any suitable type of memory. Memory 530 may optionally include one or more storage devices physically located remote from processor 510.

In some embodiments, memory 530 may be capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

An operating system 531, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

a network communication module 532 for communicating to other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: bluetooth, wireless compatibility authentication (WiFi), and Universal Serial Bus (USB), etc.;

in some embodiments, the upgrading apparatus for the database provided by the embodiments of the present invention may be implemented in software, and fig. 2 illustrates the upgrading apparatus 533 for the database stored in the storage 530, which may be software in the form of programs and plug-ins, and includes the following software modules: a control management service module 5331, a search service module 5332, and a database agent module 5333, which are logical and thus can be arbitrarily combined or further separated according to the functions implemented. The functions of the respective modules will be explained below.

In other embodiments, the upgrading apparatus for the database provided by the embodiments of the present invention may be implemented in hardware, and as an example, the upgrading apparatus for the database provided by the embodiments of the present invention may be a processor in the form of a hardware decoding processor, which is programmed to execute the upgrading method for the database provided by the embodiments of the present invention, for example, the processor in the form of the hardware decoding processor may employ one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable logic devices (P L D, Programmable L analog devices), Complex Programmable logic devices (CP L D, Complex Programmable L analog devices), Field Programmable Gate arrays (FPGAs, Field-Programmable Gate arrays), or other electronic components.

Next, a method for upgrading a database provided in an embodiment of the present invention will be described as an example, in which the control management service 100, the search service 200, the database agent 300, and the database 400 in fig. 1 are integrated in the same server. Referring to fig. 3, fig. 3 is a schematic flowchart of an upgrading method for a database according to an embodiment of the present invention, and will be described with reference to the steps shown in fig. 3.

In step S101, new version data of meta information is acquired.

Here, the database includes data records, where the data records include meta information, and old version data and new version data corresponding to the meta information. The meta information is information for describing data, and includes database identification, number of pages in the library, object identification, and the like. The data in the database may be features such as human face features, vehicle features, or caricature features.

In some embodiments, the new version data may be obtained externally, for example, new version data of the meta-information may be imported by the operation and maintenance personnel.

In other embodiments, the new version data may be extracted from the source data corresponding to the old version data by: traversing each data record in the database to perform the following operations: searching an object identifier in the data record in a storage page corresponding to the page number in the database of the data record; searching corresponding source data (or called original data) in a database according to the searched object identifier; extracting new version data from the searched source data according to the new data model; and extracting the old version data from the source data corresponding to the object identification according to the old data model.

Taking the data in the database as the face features as an example, the source data is a face image, the old data model is the face feature extraction model of the old version, and the new data model is the face feature extraction model of the new version. Because the input, output and calculation methods of the old version of the face feature extraction model and the new version of the face feature extraction model are different, the new version and the old version of the face feature extraction model have certain difference. The process of acquiring the new version of the face features specifically comprises the following steps: acquiring old version data and corresponding meta-information, wherein the meta-information comprises belonged face database information, the number of pages in a database, personnel identification and face identification; searching a storage page corresponding to the number of pages in a database in correspondence to the face database information; searching personnel information corresponding to the personnel identification in a storage page; searching a target face image corresponding to the face identification in the personnel information; and extracting the face features of the new version from the target face image through the face feature extraction model of the new version.

Taking fig. 1 as an example, the control management service 100 searches the database 400 for source data corresponding to the old version data through the database proxy 300, and extracts new version data from the source data.

According to the embodiment of the invention, the new version data is extracted from the source data corresponding to the old version data according to the new data model, a large amount of resources are not needed to be spent to import the new version data from the outside, and the updating speed and accuracy of the database can be improved.

In step S102, the new version data is written into the data record of the corresponding meta information in the database.

Here, the data record refers to a complete set of related information corresponding to a line of information in the data source, and includes meta information and old version data corresponding to the meta information.

In some embodiments, in the data record of the corresponding meta information of the database, a standby data directory for storing the new version data is searched; the new version data is written into the spare data directory (or Feature Index).

Here, the spare data directory does not include the old version data for responding to the data operation; wherein the data operation comprises at least one of: a data increment operation; data deletion operation; data modification operation; and (5) data query operation.

Following the example of fig. 1, the control and management service 100 writes the new version data into the data record of the corresponding meta information in the database 400 through the database agent 300.

The embodiment of the invention respectively stores the old version data and the new version data in different data directories of the data record, and can be convenient for subsequent switching of the status bit of the data version used for responding to the data operation in the indication data record, thereby ensuring that the change of the status bit is atomic, namely the data versions used for responding to the data operation are the same, and ensuring the consistency of data response.

In step S103, during writing of the new version data to the data record, the data operation for the meta information is responded to according to the old version data in the data record.

Here, the types of data operations for the meta information include: data add operations, data delete operations, data modify operations, and data query operations.

In some embodiments, during writing of the new version data to the data record, a corresponding type of data operation is performed on the old version data of the meta-information to obtain a corresponding result, according to the type to which the data operation for the meta-information belongs.

Following the example of fig. 1, when the type of data operation for the meta-information is a data query operation, during writing of the new version data to the data record, the retrieval service 200 performs the data query operation on the old version data of the meta-information to obtain a corresponding data query result.

Here, the retrieval service includes a cache in which a plurality of data that have responded to the user operation within a response period threshold (for example, 10 days or 20 hours) are stored; or a plurality of data having a response frequency exceeding a response frequency threshold.

For example, after receiving the data query operation for the meta information, the retrieval service 200 may directly perform the data query operation on the old version data in the database 400 through the database proxy 300 to obtain the data query result. Or, first, the old version data corresponding to the meta information is searched in the cache of the retrieval service 200, and when the old version data corresponding to the meta information does not exist in the cache of the retrieval service 200, the database proxy 300 performs a data query operation on the old version data in the database 400 to obtain a data query result; when there is old version data corresponding to the meta information in the cache of the retrieval service 200, a data query operation is directly performed on the old version data corresponding to the meta information in the cache of the retrieval service 200 to obtain a data query result.

As another example accepting fig. 1, when the type of the data operation for the meta information is a data add/delete operation (including a data add operation, a data delete operation, or a data modify operation), the control management service 100, during writing of the new version data to the data record, controls the management service 100 to perform the data add/delete operation on the old version data of the meta information to obtain a result of the corresponding data add/delete operation, and to save the result of the data add/delete operation to the old version data in the data record in the database 400.

According to the embodiment of the invention, different response measures are taken according to different data operation requests, so that the speed of responding to the user request can be improved, and the user experience is improved; and by putting the hot data which can be inquired at any time into the retrieval service, the speed of data inquiry can be improved, and the problem that the service cannot be provided for the user in time due to the fact that the speed of acquiring the corresponding data from the database is too low is avoided.

In step S104, the result of the data operation is acquired, and the result of the data operation is synchronously updated to the new version data in the data record.

When the type of the data operation for the meta information is a data adding operation, a data deleting operation, or a data modifying operation, changes may be caused to the data stored in the database, and therefore, data in the database (including the new version data and the old version data) needs to be synchronized, which may be data that has changed synchronously in real time or data that has changed synchronously at regular intervals, for example, all changed data that has been synchronized once every 10 minutes or 1 hour.

In some embodiments, when the type of the data operation for the meta-information is a data add operation, a data delete operation, or a data modify operation, the corresponding result of the data operation is synchronously updated to the new version data in the data record.

Following the example of fig. 1, the control management service 100 obtains the corresponding results of the data operation and synchronously updates the corresponding results of the data operation to the new version data in the data record in the database 400 through the database proxy 300.

Next, an example of step S103 and step S104 is explained.

As an example, during writing of new version data to a data record, in response to a data addition operation for meta information, additional source data is acquired; extracting the newly added new version data and the newly added old version data from the newly added source data at the same time; and respectively writing the newly added new version data and the newly added old version data in the data records.

As another example, during writing of a new version of data to a data record, modified source data is obtained in response to a data modification operation for the meta-information; extracting the modified new version data and the modified old version data from the modified source data at the same time; and in the data record, replacing the modified new version data with the corresponding unmodified new version data and replacing the modified old version data with the corresponding unmodified old version data.

As yet another example, during writing of new version data to a data record, in response to a data deletion operation for meta-information, the new version data and old version data of the meta-information corresponding to the data deletion operation are deleted in the data record.

As yet another example, during writing of the new version data to the data record, in response to a data query operation for the meta information, identification information of the data to be queried is obtained; and searching the old version data corresponding to the identification information of the data to be inquired in the data record, and returning the old version data corresponding to the identification information of the data to be inquired.

In the embodiment of the invention, during the updating period of the database, the data operation is responded according to the old version data, so that the service can be continuously provided for the user; and by starting a double-write mechanism, not only the result of the data operation is written into the old version data in the data record, but also the corresponding new version data written into the data record is synchronized, so that the problem that the data after the user data operation is not matched with the data stored in the database can be avoided, and the consistency of the data is ensured.

In step S105, when writing of the new version data is completed, a data operation for the meta information is responded according to the new version data in the data record.

Here, the data record includes a status bit for indicating a data version of the data record for responding to the data operation; when the writing of the new version data is finished, uniformly updating the status bit in each data record in the database to indicate the new version data to respond to the data operation; when the update of the status bit of any data record in the database fails, the status bit of each data record is rolled back to the position before the update so as to ensure that the change of the status bit is atomic, that is, the version of the data used in response to the data operation is the same, and the new version data and the old version data cannot be adulterated for use, so as to ensure the consistency of the data.

In some embodiments, when the writing of the new version data is completed, the status bit in each data record in the database is uniformly updated to indicate the new version data to respond to the data operation; in this manner, data operations for the meta-information are responded to in accordance with the new version data in the data record.

As an example, in response to an update operation for the database, status bits in each data record in the database are uniformly updated to indicate a new version of data to respond to the data operation, and after the update of the database is completed, that is, after all status bits are updated, old version data stored in the database is deleted, so as to avoid a situation that no cache data in the database is available after the update fails, and ensure response efficiency of a retrieval service and a control management service.

Following the example of fig. 1, when the writing of the new version data is completed, the control management service 100 responds to the data add/delete operation for the meta information according to the new version data in the data record; the retrieval service 200 responds to data query operations for meta-information based on the new version of data in the data record.

The embodiment of the invention supports one-key updating of the data in the database, the updated data in the database is immediately effective, and the old version data in the database is deleted after the updating is finished, so that the storage resource of the database can be saved.

Next, the implementation process of step S101 to step S105 will be described in a detailed example with reference to the control management service 100, the retrieval service 200, the database agent 300, and the database 400 in fig. 1, referring to fig. 4, where fig. 4 is a flowchart illustrating an upgrading method of a database according to an embodiment of the present invention.

The control management service 100 acquires the new version data and writes the new version data into the database 400 through the database agent 300.

During the writing of the new version data, the control management service 100 responds to the data add/delete operation (including the data add operation, the data modify operation, or the data delete operation) of the user according to the old version data in the data record, obtains the result of the data add/delete operation, and synchronously writes the result of the data add/delete operation into the new version data in the data record in the database 400. The control management service 100 synchronously writes the results of the data add/delete operation into the cache of the retrieval service 200. The retrieval service 200 responds to the data query operation of the user, queries whether relevant data exist in the cache, and returns a data query result when the relevant data exist in the cache; when there is no relevant data in the cache, the database proxy 300 sends a data query operation to the database 400, so that the database 400 responds to the data query operation according to the old version data, obtains a data query result, and returns the data query result to the user.

After the writing of the new version data is completed, the control and management service 100 responds to the data adding and deleting operations (including data adding operations, data modifying operations, or data deleting operations) of the user according to the new version data in the data record, obtains the results of the data adding and deleting operations, and synchronously writes the results of the data adding and deleting operations into the database 400. The control management service 100 synchronously writes the results of the data add/delete operation into the cache of the retrieval service 200. The retrieval service 200 responds to the data query operation of the user, queries whether relevant data exist in the cache, and returns a data query result when the relevant data exist in the cache; when there is no relevant data in the cache, the database agent 300 sends a data query operation to the database 400, so that the database 400 responds to the data query operation according to the new version data, obtains a data query result, and returns the data query result to the user.

Referring to fig. 5, fig. 5 is a flowchart illustrating an upgrading method for a database according to an embodiment of the present invention, based on fig. 3, step S106 may be included after step S104, and step S105 may be replaced with step S107.

In step S106, in response to the rollback request for the meta information, writing of the new version data into the data record of the corresponding meta information in the database is stopped, and the new version data already written into the data record is deleted.

Here, rollback refers to the deletion of updates performed by one or more partially completed transactions.

As an example, when the writing of the new version data is completed, all the new version data that has been written to the data record is deleted in the database in response to a rollback request for the meta information.

As another example, during writing of new version data to a data record, in response to a rollback request for meta-information, writing of the new version data to the data record of the corresponding meta-information in the database is stopped, and all new version data that has been written to the data record is deleted.

Following the example of fig. 1, the control management service 100, in response to the rollback request for the meta information, stops writing the new version data into the data record of the corresponding meta information in the database 400 through the database agent 300, and deletes the new version data in the data record that has been written into the database 400.

In step S107, a response is made to the data operation for the meta information based on the old version data in the data record.

In some embodiments, a status bit is included in the data record, the status bit indicating a version of the data in the data record for responding to the data operation; uniformly rolling back the status bit in each data record in the database to indicate the old version data to respond to the data operation; and when the rollback of the status bit of any data record in the database fails, rolling back the status bit of each data record to the position before the rollback.

Bearing the example of fig. 1, the control management service 100 responds to data add/delete operations for meta information according to old version data in the data record; the retrieval service 200 responds to data query operations for meta-information based on the old version data in the data records.

The embodiment of the invention supports one-key rollback of the data in the database, the data after rollback in the database is immediately effective, and the new version data in the database is deleted after the rollback is finished, so that the storage resources of the database can be saved.

Referring to fig. 6, fig. 6 is a flowchart illustrating an upgrading method for a database according to an embodiment of the present invention, and based on fig. 3, step S108 may be included after step S104.

In step S108, during writing of the new version data to the data record, when a data operation for the meta information is received, the data record of the corresponding meta information in the database is synchronized to the cache of the retrieval service, and the result of the data operation is synchronized to the data record stored in the cache of the retrieval service.

In some embodiments, during writing of new version data to a data record, when a data operation is received for meta-information, the data record including old version data and new version data corresponding to the meta-information is synchronized into a cache of a retrieval service, and an operation result of the data operation is synchronized into the old version data and the new version data of the corresponding data record in the cache.

In some embodiments, when the cache of the retrieval service of the database is started, the data records in the database are not loaded; when the hit rate of the cache of the retrieval service is lower than a hit rate threshold value or the version of data stored in the cache of the retrieval service is lower than a version threshold value, responding to the data operation aiming at the meta-information, loading the data record corresponding to the meta-information into the cache of the retrieval service, and synchronizing the result of the data operation in the loaded data record; responding to a new data operation for the meta-information based on the data records stored in the cache of the retrieval service.

Here, the data to be operated on by the data operation exists in the cache of the retrieval service, which is called Hit (Hit); therefore, the hit rate is the number of data to be operated by the data operation existing in the cache of the search service/the total amount of data in the cache of the search service. The higher the hit rate is, the more times the data stored in the cache of the representation retrieval service is operated by the user, that is, the more effective the data stored in the cache of the retrieval service is; the lower the hit rate, the fewer times the data stored in the cache characterizing the retrieval service is operated by the user, i.e. the more invalid the data stored in the cache characterizing the retrieval service is.

The version threshold refers to a status bit in the database indicating the version of the data in response to the data operation. The version of the data stored in the cache of the retrieval service is lower than the version threshold, which means that if the status bit in the database indicates that the version of the data is N, the data operation is responded, but the version of the data stored in the cache of the retrieval service is lower than N, where N is the version threshold.

The embodiment of the invention stores the result of the data operation into the cache of the retrieval service, ensures that if the subsequent operation is carried out on the same data, the corresponding data can be directly obtained through the cache of the retrieval service for carrying out the data operation without accessing the database again to obtain the data, and improves the efficiency of the data operation.

In some embodiments, after step S108, the method may further include: deleting new version data in the cache of the retrieval service when a rollback request for the meta-information is received; and when the database completes the writing of the new version data of the corresponding meta information, deleting the old version data in the cache of the retrieval service.

Following the example of fig. 1, when the control management service 100 receives a data operation for meta information during writing of a new version of data to a data record, the data record of the corresponding meta information in the database 400 is synchronized into the cache of the retrieval service 200, and the corresponding result of the data operation is synchronized into the data record stored by the cache of the retrieval service 200.

In the embodiment of the invention, the new version data and/or the old version data always exist in the cache of the retrieval service, and when the data in the database is rolled back according to the rolling back request, the state bit in each data record in the database indicates the old version data to respond to the data operation, so that the new version data existing in the cache of the retrieval service is useless, and the new version data needs to be deleted to save the cached resources of the retrieval service. When the database completes writing the new version data of the corresponding meta information, the status bit in each data record in the database indicates the new version data to respond to the data operation, so that the old version data existing in the cache of the retrieval service is useless, and the old version data needs to be deleted to save the cache resources of the retrieval service.

In some embodiments, the database 400 described above may be deployed in a blockchain network such that data acquisition requests for new and old versions of data are responded to by nodes in the blockchain network, each with a copy of the database deployed therein.

Next, an exemplary application of the method for upgrading a database provided by the embodiment of the present invention is described by taking the server 500 in fig. 2 as an example of deploying the database 400 in a blockchain network. Referring to fig. 7, fig. 7 is an application schematic diagram of the database upgrade method provided in the embodiment of the present invention, and includes a blockchain network 600 (exemplarily illustrating a consensus node 610-1 to a consensus node 610-3), an authentication center 700, and a service principal 800/900, which are respectively described below.

The type of blockchain network 600 is flexible and may be, for example, any of a public chain, a private chain, or a federation chain. Taking the public chain as an example, any electronic device (e.g., server 500) of a business entity may access the blockchain network 600 as a client node without authorization; taking a federation chain as an example, after being authorized, a business entity can access the electronic device under its jurisdiction to the blockchain network 600 to become a client node.

By way of example, when blockchain network 600 is a federation chain, business entity 800/900 registers from certificate authority 700 to obtain respective digital certificates that include the public key of the business entity and a digital signature signed by certificate authority 700 for the public key and identity information of business entity 800/900, to be appended to a transaction (e.g., for uplink storage of new version data and old version data, or to query for new version data and old version data) along with the business entity's digital signature for the transaction, and sent to blockchain network 600, for the blockchain network 600 to take the digital certificate and the digital signature from the transaction, verify the authenticity of the transaction (i.e., whether it has not been tampered with) and the identity information of the service entity sending the message, and the blockchain network 600 will verify the identity, for example, whether it has the right to initiate the transaction.

In some embodiments, the client node may act as a mere watcher of the blockchain network 600, i.e., provide support for the business entity to initiate transaction functions, and may be implemented by default or selectively (e.g., depending on the specific business requirements of the business entity) for the functions of the consensus node 610 of the blockchain network 600, such as a ranking function, a consensus service, and an ledger function, etc. Therefore, the data and the service processing logic of the service subject can be migrated to the blockchain network 600 to the maximum extent, and the credibility and traceability of the data and service processing process are realized through the blockchain network 600.

Consensus nodes in blockchain network 600 receive transactions submitted by client nodes from different business entities (e.g., business entity 800/900 shown in fig. 7), perform transactions to update the ledger or query the ledger, and various intermediate or final results of performing transactions may be returned for display in the business entity's client nodes.

An exemplary application of a blockchain network is described below, taking as an example the server 500 uploading new and old version data to the blockchain network for saving, see fig. 7, where the client node 810 may be the server 500.

First, when the client node 810 sets the logic of the uplink of the new version data and the old version data, for example, when the new version data and the old version data are obtained, the client node 810 sends the new version data and the old version data to the blockchain network 600, and generates a corresponding transaction, which includes: the intelligent contract which needs to be called for chaining the new version data and the old version data and the parameters transferred to the intelligent contract; the transaction also includes the client node's 810 digital certificate, signed digital signature, and broadcasts the transaction to the consensus node 610 in the blockchain network 600.

Here, the client node 810 may periodically or in real-time send new and old versions of data into the blockchain network 600. For example, it may be that the new version data and the old version data are sent to the blockchain network 600 whenever they are obtained; or, every certain time period (for example, 2 hours or 3 days), all the new version data and the old version data obtained in the time period are sent to the blockchain network 600, so as to ensure the reliability of obtaining the new version data and the old version data by the blockchain network 600.

Then, when the transaction is received in the consensus node 610 in the blockchain network 600, the digital certificate and the digital signature carried in the transaction are verified, and after the verification is successful, whether the service entity 800 has the transaction right is determined according to the identity of the service entity 800 carried in the transaction, and any verification judgment of the digital signature and the right verification will result in a transaction failure. After verification is successful, the consensus node 610 signs its own digital signature (e.g., by encrypting the digest of the transaction using the private key of node 610-1) and continues to broadcast in the blockchain network 600.

Finally, after the consensus node 610 in the blockchain network 600 receives the transaction that is successfully verified, the transaction is filled into a new block and broadcast. When a new block is broadcasted by the consensus node 610 in the block chain network 600, the new block is verified, for example, whether the digital signature of the transaction in the new block is valid is verified, if the verification is successful, the new block is appended to the tail of the block chain stored in the new block, and the state database is updated according to the transaction result to execute the transaction in the new block: for committed transactions that store the new and old versions of data, key-value pairs that include the new and old versions of data are added to the state database.

An exemplary application of the blockchain network is described by taking the server 510 as an example to query the new version data and the old version data in the blockchain network 600. Referring to fig. 7, a client node 910 may be a server 510.

Here, the server 510 is a server that needs database update, and the data stored in the databases of the server 500 and the server 510 are the same. In this way, the new version data and the old version data required by the server 500 and the server 510 are the same, and therefore, the server 510 can directly acquire the new version data and the old version data through the blockchain network 600, so that the server 510 can be prevented from extracting data from the source data again for acquiring the new version data and the old version data, and further, the resource consumption of the server is saved. Server 510 and server 500 may be the same server, and server 500/510 stores the new version data and the old version data in the blockchain network, which is equivalent to data backup, so that the lost data can be safely and reliably retrieved after the data is lost.

In some embodiments, the types of data that the client node 910 can query in the blockchain network 600 may be implemented by the consensus node 610 by restricting the rights of transactions that the client phase of the business entity can initiate, when the client node 910 has the right to initiate querying of the new and old version data, transactions for querying the new and old version data may be generated by the client node 910 and submitted into the blockchain network 600, transactions are performed from the consensus node 610 to query the corresponding new and old version data from the state database, and returned to the client node 910.

Continuing with fig. 2, an exemplary structure of the upgrading apparatus 533 for the database provided by the embodiment of the present invention implemented as a software module is described below, and in some embodiments, as shown in fig. 2, the software module stored in the upgrading apparatus 533 for the database in the memory 530 may include: a control management service module 5331, a search service module 5332, and a database agent module 5333.

The control management service module 5331 is configured to obtain new version data of the meta information; writing the new version data into a data record corresponding to the meta-information in the database, wherein the data record comprises old version data corresponding to the meta-information;

a retrieval service module 5332 configured to respond to a data operation for the meta information according to the old version data in the data record during writing the new version data to the data record;

the database agent module 5333 is configured to synchronously update the result of the data operation to the new version data in the data record;

the search service module 5332 is further configured to respond to the data operation for the meta information according to the new version data in the data record when the writing of the new version data is completed.

In some embodiments, the meta-information includes the number of pages in the library and the identification of objects; the old version data is extracted from the source data corresponding to the object identification according to an old data model; the control management service module 5331 is further configured to traverse each data record in the database to perform the following operations: searching an object identifier in the data record in a storage page corresponding to the page number in the database of the data record; searching corresponding source data in the database according to the searched object identifier; and extracting new version data from the searched source data according to the new data model.

In some embodiments, the control management service module 5331 is further configured to search, in a data record of the database corresponding to the meta information, a spare data directory for storing the new version data; and writing the new version data into the standby data directory.

In some embodiments, the type of data operation for the meta-information comprises: a data increment operation; data deletion operation; data modification operation; data query operation; the search service module 5332 is further configured to perform a corresponding type of data operation on the old version data of the meta information according to a type to which the data operation for the meta information belongs, so as to obtain a corresponding result.

In some embodiments, the database agent module 5333 is further configured to, when the type of the data operation for the meta information is a data add operation, a data delete operation, or a data modify operation, synchronously update a result corresponding to the data operation to the new version data in the data record.

In some embodiments, a status bit is included in the data record, the status bit indicating a version of data in the data record used in response to the data operation; the retrieval service module 5332 is further configured to uniformly update the status bit in each data record in the database to indicate the new version data to respond to the data operation; and when the updating of the status bit of any data record in the database fails, the status bit of each data record is rolled back to the position before updating.

In some embodiments, the control management service module 5331 is further configured to, in response to a rollback request for the meta information, stop writing the new version data into the data record corresponding to the meta information in the database, and delete the new version data already written into the data record; responding to data operations directed to the meta-information in accordance with the old version data in the data record.

In some embodiments, the retrieval service module 5332 is configured to, during writing the new version data into the data record, synchronize, when a data operation for the meta information is received, the data record in the database corresponding to the meta information into a cache of a retrieval service, and synchronize a result of the data operation into a data record stored in the cache of the retrieval service.

In some embodiments, the retrieval service module 5332 is further configured to delete the new version data in the cache of the retrieval service when a rollback request for the meta information is received; and deleting the old version data corresponding to the meta information in the cache of the retrieval service when the database completes the writing of the new version data corresponding to the meta information.

In some embodiments, the retrieval service module 5332 is further configured to, in response to a data operation on the meta information, load a data record corresponding to the meta information into the cache of the retrieval service when a hit rate of the cache of the retrieval service is lower than a hit rate threshold or a version of data stored in the cache of the retrieval service is lower than a version threshold, and synchronize a result of the data operation in the loaded data record; responding to a new data operation for the meta-information according to the data record stored in the cache of the retrieval service.

Embodiments of the present invention provide a computer-readable storage medium storing executable instructions, which, when executed by a processor, cause the processor to perform a method for upgrading a database provided by embodiments of the present invention, for example, a method for upgrading a database as shown in fig. 3, 4, 5 or 6, where the computer includes various computing devices including an intelligent terminal and a server.

In some embodiments, the computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions can correspond, but do not necessarily correspond, to files in a file system, and can be stored in a portion of a file that holds other programs or data, e.g., in one or more scripts stored in a hypertext markup language document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.

Next, an example in which data in the database is a human face feature is taken as an example, and the method for upgrading the database provided by the embodiment of the present invention is described.

Upgrading the face feature library (or called face feature database) (i.e. updating the face feature library) requires new version feature extraction for the face data. The scheme adopted in the related technology is that stock data extracts new version characteristics off line, the incremental characteristic data is stopped to be updated, and then the service is recovered. For large feature library levels, the entire process may have service unavailability on the order of minutes or even hours.

The face feature library upgrading scheme in the related art is applied to a face retrieval system deployed offline, and a user can tolerate the condition that service is unavailable for a certain time. However, for a public cloud face retrieval system, the situation that the service is unavailable for a certain time cannot occur, and the service is required to be always available because time zones of users who serve a large number of users are inconsistent. In addition, the upgrading of the face feature library of the public cloud is dominated by the client, a specially-assigned person can not be arranged for each client for butt joint, the upgrading cost of the client is reduced as much as possible, the upgrading willingness of the client is improved, and the rapid popularization of a new version is facilitated.

In order to solve the above problems, embodiments of the present invention provide a method for upgrading a database, in which services such as face retrieval and face feature update are always available during the upgrading process, and the database is not locked and is not stopped. After the upgrade is completed, the upgrade can be performed by one-key gray scale new version service (namely, one-key automatic upgrade is realized) or rollback. The feature version changes in the face retrieval system are effected immediately, whether upgraded or rolled back.

Referring to fig. 8, fig. 8 is a schematic structural diagram of an upgrade system of a database according to an embodiment of the present invention, and three links in fig. 8 will be separately described below.

Write link (one):

1) user to feature management and retrieval system (Zeus Access): the user triggers the request of adding and deleting and revising the characteristic through the interface (hereinafter, the request is referred to as adding and deleting and revising the characteristic).

Here, the operations of adding, deleting and modifying features, that is, the operations of adding, deleting and modifying data described above include a request for adding a feature (or operation), a request for deleting a feature, a request for modifying a feature, and a request for querying a feature.

2) Feature management and retrieval System to Feature management center (Feature Manager): and the access layer of the characteristic management and retrieval system transparently transmits the request of adding, deleting and modifying the characteristics to the characteristic management center.

Here, the feature management center is responsible for feature management and retrieving features in the system and interacting with the persistence layer (i.e., the database). The add/delete/modify request of the feature, i.e. the above-mentioned add/delete/modify operation of the data, includes an add request (or called operation) of the feature, a delete request of the feature, or a modify request of the feature.

3) Feature management center to database Proxy (DB Proxy): the feature management center interacts with the database agent, and performs persistence processing on the information such as features and identification Identity (ID, Identity Document) (including the page number and object identification in the database) through the database agent.

Here, the database broker interacts with the database as an abstraction (i.e., intermediary as described above) between upper level requests and the database.

4) Feature management center to synchronization trigger (sync): after receiving the request for adding and deleting the features, the feature management center synchronizes the result after the request is validated to the cache of the feature retrieval service through the synchronization trigger.

5) Synchronization trigger to Feature Cache (Feature Cache): the characteristic Cache synchronization (Sync Cache) synchronizes the characteristics of the Cache among a plurality of retrieval services, and the synchronization trigger synchronizes the results after the characteristic is added, deleted, modified and checked to each Cache.

6) Database proxy to feature caching: when the cache is missed or the version of the features in the cache is lower than the latest version in the system, a lazy loading mechanism is triggered to acquire the feature data of the latest version from the database.

(II) search Link

1) The Central Processing Unit (CPU) retrieves the link, and the access layer of the characteristic management and retrieval system determines the node where the current retrieval request lands according to a consistent Hash algorithm, and splits the retrieval request to corresponding retrieval service nodes.

2) A Graphics Processing Unit (GPU) retrieves the link, which, depending on the routing table configuration, forwards the request to the corresponding GPU machine.

(III) control Link

1) The operation and maintenance personnel can adjust the data in a configuration management center (Config Manager) and a characteristic management center through a specified interface.

Here, the configuration management center and the feature management center correspond to the above-described control management service.

2) The operation and maintenance personnel can change the condition of the nodes and the operator (Worker) in the consistent Hash.

3) The operation and maintenance personnel can adjust the routing table of the GPU retrieval service.

Referring to fig. 9A, 9B, and 9C, fig. 9A, 9B, and 9C are schematic views of a storage structure of feature data according to an embodiment of the present invention.

Fig. 9A is a schematic diagram of a feature table, fig. 9B is a schematic diagram of meta information, and in fig. 9A and 9B, one face feature information record (i.e., the data record described above) includes feature meta information and feature data. Wherein the feature meta information includes: a belonging face library 901(group _ ID), a number of pages in the library 902(page _ ID), an associated person ID903(person _ ID) (i.e., the person identifier described above), and a unique flag 904(face _ ID) (i.e., the face identifier described above). The feature data uses a double Buffer (Buffer) structure 905, so that two versions of feature data (i.e., the old version data and the new version data described above) can be compatible with one face feature information record at the same time. The feature meta information and the page control information of each person are stored in a storage system and described in a structured data storage (Protobuffer) format. When the face retrieval system performs face retrieval, only features of a certain version are used and mapped into a Feature information record, that is, a Feature Index, for example, fea _0 or fea _1 in fig. 9A, and the value in the Feature Index is maintained by the configuration management center in fig. 8.

Fig. 9A is a storage interface of the feature information in the database, and feature data is stored in fields of fea _0 and fea _ 1. These two fields constitute a double Buffer mechanism. The decision of which Feature data in field is valid is made by the real-time Feature Index update mechanism.

Fig. 9C is a schematic diagram of Page control information, which refers to mapping information 906 of the belonging face library (Group) and the number of pages in the library (Page), cache routing information 907 of the pages, and the like, for example, the number of pages in the Group and which retrieval service each Page is cached on.

Referring to fig. 10, fig. 10 is a schematic flowchart of an upgrading method for a database according to an embodiment of the present invention.

1) After the characteristic management and retrieval system obtains an upgrade instruction (or called an update request), the asynchronous Worker is started to process the new version characteristic extraction of the stock data.

In some embodiments, an atomic switch is turned on, initiating a feature upgrade process, wherein the feature upgrade needs to rely on the original picture (i.e., the source data described above). The old version features (i.e., the old version data described above) are the result of extracting the original picture with an old model (i.e., the old data model described above). The new version features (i.e., the new version data described above) are the result of being extracted from the original picture with the new model (i.e., the new data model described above). Because the input, output and calculation methods of the old model and the new model are inconsistent, the characteristics of the old and new versions have a certain degree of difference.

As an example, the new version feature extraction process is to find a corresponding picture from the related information (e.g., face _ id) of the old version feature, and then re-extract the new version feature using the picture and the new model.

2) The configuration management center obtains the spare Feature Index (i.e., the spare data directory mentioned above), and the Worker writes the Feature of the new version into the spare Feature Index of the Feature information record.

Here, the spare Feature Index refers to the Feature Index that is not currently used for Feature add/drop checking. Normally, only fea _0 or fea _1 in fig. 9A needs to have valid data to normally respond to a user request, but a Feature Index needs to be used to store a new version of features during Feature upgrade, the new version of features is not used for adding or deleting modification of features before the upgrade is completed, and the new version of features is validated after the upgrade is completed.

3) During the upgrade process, when a user requesting an external service requests to change Feature data in the database, the Feature management center starts double writing, that is, for the same piece of original data (i.e., the new original data or the modified original data submitted in the user request), features of new and old versions are simultaneously extracted and respectively stored in corresponding Feature indexes for updating.

4) After the stock characteristic processing is finished, each characteristic information record in the characteristic database contains the characteristics of the new version and the old version. After the user determines the gray level, the configuration management center atom updates the Feature Index, and new retrieval and other requests use new version features. The feature management center keeps double writes to roll back over time until the user confirms that the upgrade is complete.

5) And after the user confirms that the upgrade is completed, the feature management center stops double writing, and the feature management and retrieval system discards the old version feature cache.

In the upgrading process, the change of the state bit is atomic (namely, the switching of the new version and the old version does not have an intermediate state, only the characteristics of the new version or the characteristics of the old version are definitely used, and half of the characteristics of the new version and half of the characteristics of the old version are not mixed for use), and the rolling-back capability at any time is kept in the whole process.

In summary, the embodiments of the present invention have the following beneficial effects:

(1) according to the new data model, the new version data is extracted from the source data corresponding to the old version data, a large amount of resources are not needed to be spent on importing the new version data from the outside, and the updating speed and accuracy of the database can be improved.

(2) The old version data and the new version data are respectively stored in different data directories of the data record, so that the subsequent switching of the status bit of the data version used for responding to the data operation in the data record can be conveniently indicated, and the change of the status bit is ensured to be atomic, namely the data versions used for responding to the data operation are the same, so that the consistency of data response is ensured.

(3) Different response measures are taken according to different data operation requests, so that the speed of responding to the user request can be improved, and the user experience is improved; and by putting the hot data which can be inquired at any time into the retrieval service, the speed of data inquiry can be improved, and the problem that the service cannot be provided for the user in time due to the fact that the speed of acquiring the corresponding data from the database is too low is avoided.

(4) During the updating period of the database, data operation is responded according to the old version data, so that the service can be continuously provided for the user; and by starting a double-write mechanism, not only the result of the data operation is written into the old version data in the data record, but also the corresponding new version data written into the data record is synchronized, so that the problem that the data after the user data operation is not matched with the data stored in the database can be avoided, and the consistency of the data is ensured.

(5) The data in the database is updated by one key, the updated data in the database is immediately effective, and the old version data in the database is deleted after the update is finished, so that the storage resources of the database can be saved.

(6) The data in the database is backed up by one key, the backed-up data in the database is immediately effective, and the new version data in the database is deleted after the rollback is finished, so that the storage resources of the database can be saved.

(7) The result of the data operation is stored in the cache of the retrieval service, so that the corresponding data can be directly obtained through the cache of the retrieval service to perform the data operation if the subsequent operation is performed on the same data, the database does not need to be accessed again to obtain the data, and the efficiency of the data operation is improved.

The above description is only an example of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

Claims

1. A method for upgrading a database, the method comprising:

acquiring new version data of the meta-information;

2. The method of claim 1,

the meta information comprises the page number in the library and the object identification, and the old version data is extracted from the source data corresponding to the object identification according to an old data model;

the acquiring of the new version data of the meta information comprises the following steps:

traversing each data record in the database to perform the following operations:

searching an object identifier in the data record in a storage page corresponding to the page number in the database of the data record;

searching corresponding source data in the database according to the searched object identifier;

and extracting new version data from the searched source data according to the new data model.

3. The method of claim 1, wherein the writing the new version data into the data record corresponding to the meta information in the database comprises:

searching a standby data directory for storing the new version data in a data record of the database corresponding to the meta information;

and writing the new version data into the standby data directory.

4. The method of claim 1,

the type of data operation for the meta information includes: data adding operation, data deleting operation, data modifying operation and data inquiring operation;

the responding to data operations for the meta-information from the old version data in the data record includes:

according to the type of the data operation aiming at the meta-information, performing the data operation of the corresponding type on the old version data of the meta-information to obtain a corresponding result;

the synchronously updating the result of the data operation to the new version data in the data record comprises:

and when the type of the data operation aiming at the meta information is a data adding operation, a data deleting operation or a data modifying operation, synchronously updating the result corresponding to the data operation to the new version data in the data record.

5. The method of claim 1,

the data record comprises a status bit, wherein the status bit is used for indicating a data version used for responding to the data operation in the data record;

the responding to the data operation for the meta-information according to the new version data in the data record comprises:

uniformly updating the status bit in each data record in the database to indicate the new version data to respond to the data operation;

and when the updating of the status bit of any data record in the database fails, the status bit of each data record is rolled back to the position before updating.

6. The method according to any one of claims 1 to 5, further comprising:

in response to a rollback request for the meta-information, stopping writing the new version data into the data record corresponding to the meta-information in the database, and deleting the new version data written into the data record;

responding to data operations directed to the meta-information in accordance with the old version data in the data record.

7. The method according to any one of claims 1 to 5, further comprising:

when the hit rate of a cache of a retrieval service is lower than a hit rate threshold value or the version of data stored in the cache of the retrieval service is lower than a version threshold value, responding to the data operation aiming at the meta-information, loading the data record corresponding to the meta-information into the cache of the retrieval service, and synchronizing the result of the data operation in the loaded data record;

responding to a new data operation for the meta-information according to the data record stored in the cache of the retrieval service.

8. The method according to any one of claims 1 to 5, further comprising:

during writing of the new version data to the data record, when a data operation is received for the meta-information,

and synchronizing the data records corresponding to the meta information in the database to a cache of a retrieval service, and synchronizing the result of the data operation to the data records stored in the cache of the retrieval service.

9. The method of claim 8, further comprising:

deleting new version data in the cache of the retrieval service when a rollback request for the meta-information is received;

and deleting the old version data in the cache of the retrieval service when the database completes the writing of the new version data corresponding to the meta information.

10. An apparatus for upgrading a database, comprising: