CN112612780A

CN112612780A - Database operation method and device

Info

Publication number: CN112612780A
Application number: CN202011598893.6A
Authority: CN
Inventors: 谢鑫
Original assignee: AI Speech Ltd
Current assignee: AI Speech Ltd
Priority date: 2020-12-29
Filing date: 2020-12-29
Publication date: 2021-04-06

Abstract

The invention discloses a database operation method and a device, wherein in the method, hot database data information and cold database data information corresponding to a second machine room server are obtained; and respectively updating the data of the local hot database and the local cold database by using the acquired data information of the hot database and the cold database. Therefore, data synchronization of the hot database and the cold database between servers of different machine rooms is realized, the reliability of data service is guaranteed, and the service response speed can be improved in a high-mobility service scene.

Description

Database operation method and device

Technical Field

The invention belongs to the technical field of computers, and particularly relates to a database operation method and device.

Background

At present, most of mainstream data cold-hot separation storage schemes are single-room deployment schemes, hot data clusters are mainly based on memory databases such as redis, and cold data clusters are mainly based on Hbase and elastic search.

At present, a hot database and a cold database of each machine room are mutually independent and isolated, so that hot data cannot be shared among different machine rooms. With the continuous development of modern vehicles, the position of a user terminal has high mobility, and a cold and hot data storage scheme of a single machine room causes different hot data caches of different machine room servers, so that the service response speed is influenced.

In view of the above problems, the industry has not provided a better solution for the moment.

Disclosure of Invention

An embodiment of the present invention provides a database operating method and apparatus, which are used to solve at least one of the above technical problems.

In a first aspect, an embodiment of the present invention provides a database operating method, which is applied to a first computer room server, where the method includes: acquiring hot database data information and cold database data information corresponding to a second machine room server; and respectively updating the data of the local hot database and the local cold database by using the acquired data information of the hot database and the cold database.

In a second aspect, an embodiment of the present invention provides a database operating apparatus, including: a data acquisition unit configured to acquire hot database data information and cold database data information corresponding to the second machine room server; and the data updating unit is configured to respectively update the local hot database and the local cold database by using the acquired hot database data information and cold database data information.

In a third aspect, an embodiment of the present invention provides an electronic device, including: the computer-readable medium includes at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the above-described method.

In a fourth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the above method.

The embodiment of the invention has the beneficial effects that:

the first machine room server can acquire the data information of the hot database and the data information of the cold database in the second machine room server, and perform data updating on the local hot database and the local cold database, so that data synchronization of the hot database and the cold database among the servers of different machine rooms is realized, and when the server of one machine room is down or abnormal in service, the services of users can be quickly and reliably recovered by using the servers of other machine rooms. In addition, in a high-mobility service scenario, a user request may be rapidly migrated between multiple remote machine rooms, and the hot data cache is synchronized between different machine room servers, so that the service response speed can be increased.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.

FIG. 1 shows a flow diagram of one example of a method of database operation, according to an embodiment of the invention;

FIG. 2 shows an architectural diagram of an example of a computer room database system according to an embodiment of the invention;

FIG. 3 depicts a flowchart of an example of a server reading data in response to a read data request, according to an embodiment of the invention;

FIG. 4 shows a flowchart of an example of a server writing data in response to a write data request, according to an embodiment of the invention;

FIG. 5 is a flowchart illustrating an example of a server writing data according to a data write pattern, according to an embodiment of the present invention;

FIG. 6A illustrates an operational schematic of an example of a database system according to a cold-hot synchronized write mode;

FIG. 6B illustrates an operational diagram of an example of a database system according to a hot synchronous cold asynchronous write mode;

FIG. 6C illustrates an operational diagram of an example of a database system according to a hot and cold asynchronous write mode;

FIG. 6D illustrates an operational diagram of an example of a database system according to a data read mode;

fig. 7 is a block diagram showing an example of a database operating apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.

The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

As used in this application, the terms "module," "device," and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, or software in execution. In particular, for example, an element may be, but is not limited to being, a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. Also, an application or script running on a server, or a server, may be an element. One or more elements may be in a process and/or thread of execution and an element may be localized on one computer and/or distributed between two or more computers and may be operated by various computer-readable media. The elements may also communicate by way of local and/or remote processes in accordance with a signal having one or more data packets, e.g., from a data packet interacting with another element in a local device, distributed system, and/or across a network of the internet with other systems by way of the signal.

Finally, it should be further noted that the terms "comprises" and "comprising," when used herein, include not only those elements but also other elements not expressly listed or inherent to such processes, methods, articles, or devices. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

In this context, the term "hot data" may refer to data that needs to be frequently accessed by a service node, and in some cases, stored in memory, to guarantee a response rate. The term "cold data" may refer to data that is not accessed frequently, and in some cases may be stored on a hard disk.

It should be noted that the data in the hot database and the cold database in the server are not completely fixed, and may be dynamically updated. For example, if data a in the hot database is not called for a period of time, the data a may be transferred to the cold database; if data b in the cold database is called more times over a period of time, the data b may be transferred to the hot database.

FIG. 1 shows a flow diagram of an example of a method of database operation, according to an embodiment of the invention. Regarding the execution subject of the inventive embodiment, it may be a server.

As shown in fig. 1, in step 110, the first room server obtains hot database data information and cold database data information corresponding to the second room server. For example, each room server may periodically receive hot database data information and cold database information from other room servers according to a preset period.

In step 120, the first machine room server respectively updates the local hot database and the local cold database by using the acquired hot database data information and cold database data information. For example, assuming that data s is stored in the hot database of the second room server, but data s does not exist in the hot database of the first room server, data s may be added to the hot database of the first room server.

According to the embodiment of the invention, the machine room server can acquire the hot data and the cold data in other machine room servers and update the data of the local hot database and the local cold database, so that the condition of service interruption when the server of a single machine room is abnormal can be avoided. In addition, in a high-mobility service scenario, due to the synchronization of the hot data caches among different servers, even if a user request is migrated in a multi-computer room, the high-efficiency service response speed can still be guaranteed.

In some examples of the embodiment of the present invention, specific communication interfaces may be respectively disposed in different servers, and data of the hot and cold databases in different servers is monitored through the communication interfaces, and corresponding hot and cold database update operations are performed.

Specifically, the first room server may perform communication Interface pairing with a second communication Interface of the second room server based on a preset first communication Interface (API), and when the Interface pairing is successful, the first room server may receive the hot database data information and the cold database data information from the second room server. Therefore, the communication interfaces can be configured in the machine room servers, so that the corresponding cold and hot database updating operation can be realized.

In some alternative embodiments, the SDKs may be configured in each machine room server, so as to provide data services to the outside in an SDK manner, since the SDK codes are fixed, the configuration process is more convenient, and the operation on the cold and hot databases may be directly encapsulated through the SDKs without processing through an intermediate layer service, so that the execution efficiency is higher.

However, when the service is configured through the SDK, all steps are performed at the caller, there is a risk that the caller will be brought by a code bug or memory leak, and the risk of the code bug can be effectively avoided through the configuration operation of the upper API. Moreover, if the function upgrade is involved, the code change of the caller is necessarily involved, and the upgrade can be performed in a manner of being unaware of the caller by the API.

In addition, unlike some unidirectional data synchronization across the machine room through a data synchronization tool, machine room data synchronization is realized through the API, and operation is not required to be implemented in a business low peak period, for example, the operation can be implemented in a normal business process.

Fig. 2 shows an architecture diagram of an example of a computer room database system according to an embodiment of the present invention.

As shown in fig. 2, when a multi-room database deployment is adopted, each room deploys a complete set of data storage services, including a redis-based hot data storage cluster and a mysql-based cold data storage cluster. In addition, real-time bidirectional synchronization of data can be completed among redis databases of each machine room server, and real-time bidirectional synchronization of data can also be completed among mysql databases of each machine room server. In addition, the storage service of each machine room server provides a uniform API externally, and multiple data storage schemes for dealing with a strong consistent scene (namely, access is allowed immediately after data is successfully put in storage) and dealing with a final consistent scene (namely, access is allowed after data is successfully put in storage) are provided through the API, so that a business party can select the data storage schemes according to the database requirements of the business party.

It should be noted that, in the related art, for a massive concurrent flow request, if the read cache redis (or the hot database) does not read data and the Hbase database (or the cold database) is removed to retrieve data at the same time, the data reading pressure is increased instantaneously, and even a cache breakdown fault may be caused. Therefore, some related experts or scholars propose that the cold and hot state identification of the current target data in the massive concurrent flow requests can be identified through an algorithm, and the data are searched in the corresponding hot storage or cold storage according to the identification and returned. However, the traffic identification process also causes corresponding resource consumption and time consumption, increasing the data reading pressure.

FIG. 3 shows a flowchart of an example of a server reading data in response to a read data request, according to an embodiment of the invention.

As shown in fig. 3, the server detects a read data request in step 310.

If the detection result in step 310 indicates that a read data request is detected, then a jump is made to step 320. If the detection result in step 310 indicates that no read data request is detected, a jump is made back to step 310 to perform a continuous monitor read data operation.

In step 320, the server responds to the data read request with a local database, with the hot database responding with a higher priority than the cold database. Specifically, upon receiving a data reading request, the server may perform a data reading operation on the hot database, and if the target data is not found in the hot database, perform a query in the cold database, so as to avoid a concurrent data reading operation, and reduce the database pressure, for example, refer to the related description in fig. 6D.

In the embodiment of the invention, as the hot databases of different machine room servers can be synchronized, the data information in the hot databases is relatively comprehensive, and the hit rate of the access request corresponding to the hot data can be ensured. In addition, when massive concurrent flow requests exist, the condition that the cold database and the hot database are called simultaneously does not exist, and the access pressure of the databases is reduced.

Fig. 4 is a flowchart illustrating an example of writing data in response to a write data request by a server according to an embodiment of the present invention.

As shown in FIG. 4, in step 410, the server detects a request to write data.

If the detection result in step 410 indicates that a write data request is detected, a jump is made to step 420. If the test result in step 410 indicates that no write data request is detected, then a jump is made back to step 410 to continue monitoring for write data operations.

In step 420, the server determines source data information corresponding to the data writing request, and writes the source data information into the local hot database and the local cold database, respectively. Here, the source data information may indicate new data to be put in stock, such as user registration information and the like.

It should be understood that in some examples of embodiments of the present invention, the order of writing the source data information to the hot database and the cold database may be temporarily unlimited, for example, the source data information may be written to the hot database and the cold database simultaneously.

In some application scenarios, when the database of the first machine room server receives the source data information, the data of the corresponding local hot database and cold database can be updated, and the synchronous data updating operation of the databases of other machine room servers can be realized through the API. In addition, the user can select a proper data writing mode for the database according to the service requirement of the database.

Fig. 5 is a flowchart illustrating an example of a server writing data according to a data writing mode according to an embodiment of the present invention.

In step 510, the server obtains the current data write pattern. For example, the user may select an appropriate one of the plurality of data writing modes as the current data writing mode according to the database operation requirement.

In step 520, the server determines a target writing order for the local hot database and cold database according to the current data writing mode, so that the source data information can be written to the local hot database and cold database according to the target writing order. Here, different data write patterns may indicate a write order specific to the hot and cold database.

In some examples of embodiments of the invention, the current data writing mode comprises any one selected from the group consisting of: a cold-hot synchronous write mode, a hot synchronous-cold asynchronous write mode, and a cold-hot asynchronous write mode.

In some application scenarios, a user may select a data writing mode that meets the user's desire as a current data writing mode by selecting among a plurality of data writing modes.

Specifically, the server may receive a mode selection request from the client, and may trigger sending of a plurality of preset data write modes to the client based on the mode selection request, for example, a cold and hot synchronous write mode, a hot synchronous and cold asynchronous write mode, and a cold and hot asynchronous write mode may all be sent to the client for a user to select at the client. In this way, the user can perform a selection operation on the client to select the current data writing mode from the plurality of data writing modes. Furthermore, the server can receive the current data writing mode from the client, so that a user can perform interactive operation with the server through the client to select the data writing mode meeting the database business requirements.

FIG. 6A illustrates an operational schematic of an example of a database system according to a cold-hot synchronized write mode.

As shown in fig. 6A, when the server receives source data information from a data producer (e.g., a client or a mobile terminal), the source data information may be synchronously written into the cold database and the hot database, and after data is successfully put into the cold database and the hot database, a corresponding database operation result is fed back to a user. Therefore, after the user obtains the response result of the write request, the read request can be sent out immediately, and the strong consistent scene of the database can be dealt with.

FIG. 6B illustrates an operational diagram of an example of a database system according to a hot-synchronous cold asynchronous write mode. In particular, the server may feed back (e.g., feed back to the client) the write success notification after writing the source data information to the hot database, and then write the source data information to the cold database.

As shown in fig. 6B, when the server receives the source data information from the data producer, the source data information may be synchronously written into the hot database, and a write success notification is directly fed back to the user, at this time, the user may directly access the server, and the server may also preferentially query the hot database to read corresponding data, so as to provide a storage service. Therefore, the database writing result can be fed back before the data is written into the cold database, and the service response speed is improved.

FIG. 6C illustrates an operational diagram of an example of a database system according to a hot and cold asynchronous write mode.

As shown in fig. 6C, when the server receives the source data information from the data producer, a write success notification can be fed back to the user in real time. Then, the server writes the source data information into the cold database and the hot database respectively in the background. By the method, system storage resources can be optimized, but after the write request is completed, the read request of the user for the data cannot be obtained immediately, and the method is suitable for dealing with the final consistent scene.

FIG. 6D illustrates an operational diagram of an example of a database system according to a data read mode.

As shown in fig. 6D, when receiving the read request, the server may query the hot database first, and directly feed back if the target data exists. If the target data is not found in the hot database, the cold database may be queried and the final data read returned.

Through the embodiment of the invention, the cross-machine room can respectively synchronize the cold data and the hot data, and the hot data cache can still be well hit in the service scene of the continuous and rapid change of the user position, thereby ensuring the user service. In addition, a plurality of data writing modes are provided to respectively deal with strong consistency and final consistency scenes, so that a business party can reasonably select a calling mode according to actual requirements, and the data consistency can be ensured, and the flexibility and the response speed are high.

As shown in fig. 7, the database operating device 700 includes a data acquisition unit 710 and a data update unit 720.

The data acquisition unit 710 is configured to acquire hot database data information and cold database data information corresponding to the second machine room server.

The data updating unit 720 is configured to perform data updating on the local hot database and the local cold database respectively by using the acquired hot database data information and cold database data information.

The apparatus according to the above embodiment of the present invention may be used to execute the corresponding method embodiment of the present invention, and accordingly achieve the technical effect achieved by the method embodiment of the present invention, which is not described herein again.

In the embodiment of the present invention, the relevant functional module may be implemented by a hardware processor (hardware processor).

In another aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, the program being executed by a processor to perform the steps of the above database operation method.

The product can execute the method provided by the embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.

The electronic device of embodiments of the present invention exists in a variety of forms, including but not limited to:

(1) mobile communication devices, which are characterized by mobile communication capabilities and are primarily targeted at providing voice and data communications. Such terminals include smart phones (e.g., iphones), multimedia phones, functional phones, and low-end phones, among others.

(2) The ultra-mobile personal computer equipment belongs to the category of personal computers, has calculation and processing functions and generally has the characteristic of mobile internet access. Such terminals include PDA, MID, and UMPC devices, such as ipads.

(3) Portable entertainment devices such devices may display and play multimedia content. Such devices include audio and video players (e.g., ipods), handheld game consoles, electronic books, as well as smart toys and portable car navigation devices.

(4) And other electronic devices with data interaction functions.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions substantially or contributing to the related art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A database operation method is applied to a first machine room server, and comprises the following steps:

acquiring hot database data information and cold database data information corresponding to a second machine room server;

and respectively updating the data of the local hot database and the local cold database by using the acquired data information of the hot database and the cold database.

2. The method of claim 1, wherein the method further comprises:

detecting a read data request;

when a read data request is detected, responding to the data read request with the local database, the response priority of the hot database being higher than that of the cold database.

3. The method of claim 1, wherein the method further comprises:

detecting a write data request;

and when the data writing request is detected, determining source data information corresponding to the data writing request, and writing the source data information into the local hot database and the local cold database respectively.

4. The method of claim 3, wherein said writing the source data information to the local hot and cold databases, respectively, comprises:

acquiring a current data writing mode;

determining a target writing order for the local hot and cold databases based on the current data writing pattern, such that the source data information can be written to the local hot and cold databases in the target writing order.

5. The method of claim 4, wherein the obtaining the current data write pattern comprises:

receiving a mode selection request from a client;

triggering and sending a plurality of preset data writing modes to a client based on the mode selection request, wherein the client is used for receiving user selection operation to select a current data writing mode from the plurality of data writing modes;

receiving the current data write pattern from the client.

6. The method of claim 4 or 5, wherein the current data write mode comprises any one selected from: a cold-hot synchronous write mode, a hot synchronous-cold asynchronous write mode, and a cold-hot asynchronous write mode.

7. The method of claim 6, wherein determining a target write order for the local hot and cold databases according to the current data write pattern comprises:

and if the current data writing mode is a hot synchronous cold asynchronous writing mode, feeding back a writing success notice after writing the source data information into the hot database, and then writing the source data information into the cold database.

8. A database operating apparatus comprising:

a data acquisition unit configured to acquire hot database data information and cold database data information corresponding to the second machine room server;

and the data updating unit is configured to respectively update the local hot database and the local cold database by using the acquired hot database data information and cold database data information.

9. An electronic device, comprising: at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of any one of claims 1-7.

10. A storage medium on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.