CN113139097A

CN113139097A - Data writing method, data reading method, device and equipment

Info

Publication number: CN113139097A
Application number: CN202010061401.3A
Authority: CN
Inventors: 朱辉云; 钱赟; 麦慧君; 常锐; 谢成亮
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-01-19
Filing date: 2020-01-19
Publication date: 2021-07-20

Abstract

The application provides a data writing method, a data reading method, a data writing device, a data reading device and equipment, wherein the data writing method can comprise the following steps: acquiring a data writing request, wherein the data writing request comprises an original data identifier; determining a target data table according to the original data identification and the total number of the data tables; sending the data writing request to a target database where the target data table is located, so that the target database adds a data item in the target data table according to the data writing request, wherein the index of the data item is a target data identifier corresponding to the original data identifier; wherein the target data identification is smaller than the original data identification. Through the technical scheme, the compression rate of the whole data can be improved, and storage resources are saved.

Description

Data writing method, data reading method, device and equipment

Technical Field

The present application relates to the field of databases, and in particular, to a method, an apparatus, and a device for writing data and reading data.

Background

In the big data era, data is an important component of the internet, and the realization of internet functions needs to be completed based on the data. The sources of data are many, such as travel records, consumption records, web pages browsed, messages sent, etc., and the types of data are also many, such as text data, image data, music data, sound data, video data, etc. For a large amount of data of the big data era, the data can be stored through a database, so that the related functions of the internet can be realized based on the data.

A database is a software system that stores and manages data according to a data structure, which is a long-term storage, organized, shared, and uniformly managed collection of data. The database is an entity, and is a "warehouse" capable of storing data reasonably, in which a user stores data to be managed, and the two concepts of "data" and "library" are combined into a database. The database is a data management method, and can organize data more appropriately, maintain data more conveniently, control data more closely and utilize data more effectively.

However, the internet generates a large amount of data (e.g. hundreds of millions of data) every day, and when the data is stored by using a database, the amount of data to be stored is huge, so if the data cannot be managed well, a large amount of storage resources are consumed, and the storage performance and the query performance are low.

Disclosure of Invention

The application provides a data writing method, which comprises the following steps:

acquiring a data writing request, wherein the data writing request comprises an original data identifier;

determining a target data table according to the original data identification and the total number of the data tables;

sending the data writing request to a target database where the target data table is located, so that the target database adds a data item in the target data table according to the data writing request, wherein the index of the data item is a target data identifier corresponding to the original data identifier;

wherein the target data identification is smaller than the original data identification.

determining a target data table corresponding to the data writing request;

adding a data item in the target data table according to the data writing request, wherein the index of the data item is a target data identifier corresponding to the original data identifier;

The application provides a data reading method, which comprises the following steps:

acquiring a data reading request, wherein the data reading request comprises an original data identifier;

sending the data reading request to a target database where the target data table is located, so that the target database reads data from the target data table according to a target data identifier corresponding to the original data identifier; wherein the target data identification is smaller than the original data identification;

and acquiring the read data from the target database.

determining a target data table corresponding to the data reading request;

reading data from the target data table according to the target data identification corresponding to the original data identification; wherein the target data identification is smaller than the original data identification;

and sending the read data to the proxy server.

The present application provides a data writing apparatus, the apparatus comprising:

the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a data writing request which comprises an original data identifier;

the determining module is used for determining a target data table according to the original data identification and the total number of the data tables;

a sending module, configured to send the data write request to a target database where the target data table is located, so that the target database adds a data item to the target data table according to the data write request, where an index of the data item is a target data identifier corresponding to the original data identifier;

The present application provides a data reading apparatus, the apparatus comprising:

the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a data reading request which comprises an original data identifier;

the sending module is used for sending the data reading request to a target database where the target data table is located so that the target database reads data from the target data table according to a target data identifier corresponding to the original data identifier; the target data identification is smaller than the original data identification;

and the receiving module is used for acquiring the read data from the target database.

The present application provides a proxy server comprising:

a processor and a machine-readable storage medium having stored thereon a plurality of computer instructions, the processor when executing the computer instructions performs:

sending the data writing request to a target database where the target data table is located, so that the target database adds a data item in the target data table according to the data writing request, wherein the index of the data item is a target data identifier corresponding to the original data identifier; wherein the target data identification is smaller than the original data identification;

or acquiring a data reading request, wherein the data reading request comprises an original data identifier;

and acquiring the read data from the target database.

Based on the technical scheme, in the embodiment of the application, the data item of the target data table is indexed by using the target data identifier instead of the original data identifier, and the target data identifier is smaller than the original data identifier, so that the data volume of the target data table is reduced, routing information (the relation between the original data identifier and the target data table and the target database) is fully utilized to participate in compression of key data (namely the original data identifier), the compression ratio of the whole data is improved, the characteristic of high repetition rate of the key data is ingeniously utilized, and the compression ratio of the whole data is improved. Can be used in cooperation with (simultaneously) other compression algorithms to form an overlapping compression effect. The data can be effectively managed, the storage resources are improved, and the storage performance and the query performance are high.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments of the present application or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings of the embodiments of the present application.

FIG. 1 is a flow chart illustrating a data writing method according to an embodiment of the present application;

FIG. 2 is a flow chart illustrating a data writing method according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating a data reading method according to an embodiment of the present application;

FIG. 4 is a flow chart illustrating a data reading method according to an embodiment of the present application;

FIGS. 5A and 5B are schematic diagrams of an application scenario in an embodiment of the present application;

FIGS. 6A and 6B are schematic flow charts illustrating a data writing method according to an embodiment of the present application;

FIGS. 7A and 7B are schematic flow charts illustrating a data reading method according to an embodiment of the present application;

FIG. 8A is a schematic diagram of a data writing apparatus according to an embodiment of the present application;

FIG. 8B is a schematic diagram of a data reading apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a proxy server according to an embodiment of the present application.

Detailed Description

The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Depending on the context, moreover, the word "if" as used may be interpreted as "at … …" or "when … …" or "in response to a determination".

Example 1: the embodiment of the present application provides a data writing method, which can be applied to any type of device, and is not limited thereto, as long as data writing can be implemented. Such as may be applied to a proxy server or an application server, etc. Referring to fig. 1, a schematic flow chart of a data writing method is shown, the method including:

step 101, a data write request is obtained, where the data write request includes an original data identifier.

And 102, determining a target data table according to the original data identification and the total number of the data tables.

In a possible implementation manner, a processing function (such as a hash function) may be used to perform an operation on the original data identifier and the total number of the data tables to obtain a hash value corresponding to the original data identifier; wherein the processing function comprises: and a processing function capable of making the total number of the hash values obtained after the operation the same as the total number of the data tables. A target data table corresponding to the hash value may then be determined.

Step 103, sending the data writing request to a target database where the target data table is located, so that the target database adds a data item in the target data table according to the data writing request, wherein an index of the data item is a target data identifier corresponding to the original data identifier, and the target data identifier is smaller than the original data identifier.

For example, after the data write request is obtained, the target data identifier corresponding to the original data identifier may be determined according to the original data identifier and the total number of the data tables, and the target data identifier is sent to the target database. For example, the target data identifier and the data write request may be sent to the target database together, so that the target database may obtain the target data identifier corresponding to the original data identifier, and then record the index of the data item as the target data identifier corresponding to the original data identifier.

In one possible implementation, in order to determine the target data identifier corresponding to the target data table and the original data identifier, the following may be implemented: performing remainder operation according to the original data identification and the total amount of the data table; determining a target data identifier corresponding to the original data identifier according to the quotient obtained after the operation; and determining a target data table corresponding to the original data identification according to the remainder obtained after the operation.

Illustratively, the target database may include: a time series database for storing time series data.

In an example, the execution sequence is only an example given for convenience of description, and in practical applications, the execution sequence between steps may also be changed, and the execution sequence is not limited. Moreover, in other embodiments, the steps of the respective methods do not have to be performed in the order shown and described herein, and the methods may include more or less steps than those described herein. Moreover, a single step described in this specification may be broken down into multiple steps for description in other embodiments; multiple steps described in this specification may be combined into a single step in other embodiments.

Example 2: the embodiment of the present application provides a data writing method, which can be applied to any type of device, and is not limited thereto, as long as data writing can be implemented. Such as in a database (i.e., a database server, i.e., a server that implements data storage functionality, which may also be referred to as a storage server). For example, for a database, the database may be a time series database for storing time series data. Referring to fig. 2, a flow chart of a data writing method is shown, the method includes:

step 201, a data write request is obtained, where the data write request includes an original data identifier.

At step 202, a target data table corresponding to the data write request is determined.

In a possible implementation manner, when the proxy server sends the data write request to the database, the data write request may further include information of the target data table, based on which, the database may determine the target data table corresponding to the data write request according to the information of the target data table.

In another possible implementation, the database may determine the target data table according to the original data identification and the total number of data tables. For example, a processing function (such as a hash function) may be used to perform an operation on the total number of the original data identifier and the data table, so as to obtain a hash value corresponding to the original data identifier; wherein the processing function comprises: and a processing function capable of making the total number of the hash values obtained after the operation the same as the total number of the data tables. A target data table corresponding to the hash value may then be determined.

Step 203, adding a data item in a target data table according to the data writing request, wherein the index of the data item is a target data identifier corresponding to the original data identifier; the target data identity is smaller than the original data identity.

In one possible implementation, the target data identifier corresponding to the original data identifier may be obtained from the proxy server. For example, when the proxy server sends the data write request to the database, the data write request may further include a target data identifier corresponding to the original data identifier, based on which, the database may determine the target data identifier corresponding to the original data identifier, and then record the index of the data item as the target data identifier corresponding to the original data identifier, instead of recording the index as the original data identifier.

In another possible implementation, the target data identifier corresponding to the original data identifier may be determined according to the original data identifier and a total number of preconfigured data tables (e.g., the total number of data tables of all databases). For example, the remainder operation may be performed according to the original data identifier and the total number of data tables; and determining a target data identifier corresponding to the original data identifier according to the quotient obtained after the operation.

Example 3: the embodiment of the present application provides a data reading method, which can be applied to any type of device, and is not limited thereto, as long as data reading can be achieved. Such as may be applied to a proxy server or an application server, etc. Referring to fig. 3, a schematic flow chart of a data reading method is shown, where the method includes:

step 301, a data reading request is obtained, where the data reading request includes an original data identifier.

Step 302, determining a target data table according to the original data identification and the total number of the data tables.

Step 303, sending the data reading request to a target database where a target data table is located, so that the target database reads data from the target data table according to a target data identifier corresponding to the original data identifier; wherein the target data identification is smaller than the original data identification.

For example, after the data reading request is obtained, the target data identifier corresponding to the original data identifier may be determined according to the original data identifier and the total number of the data tables, and the target data identifier is sent to the target database. For example, the target data identifier and the data reading request may be sent to the target database together, so that the target database may know the target data identifier corresponding to the original data identifier.

Step 304, the read data is obtained from the target database.

Example 4: the embodiment of the present application provides a data reading method, which can be applied to any type of device, and is not limited thereto, as long as data reading can be achieved. Such as in a database (i.e., a database server, i.e., a server that implements data storage functionality, which may also be referred to as a storage server). For example, for a database, the database may be a time series database for storing time series data. Referring to fig. 4, a schematic flow chart of a data reading method is shown, where the method includes:

step 401, a data reading request is obtained, where the data reading request includes an original data identifier.

At step 402, a target data table corresponding to the data read request is determined.

In a possible implementation manner, when the proxy server sends the data reading request to the database, the data reading request may further include information of the target data table, based on which, the database may determine the target data table corresponding to the data reading request according to the information of the target data table.

Step 403, reading data from the target data table according to the target data identifier corresponding to the original data identifier; wherein the target data identification is smaller than the original data identification.

In one possible implementation, the target data identifier corresponding to the original data identifier may be obtained from the proxy server. For example, when the proxy server sends the data reading request to the database, the data reading request may further include a target data identifier corresponding to the original data identifier, based on which, the database may determine the target data identifier corresponding to the original data identifier, and then read data from the target data table according to the target data identifier, instead of reading data from the target data table according to the original data identifier.

Step 404, sending the read data to the proxy server.

The following describes the technical solution of the embodiment of the present application with reference to a specific application scenario.

Referring to fig. 5A, which is a schematic view of an application scenario of the embodiment of the present application, a client may be deployed in a user equipment (such as a terminal device, a personal computer, a notebook computer, a smart phone, a tablet computer, and the like, which is not limited thereto). An application server is a device that provides services to clients. A database, that is, a database server, which may also be referred to as a storage server, is a device that implements a data storage function and a data access function.

For example, the database may include a plurality of data tables, and store data through the data tables, which is not limited to this storage manner. The database may be a time series database for storing time series data, or may be another type of database, and the type of the database is not limited. The time-series database is generally called a time-series database, and the time-series database is mainly used for processing data with time tags (which are changed according to a time sequence, namely, time-series data), and the data with time tags is also called time-series data.

Illustratively, at least one database may be deployed, as shown in fig. 5A, taking the deployment of 3 databases as an example, and of course, the number of databases may be more, which is not limited to this. Each database may include at least one data table, as shown in fig. 5A, taking as an example that database 1 includes data table 0, data table 1, and data table 2, database 2 includes data table 3, data table 4, and data table 5, and database 3 includes data table 6 and data table 7. Of course, the number of data tables included in each database may be larger, and is not limited thereto.

In order to implement data compression, in this embodiment, a functional module having an agent function may be additionally deployed, and data writing and reading are implemented by the functional module. For example, the functional module may be deployed in an existing device, such as a database, or may be used as an independent device, and for convenience of description, the functional module is taken as an independent device for example, and this device is referred to as a proxy server.

Proxy (Proxy) is a special network service that allows a network terminal (typically a client) to make an indirect connection with another network terminal (typically a server) through the service.

Referring to fig. 5B, a schematic diagram of an application scenario after the proxy server is deployed, based on the application scenario shown in fig. 5A, the proxy server is deployed between the application server and the database.

Based on the above application scenarios, the following describes the above schemes with reference to several specific embodiments.

Example 5: the present embodiment proposes a data writing method, as shown in fig. 6A, the method includes:

step 601, the client sends a data write request to the application server.

In step 602, the application server sends a data write request to the proxy server.

Specifically, the application server may be configured with an address of the proxy server, and based on the address of the proxy server, the application server may send the data write request to the proxy server.

Step 603, the proxy server receives a data write request, where the data write request includes an original data identifier.

For example, in order to distinguish data of different users, the data write request may include a data identifier, and the type of the data identifier is not limited as long as the data can be distinguished. For the sake of convenience of distinction, the data identification in the data write request may be referred to as the original data identification.

Step 604, the proxy server determines a target data table corresponding to the original data identifier and a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of the data tables.

The processing function can make the total number of the hash values obtained after the operation the same as the total number of the data tables, so that each hash value can uniquely correspond to one data table, that is, the data table corresponding to each hash value can be determined. Further, for each hash value, only one data table corresponding to the hash value is determined, and a plurality of data tables corresponding to the hash value are not determined at the same time.

The processing function may be a hash function, and the hash function enables the total number of the hash values obtained after the operation to be the same as the total number of the data tables. For example, the hash function may be a remainder hash function, and a remainder of the original data identifier divided by the total number of the data tables may be used as a hash value corresponding to the original data identifier. If the hash value is 0, the target data table corresponding to the hash value 0 is a data table 0; if the hash value is 1, the target data table corresponding to the hash value 1 is the data table 1, and so on. Of course, the hash function is only an example, and is not limited thereto, as long as the total number of hash values obtained after the operation is the same as the total number of the data tables.

In a possible implementation manner, a selection function (such as a hash function) may be used to perform an operation on the original data identifier and the total number of the data tables to obtain a target data identifier corresponding to the original data identifier; the selection function includes: the target data identification obtained after operation can be smaller than the selection function of the original data identification, the selection function is not limited, and the target data identification is smaller than the original data identification.

For example, the selection function may be a hash function, and taking the remainder hash function as an example, a quotient obtained by dividing the original data identifier by the total number of the data tables may be used as the target data identifier corresponding to the original data identifier. For example, if the quotient obtained by dividing the original data identifier by the total number of the data tables is 1, the target data identifier corresponding to the original data identifier is 1; if the quotient of dividing the original data identifier by the total number of the data tables is 2, the target data identifier corresponding to the original data identifier is 2, and so on. Of course, the hash function is only an example, and is not limited thereto, as long as it is ensured that the target data identifier is smaller than the original data identifier.

In the above embodiment, the processing function may be denoted as f1, then table (key) is f1(S, K), table (key) represents the target data table, S represents the original data identifier, and K represents the total number of data tables, that is, the original data identifier and the total number of data tables are used as inputs, and the target data table may be obtained based on the processing function.

In the above embodiment, the selection function may be denoted as f2, and if id (key) is f2(S, K), id (key) represents the target data identifier, S represents the original data identifier, and K represents the total number of data tables, that is, the original data identifier and the total number of data tables are used as inputs, and the target data identifier is obtained based on the selection function.

For example, in order to determine the target data table and the target data identifier, taking a remainder hash function as an example, a remainder operation may be performed according to the original data identifier and the total number of the data tables (that is, the total number of the data tables deployed by all databases); determining a target data identifier corresponding to the original data identifier according to the quotient obtained after the operation; and determining a target data table corresponding to the original data identification according to the remainder obtained after the operation.

For example, referring to fig. 5B, where the total number of data tables is 8, the original data identification may be 0, 1, 2, etc. If the original data identifier is 0, performing a remainder operation on 0/8, where the quotient is 0 and the remainder is 0, so that the target data identifier is 0 and the target data table is data table 0; if the original data identifier is 1, performing a remainder operation on 1/8, where the quotient is 0 and the remainder is 1, so that the target data identifier is 0 and the target data table is data table 1; if the original data identifier is 7, performing a remainder operation on 7/8, where the quotient is 0 and the remainder is 7, so that the target data identifier is 0 and the target data table is data table 7; if the original data identifier is 8, performing a remainder operation on 8/8, where the quotient is 1 and the remainder is 0, so that the target data identifier is 1 and the target data table is data table 0; if the original data identifier is 15, performing a remainder operation on 15/8, where the quotient is 1 and the remainder is 7, so that the target data identifier is 1 and the target data table is data table 7; if the original data identifier is 16, performing a remainder operation on 16/8, where the quotient is 2 and the remainder is 0, so that the target data identifier is 2 and the target data table is data table 0; by analogy, the processing process of other original data identifiers is not described again.

In summary, if the original data flags are numerical values such as 0, 8, 16, 24, 32, etc., the target data table corresponding to the original data flags is data table 0. Original data mark 0 corresponds to target data mark 0, original data mark 8 corresponds to target data mark 1, original data mark 16 corresponds to target data mark 2, original data mark 24 corresponds to target data mark 3, and so on, obviously, the target data mark is smaller than the original data mark. Moreover, different original data identifications may correspond to different target data identifications.

When the original data marks are numerical values such as 1, 9, 17, 25, 33, etc., the target data table corresponding to these original data marks is data table 1. Original data identifier 1 corresponds to target data identifier 0, original data identifier 9 corresponds to target data identifier 1, original data identifier 17 corresponds to target data identifier 2, original data identifier 25 corresponds to target data identifier 3, and so on, obviously, the target data identifier is smaller than the original data identifier. Moreover, different original data identifications may correspond to different target data identifications.

If the original data id is a numerical value such as 2, 10, 18, 26, 34, etc., the target data table corresponding to the original data id is the data table 2. Original data mark 2 corresponds to target data mark 0, original data mark 10 corresponds to target data mark 1, original data mark 18 corresponds to target data mark 2, original data mark 26 corresponds to target data mark 3, and so on, obviously, the target data mark is smaller than the original data mark. Moreover, different original data identifications may correspond to different target data identifications.

By analogy, for other original data identifiers, the characteristics are similar to those described above, and are not described again here.

In step 605, the proxy server determines a target database where the target data table is located.

For example, referring to fig. 5B, database 1 includes data table 0, data table 1, and data table 2, database 2 includes data table 3, data table 4, and data table 5, and database 3 includes data table 6 and data table 7, and based on the above correspondence, the target database in which the target data table is located may be determined. For example, if the target data table is data table 1, the target database is determined to be database 1.

In step 606, the proxy server sends a data write request to the target database.

Illustratively, the proxy server may establish a connection (e.g., a TCP connection) with the target database, based on which the data write request may be sent to the target database.

For example, when sending the data write request to the target database, the proxy server may send the information of the target data table, the target data identifier, and the data write request together to the target database. For example, the proxy server may add the information of the target data table and the target data identifier to the data write request, and send the modified data write request to the target database.

In step 607, the target database determines the target data table and the target data id corresponding to the original data id after receiving the data write request (including the original data id).

Specifically, the proxy server sends the information of the target data table and the target data identifier to the target database, so that the target database can determine the target data table and the target data identifier.

And step 608, the target database adds a data item in the target data table according to the data write request, wherein the index of the data item is the target data identifier corresponding to the original data identifier, and the index of the data item is not the original data identifier. Also, the target data identification may be smaller than the original data identification.

Illustratively, referring to table 1, for an example of a target data table, each entry of the target data table is referred to as a data item, and the data item may include an index (i.e., key) and an attribute value. When a data item is added in the target data table, the index of the data item is a target data identifier, the attribute value of the data item may be user data, and the user data is parsed from the data write request, without limitation to the user data.

TABLE 1

Index	Attribute value
		0	User data a0
1	User data a1
		…	…
1000	User data a1000
		…	…
8000	User data a8000

Of course, table 1 is only one example of a data table, and the content of the data table is not limited.

For example, since the target data identifier is smaller than the original data identifier, when the target data identifier is used as an index of the data item, the data amount of the index can be reduced, thereby achieving the effect of data compression.

For example, assuming that the total number of data tables is 65536 (i.e. 2^16) and the index of the data item is represented in binary, assuming that the original data identification is 65536, if the index of the data item is the original data identification, 1111111111111 needs to be stored, i.e. 16 bits are consumed to store the original data identification, but if the index of the data item is the target data identification (the quotient of 65536 divided by 65536 is 1), only 1 needs to be stored, i.e. 1 bit is consumed to store the target data identification, obviously, the bit number can be saved significantly.

Illustratively, when a data item is added to the target data table in the target database, a compression algorithm may be further used to perform data compression on the data item, so as to further save storage resources and improve storage performance.

Data compression refers to reducing the data volume to reduce the storage space and improve the transmission, storage and processing efficiency of the data on the premise of not losing useful information, or reorganizing the data according to a certain algorithm to reduce the redundancy and storage space of the data, wherein the data compression comprises lossy data compression and lossless data compression.

For example, the data item may be data compressed using a gzip (gnuzip) data compression algorithm, or data compressed using a Snappy data compression algorithm. Of course, the above are only two examples, and the method is not limited to this, and any data compression algorithm may be used to perform data compression on the data items.

Through compressing data, can effectively manage data, promote storage performance and inquiry performance to solve the data storage cost problem that brings after the networking of a large amount of thing networking equipment under the thing networking scene.

Example 6: the present embodiment proposes a data writing method, as shown in fig. 6B, the method includes:

in step 611, the client sends a data write request to the application server.

Step 612, the application server sends a data write request to the proxy server.

Step 613, the proxy server receives a data write request, where the data write request includes an original data identifier.

And 614, the proxy server determines a target data table corresponding to the original data identifier according to the original data identifier and the total number of the data tables, and determines a target database where the target data table is located.

In step 615, the proxy server sends a data write request to the target database.

For example, steps 611 to 615 can refer to embodiment 5, and unlike embodiment 5, the proxy server does not determine the target data id corresponding to the original data id. And when the proxy server sends the data writing request to the target database, the information of the target data table and the target data identification are not sent at the same time.

In step 616, the target database receives a data write request, where the data write request includes an original data identifier, and determines a target data table corresponding to the original data identifier and a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of data tables.

The method for determining the target data table and the target data identifier by the target database may be as in embodiment 5, but the execution subject is changed from the proxy server to the target database, and details are not described here.

In step 617, the target database adds a data item in the target data table according to the data write request, where the index of the data item is the target data identifier corresponding to the original data identifier, and the index of the data item is not the original data identifier. Also, the target data identification may be smaller than the original data identification.

Example 7: the present embodiment proposes a data reading method, as shown in fig. 7A, the method includes:

in step 701, a client sends a data reading request to an application server.

In step 702, the application server sends a data reading request to the proxy server.

In step 703, the proxy server receives a data reading request, where the data reading request includes an original data identifier.

Step 704, the proxy server determines a target data table corresponding to the original data identifier and a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of the data tables.

Step 705, the proxy server determines the target database where the target data table is located.

In step 706, the proxy server sends a data read request to the target database.

For example, when sending the data reading request to the target database, the proxy server may send the information of the target data table, the target data identifier, and the data reading request together to the target database. For example, the proxy server may add the information of the target data table and the target data identifier to the data reading request, and send the modified data reading request to the target database.

In step 707, the target database determines a target data table and a target data identifier corresponding to the original data identifier after receiving the data reading request (including the original data identifier).

At step 708, the target database reads data from the target data table according to the target data identification.

Specifically, the target database may query the target data table shown in table 1 according to the target data identifier, to obtain data corresponding to the target data identifier, such as the user data a 500.

Step 709, the target database sends the read data to the proxy server, the proxy server sends the data to the application server, and the application server sends the data to the client.

Example 8: the present embodiment proposes a data reading method, as shown in fig. 7B, the method includes:

in step 711, the client sends a data reading request to the application server.

At step 712, the application server sends a data read request to the proxy server.

In step 713, the proxy server receives a data read request, where the data read request includes an original data identifier.

In step 714, the proxy server determines a target data table corresponding to the original data identifier and determines a target database where the target data table is located according to the original data identifier and the total number of the data tables.

In step 715, the proxy server sends a data read request to the target database.

Illustratively, steps 711 to 715 can refer to embodiment 7, and unlike embodiment 7, the proxy server does not determine the target data id corresponding to the original data id. When the proxy server sends the data reading request to the target database, the information of the target data table and the target data identification are not sent at the same time.

In step 716, the target database receives a data reading request, where the data reading request includes an original data identifier, and determines a target data table corresponding to the original data identifier and a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of data tables.

In step 717, the target database reads data from the target data table based on the target data identification.

Step 718, the target database sends the read data to the proxy server, the proxy server sends the data to the application server, and the application server sends the data to the client.

Example 9: based on the same application concept as the method described above, an embodiment of the present application further provides a data writing apparatus applied to a proxy server, as shown in fig. 8A, which is a structural diagram of the apparatus, including:

an obtaining module 811, configured to obtain a data write request, where the data write request includes an original data identifier; a determining module 812, configured to determine a target data table according to the original data identifier and the total number of data tables; a sending module 813, configured to send the data write request to a target database where the target data table is located, so that the target database adds a data item in the target data table according to the data write request, where an index of the data item is a target data identifier corresponding to the original data identifier;

The determining module 812 is further configured to: determining a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of the data tables; the sending module 813 is further configured to: and sending the target data identification to the target database.

The determining module 812 is specifically configured to, when determining the target data table according to the original data identifier and the total number of data tables: calculating the original data identification and the total number of the data tables by using a processing function to obtain a hash value corresponding to the original data identification; wherein the processing function comprises: a processing function capable of making the total number of the hash values obtained after operation the same as the total number of the data tables;

and determining a target data table corresponding to the hash value.

The determining module 812 is specifically configured to: performing remainder operation according to the original data identification and the total amount of the data table; determining a target data identifier corresponding to the original data identifier according to the quotient obtained after the operation; and determining a target data table corresponding to the original data identification according to the remainder obtained after the operation.

Example 10: based on the same application concept as the method, an embodiment of the present application further provides a data reading apparatus applied to a proxy server, as shown in fig. 8B, which is a structural diagram of the apparatus, including:

an obtaining module 821, configured to obtain a data read request, where the data read request includes an original data identifier; a determining module 822, configured to determine a target data table according to the original data identifier and the total number of data tables; a sending module 823, configured to send the data reading request to a target database where the target data table is located, so that the target database reads data from the target data table according to a target data identifier corresponding to the original data identifier; the target data identification is smaller than the original data identification; a receiving module 824, configured to obtain the read data from the target database.

The determining module 822 is further configured to: determining a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of the data tables; the sending module 823 is further configured to: and sending the target data identification to the target database.

The determining module 822 is specifically configured to, when determining the target data table according to the original data identifier and the total number of the data tables: calculating the original data identification and the total number of the data tables by using a processing function to obtain a hash value corresponding to the original data identification; wherein the processing function comprises: a processing function capable of making the total number of the hash values obtained after operation the same as the total number of the data tables;

and determining a target data table corresponding to the hash value.

The determining module 822 is specifically configured to: performing remainder operation according to the original data identification and the total amount of the data table; determining a target data identifier corresponding to the original data identifier according to the quotient obtained after the operation; and determining a target data table corresponding to the original data identification according to the remainder obtained after the operation.

Based on the same application concept as the method, an embodiment of the present application further provides a proxy server, including: a processor and a machine-readable storage medium having stored thereon a plurality of computer instructions, the processor when executing the computer instructions performs:

and acquiring the read data from the target database.

The embodiment of the application also provides a machine-readable storage medium, wherein a plurality of computer instructions are stored on the machine-readable storage medium; the computer instructions when executed perform the following:

or acquiring a data reading request, wherein the data reading request comprises an original data identifier; determining a target data table according to the original data identification and the total number of the data tables; sending the data reading request to a target database where the target data table is located, so that the target database reads data from the target data table according to a target data identifier corresponding to the original data identifier; wherein the target data identification is smaller than the original data identification; and acquiring the read data from the target database.

Referring to fig. 9, which is a structural diagram of a proxy server proposed in the embodiment of the present application, the proxy server 90 may include: a processor 91, a network interface 92, a bus 93, and a memory 94. Memory 94 may be any electronic, magnetic, optical, or other physical storage device that may contain or store information such as executable instructions, data, and the like. For example, the memory 94 may be: RAM (random Access Memory), volatile Memory, non-volatile Memory, flash Memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., a compact disk, a dvd, etc.).

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Furthermore, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method of writing data, the method comprising:

2. The method of claim 1,

after the obtaining the data write request, the method further comprises:

and determining a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of the data tables, and sending the target data identifier to the target database.

3. The method of claim 1,

the determining a target data table according to the original data identifier and the total number of the data tables comprises:

calculating the original data identification and the total number of the data tables by using a processing function to obtain a hash value corresponding to the original data identification; wherein the processing function comprises: a processing function capable of making the total number of the hash values obtained after operation the same as the total number of the data tables;

and determining a target data table corresponding to the hash value.

4. The method according to claim 1 or 2,

the determining a target data table and a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of the data tables includes:

performing remainder operation according to the original data identification and the total amount of the data table;

determining a target data identifier corresponding to the original data identifier according to the quotient obtained after the operation;

and determining a target data table corresponding to the original data identification according to the remainder obtained after the operation.

5. The method according to claim 1 or 2,

the target database includes: a time series database for storing time series data.

6. A method of writing data, the method comprising:

determining a target data table corresponding to the data writing request;

7. The method of claim 6, wherein prior to adding a data item in the target data table according to the data write request, the method further comprises:

determining a target data identifier corresponding to the original data identifier according to the original data identifier and the total number of the preconfigured data tables; alternatively, the first and second electrodes may be,

and acquiring a target data identifier corresponding to the original data identifier from the proxy server.

8. The method of claim 7, wherein determining a target data identifier corresponding to the original data identifier according to the original data identifier and a total number of preconfigured data tables comprises:

and determining a target data identifier corresponding to the original data identifier according to the quotient obtained after the operation.

9. A method of reading data, the method comprising:

and acquiring the read data from the target database.

10. The method of claim 9,

after the obtaining the data read request, the method further includes:

11. The method of claim 9,

and determining a target data table corresponding to the hash value.

12. The method according to claim 9 or 10,

13. The method according to claim 9 or 10,

14. A method of reading data, the method comprising:

determining a target data table corresponding to the data reading request;

and sending the read data to the proxy server.

15. The method according to claim 14, wherein before reading data from the target data table according to the target data id corresponding to the original data id, the method further comprises:

16. The method of claim 15, wherein determining a target data identifier corresponding to the original data identifier according to the original data identifier and a total number of preconfigured data tables comprises:

17. A data writing apparatus, characterized in that the apparatus comprises:

18. A data reading apparatus, characterized in that the apparatus comprises:

19. A proxy server, comprising:

and acquiring the read data from the target database.