CN112699115A - Data storage and query method and device, electronic equipment and storage medium - Google Patents
Data storage and query method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112699115A CN112699115A CN201911013562.9A CN201911013562A CN112699115A CN 112699115 A CN112699115 A CN 112699115A CN 201911013562 A CN201911013562 A CN 201911013562A CN 112699115 A CN112699115 A CN 112699115A
- Authority
- CN
- China
- Prior art keywords
- data
- sub
- field
- stored
- division
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 93
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000013507 mapping Methods 0.000 claims abstract description 82
- 238000004422 calculation algorithm Methods 0.000 claims description 38
- 230000006870 function Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/2443—Stored procedures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24535—Query rewriting; Transformation of sub-queries or views
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the disclosure discloses a data storage and query method, a data storage and query device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a received data storage statement; when the data to be stored in the data storage statement comprises a table division field and at least one preset non-table division field, establishing a mapping relation between the table division field and the non-table division field; and determining the sub-table storage position of the data to be stored according to the sub-table field. According to the embodiment of the invention, after the data storage statement is acquired by the database middleware, the mapping relation between the non-table-division fields and the table-division fields in the data storage statement is established, so that when the data is queried by using the non-table-division fields, the table-division fields can be determined by using the mapping relation, and then the table-division positions to be queried are determined according to the table-division fields, thereby avoiding querying all tables and improving the query efficiency of the table-division storage data.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data storage and query method and apparatus, an electronic device, and a storage medium.
Background
With the gradual increase of data storage, a database usually realizes functions of storing and querying in sub-tables by adding a database middleware, that is, according to values of preset fields in data storage or data query statements, a sub-table to which current data is stored or from which data is queried is calculated. The existing table dividing technology generally uses one dimension or field to divide a table, that is, a table dividing algorithm such as hashing, modulus and the like is used to determine to which table the row of data is stored according to the value of a certain dimension or field. However, this table-dividing technique does not support querying the data stored in the table-dividing table according to another dimension (or field), so that only all the table-dividing tables can be queried in this case, which greatly reduces the query performance of the table-dividing data.
Disclosure of Invention
The embodiment of the disclosure provides a data storage and query method, a data storage and query device, electronic equipment and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a data storage method.
Specifically, the data storage method includes:
acquiring a received data storage statement;
when the data to be stored in the data storage statement comprises a table division field and at least one preset non-table division field, establishing a mapping relation between the table division field and the non-table division field;
and determining the sub-table storage position of the data to be stored according to the sub-table field.
With reference to the first aspect, in a first implementation manner of the first aspect, the present disclosure further includes:
and sending the data to be stored and the sub-table storage position to a database so as to store the data to be stored in the sub-table storage position.
With reference to the first aspect and/or the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the determining, according to the sub-table field, a sub-table storage location of the data to be stored includes:
and determining the sub-table storage position of the data to be stored according to the sub-table field and a preset sub-table algorithm.
In a second aspect, a data query method is provided in an embodiment of the present disclosure.
Specifically, the data query method includes:
acquiring a received data query statement; the data to be queried of the data query statement is stored in a table-splitting mode;
when the query condition in the data query statement comprises a preset non-table-division field, determining a table-division field having a preset mapping relation with the non-table-division field;
and determining the sub-table storage position of the data to be queried of the data query statement according to the sub-table field.
With reference to the second aspect, in a first implementation manner of the second aspect, the present disclosure further includes:
and sending the data query statement and the sub-table storage position to a database so as to obtain the data to be queried of the data query statement from the sub-table storage position of the database.
With reference to the second aspect and/or the first implementation manner of the second aspect, in a second implementation manner of the second aspect, when the query condition in the data query statement includes a preset non-table-partitioning field, determining a table-partitioning field having a preset mapping relationship with the non-table-partitioning field, includes:
acquiring a query condition in the data query statement;
matching the non-table fields in the query conditions with a pre-established mapping relation table;
obtaining the sublist field having a mapping relation with the non-tabular field from the matching result.
With reference to the second aspect, the first implementation manner of the second aspect, and/or the second implementation manner of the second aspect, in a third implementation manner of the second aspect, the determining, according to the table division field, a table division storage location where data to be queried by the data query statement is located includes:
and determining the storage position of the sub-table according to the sub-table field and a preset sub-table algorithm.
In a third aspect, a data storage device is provided in embodiments of the present disclosure.
Specifically, the data storage device includes:
a first obtaining module configured to obtain the received data storage statement;
the establishing module is configured to establish a mapping relation between the table-divided fields and the non-table-divided fields when the data to be stored in the data storage statement comprises the table-divided fields and at least one preset non-table field;
the first determining module is configured to determine a sub-table storage position of the data to be stored according to the sub-table field.
With reference to the third aspect, in a first implementation manner of the third aspect, the present disclosure further includes:
the first sending module is configured to send the data to be stored and the sub-table storage position to a database so as to store the data to be stored in the sub-table storage position.
With reference to the third aspect and/or the first implementation manner of the third aspect, in a second implementation manner of the third aspect, the first determining module includes:
the first determining submodule is configured to determine the sub-table storage position of the data to be stored according to the sub-table field and a preset sub-table algorithm.
The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above-described functions.
In one possible design, the data storage device has a structure including a memory for storing one or more computer instructions that support the data storage device to perform the data storage method of the first aspect, and a processor configured to execute the computer instructions stored in the memory. The data storage device may also include a communication interface for the data storage device to communicate with other devices or a communication network.
In a fourth aspect, a data query apparatus is provided in the embodiments of the present disclosure.
Specifically, the data query apparatus includes:
a second obtaining module configured to obtain the received data query statement; the data to be queried of the data query statement is stored in a table-splitting mode;
the second determination module is configured to determine a sub-table field having a preset mapping relation with a non-sub-table field when a query condition in the data query statement includes the preset non-sub-table field;
and the third determining module is configured to determine a sub-table storage position where the data to be queried of the data query statement is located according to the sub-table field.
With reference to the fourth aspect, in a first implementation manner of the fourth aspect, the present disclosure further includes:
and the second sending module is configured to send the data query statement and the sub-table storage position to a database so as to obtain the data to be queried by the data query statement from the sub-table storage position of the database.
With reference to the fourth aspect and/or the first implementation manner of the fourth aspect, in a second implementation manner of the fourth aspect, the second determining module includes:
a first obtaining sub-module configured to obtain a query condition in the data query statement;
the matching submodule is configured to match the non-table fields in the query conditions with a mapping relation table established in advance;
and the second obtaining submodule is configured to obtain the sub-table fields with mapping relation with the non-table fields from the matching result.
With reference to the fourth aspect, the first implementation manner of the fourth aspect, and/or the second implementation manner of the fourth aspect, in a third implementation manner of the fourth aspect, the third determining module includes:
and the second determining submodule is configured to determine the sub-table storage position according to the sub-table field and a preset sub-table algorithm.
The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above-described functions.
In one possible design, the data query device includes a memory and a processor, the memory is used for storing one or more computer instructions for supporting the data query device to execute the data query method in the second aspect, and the processor is configured to execute the computer instructions stored in the memory. The data query apparatus may further include a communication interface for the data query apparatus to communicate with other devices or a communication network.
In a fifth aspect, embodiments of the present disclosure provide an electronic device, including a memory and at least one processor; wherein the memory is to store one or more computer instructions, wherein the one or more computer instructions are to be executed by the at least one processor to implement the method of the first aspect and/or the second aspect.
In a sixth aspect, the disclosed embodiments provide a computer-readable storage medium for storing computer instructions for a data query device and/or a data storage device, which contains computer instructions for performing any of the methods described above.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
in the embodiment of the disclosure, after a data storage statement of data to be stored in a table is received through a database middleware, when the data to be stored in the data storage statement includes a table dividing field and a non-table dividing field, a mapping relation between the table dividing field and the non-table dividing field is established, and a table dividing storage position of the data to be stored is determined according to the table dividing field. By the method, after the data storage statement is acquired by the database middleware, the mapping relation between the non-table-division fields and the table-division fields in the data storage statement is established, so that when the data is queried by the non-table-division fields, the table-division fields can be determined by the mapping relation, the positions of the tables to be queried are determined according to the table-division fields, query of all the tables is avoided, and query efficiency of the table-division storage data is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 shows a flow diagram of a data storage method according to an embodiment of the present disclosure;
FIG. 2 illustrates a flow diagram of a data query method according to an embodiment of the present disclosure;
FIG. 3 shows a flowchart of step S202 according to the embodiment shown in FIG. 2;
FIG. 4 shows a block diagram of a data storage device according to an embodiment of the present disclosure;
FIG. 5 shows a block diagram of a data query device according to an embodiment of the present disclosure;
FIG. 6 illustrates a block diagram of a second determination module 502 according to the embodiment shown in FIG. 5;
fig. 7 is a schematic structural diagram of an electronic device suitable for implementing a data query and/or data storage method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.
In the present disclosure, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the disclosed features, numbers, steps, behaviors, components, parts, or combinations thereof, and are not intended to preclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or combinations thereof may be present or added.
It should be further noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 shows a flow chart of a data storage method according to an embodiment of the present disclosure. As shown in fig. 1, the data storage method includes the following steps:
in step S101, a received data storage statement is acquired;
in step S102, when the data to be stored in the data storage statement includes a table-partitioned field and at least one preset non-table-partitioned field, establishing a mapping relationship between the table-partitioned field and the non-table-partitioned field;
in step S103, a sub-table storage location of the data to be stored is determined according to the sub-table field.
In this embodiment, the data storage method may be executed on the database middleware. The database middleware is positioned between the user and the database and is used for receiving the database operation statements sent by the user and forwarding the data operation statements to the database, and the database returns the operation results to the database middleware after executing the database operation statements; and the database middleware forwards the operation result to the user.
After receiving the data storage statement, the database middleware firstly analyzes the data storage statement and extracts a field included in the data to be stored and a field value corresponding to the field. The sub-table algorithm usually calculates the actual sub-table storage location according to the field value corresponding to a predetermined field in the database table.
For example, the database middleware receives the data storage statement as follows:
insert into user(userId,phone,name,city)values(12345,'13712345678',
'terry','shanghai')
the data storage statement represents storing a row of data in a data table with a table name "user", the row of data including the following fields "userId, phone, name, city", and the corresponding field values are "12345," '13712345678, ",' term," 'shanghai' ", respectively. In the sub-table storage manner, data in the table "user" may be stored in a plurality of sub-tables, that is, the data is stored in the table "user" as viewed by a user, and actually is stored in a plurality of sub-tables corresponding to the table "user" in the database in a dispersed manner.
When storing the sub-table, the sub-table position to which the row of data is actually stored is calculated for one of the fields, for example, the sub-table algorithm corresponds to the field "userld", that is, the sub-table field in the data storage statement is "userld", and the corresponding field value is "12345". A sub-table field may be understood as a field in a data table used to calculate a storage location of a sub-table, and a non-sub-table field may be one or more fields other than a sub-table field.
According to the existing table division algorithm, the database middleware directly calculates the table division storage position according to the field value "12345" of the table division field "userId" in the data storage statement, and sends the table division storage position and the data to be stored in the data storage statement to the database for storage. However, in the data stored in this way, only the field value corresponding to the sub-table field "userld" can be used as an index for querying subsequently, but the field values corresponding to other non-sub-table fields cannot be used as indexes for querying, and if the query condition of the data query statement includes the field values corresponding to the non-sub-table fields, the query result can be obtained only by querying all sub-tables, so that the query efficiency is low.
Therefore, in the embodiment of the present disclosure, when the to-be-stored data in the received data storage statement includes the table-partitioned field and the non-table-partitioned field, a mapping relationship is first established between the table-partitioned field and the non-table-partitioned field, and then a table-partitioned storage location to which the to-be-stored data is to be actually stored is determined according to the non-table field. The established mapping relation can be stored in a mapping table, so that after a data query statement is received subsequently, if the query condition of the data query statement comprises a non-sub-table field, the corresponding sub-table field can be found according to the mapping relation, the sub-table storage position where the data to be queried is stored is further determined according to the sub-table field, the data to be queried is obtained from the sub-table storage position, the data to be queried does not need to be obtained by querying all sub-tables, and the data query efficiency is greatly improved.
Still taking the above-mentioned data storage statement "insert inter user (userId, phone, name, city) values (12345, '13712345678', 'term', 'shanghai')", the database middleware may determine, according to a preset table-splitting algorithm, that the data to be stored in the data storage statement includes a table-splitting field "userId" and a non-table field "phone", so that a mapping relationship between the table-splitting field "userId ═ 12345" and the non-table field "phone ═ 13712345678" may be established. It should be noted that a mapping relationship between a preset sub-table field and a preset non-table field may be established, and the preset sub-table field may be determined according to actual storage data of the data table, which is not limited herein.
The mapping relationship may be stored in a mapping table established in advance, and may be represented as follows: user _ userId _ phone _ mapping _ tbl (phone varchar (255), userId int).
According to a preset sub-table algorithm, the sub-table storage position to which the line of data is actually stored can be calculated according to the field value "12345" of the sub-table field userId. For example, if the preset sub-table algorithm obtains the sub-table storage location by taking the modulus, the result obtained by taking the modulus of "userId" of 12345 "as 100 is 45, that is, the sub-table storage location where the data is to be stored is the 45 th sub-table. After that, the database middleware may send the data to be stored and the sub-table storage location to the database, so that the database stores the data to be stored in the 45 th sub-table.
In the embodiment of the disclosure, after a data storage statement of data to be stored in a table is received through a database middleware, when the data to be stored in the data storage statement includes a table dividing field and a non-table dividing field, a mapping relation between the table dividing field and the non-table dividing field is established, and a table dividing storage position of the data to be stored is determined according to the table dividing field. By the method, after the data storage statement is acquired by the database middleware, the mapping relation between the non-table-division fields and the table-division fields in the data storage statement is established, so that when the data is queried by the non-table-division fields, the table-division fields can be determined by the mapping relation, the positions of the tables to be queried are determined according to the table-division fields, query of all the tables is avoided, and query efficiency of the table-division storage data is improved.
In an optional implementation manner of this embodiment, the method further includes the following steps:
and sending the data to be stored and the sub-table storage position to a database so as to store the data to be stored in the sub-table storage position.
In the optional implementation manner, the database middleware establishes a mapping relationship between the sub-table field and the non-sub-table field, and after determining the sub-table storage location of the data to be stored according to the sub-table field, the data to be stored and the sub-table storage location may be sent to the database, so that the database stores the data to be stored into the sub-table corresponding to the sub-table storage location.
In an optional implementation manner of this embodiment, the step S103, that is, the step of determining the sub-table storage location of the data to be stored according to the sub-table field, further includes the following steps:
and determining the sub-table storage position of the data to be stored according to the sub-table field and a preset sub-table algorithm.
In this optional implementation manner, in order to store the data to be stored in the data storage statement sent by the user into the data table specified by the user, when the sub-table storage is adopted, it is further required to determine, according to a preset sub-table algorithm, to which sub-table corresponding to the data table specified by the user the data to be stored needs to be stored. Therefore, which field in the data table is a sub-table field can be preset, and the sub-table field included in the data to be stored currently is processed according to a preset sub-table algorithm when the sub-table is stored, so that the sub-table storage position to which the data to be stored is determined. The preset table splitting algorithm may include a calculation manner how to obtain the table splitting storage location according to the table splitting field, for example, the calculation manner may be a hash function, a modulo function, or the like, and then the table splitting storage location may be obtained by performing hash calculation or modulo calculation on the table splitting storage value in the data to be stored.
FIG. 2 shows a flow diagram of a data query method according to an embodiment of the present disclosure. As shown in fig. 2, the data query method includes the following steps:
in step S201, a received data query statement is acquired; the data to be queried of the data query statement is stored in a table-splitting mode;
in step S202, when the query condition in the data query statement includes a preset non-table-division field, determining a table-division field having a preset mapping relationship with the non-table-division field;
in step S203, a sub-table storage location where the data to be queried by the data query statement is located is determined according to the sub-table field.
In this embodiment, the data query method may be executed on the database middleware, and the data for which the data query method is directed may be stored by using the data storage method as described above. The specific storage details may be referred to in the above description of the data storage method, and are not described herein again.
After the database middleware receives the data query statement, if the query condition comprises a preset non-table-division field, the preset table-division field is a field value which establishes a mapping relation with the table-division field when the data is stored.
For example, the database middleware receives the query statement as follows:
select*from user where phone＝'13712345678'
the data query statement is used to query the "phone" from the table "user" for data whose field value is '13712345678'. It is assumed that the data in the table "user" is stored based on the data storage manner described by way of example in the above-described data storage method embodiment. Then, it can be known that the sublist field in the table "user" is "userId", and "phone" is a non-sublist field, however, when storing data in the above embodiment, a mapping relationship between the non-table-division field "phone ═ 13712345678'" and the table-division field "userId ═ 12345" is established, therefore, if the sublist field corresponding to "phone" 13712345678' "in the query condition is" userId "12345" can be obtained through the above mapping relationship, the sublist storage location where the data to be queried is stored can be determined through the sublist field "userId" 12345 ", for example, modulo 100 is taken for" userId "12345, the result is 45, then it can be determined that the data to be queried is stored in the 45 th sub-table corresponding to the table "user", the database middleware may then send the sub-table storage location "45 th sub-table" and the data query statement to the database, so that the database can obtain the data to be queried by the data query statement from the 45 th sub-table.
By the method, when the query condition of the data query statement comprises the non-sub-table field, query data can be quickly obtained from the sub-tables of the database according to the mapping relation between the pre-established sub-table field and the non-sub-table field, and the speed of querying the data by the sub-tables is improved.
In an optional implementation manner of this embodiment, the method further includes the following steps:
and sending the data query statement and the sub-table storage position to a database so as to obtain the data to be queried of the data query statement from the sub-table storage position of the database.
In this optional implementation manner, after the database middleware determines the sub-table storage location where the data to be queried by the data query statement is located, the data query statement and the sub-table storage location may be sent to the database, so that the database may query from the sub-table storage location according to the query condition in the data query statement to obtain a query result.
In an optional implementation manner of this embodiment, as shown in fig. 3, in the step S202, that is, when the query condition in the data query statement includes a preset non-table-splitting field, the step of determining a table-splitting field having a preset mapping relationship with the non-table-splitting field further includes the following steps:
in step S301, obtaining a query condition in the data query statement;
in step S302, matching the non-table fields in the query condition with a pre-established mapping relationship table;
in step S303, the tabular field having a mapping relationship with the non-tabular field is obtained from the matching result.
In this optional implementation manner, if the query condition of the data query statement received by the database middleware includes a non-table-splitting field, the non-table-splitting field may be matched with a mapping relationship table established in advance, where the mapping relationship table is established in a data storage stage, and may specifically participate in the description of the data storage method, which is not limited herein.
If the matching result is obtained from the mapping relation table, the mapping relation is established between the non-table-division field and the table-division field in advance, so that the table-division field having the mapping relation with the non-table-division field can be determined from the matching result, and the table-division storage position of the data to be inquired can be determined according to the value corresponding to the table-division field.
In an optional implementation manner of this embodiment, the step S203, that is, the step of determining, according to the table-divided field, a table-divided storage location where data to be queried by the data query statement is located further includes the following steps:
and determining the storage position of the sub-table according to the sub-table field and a preset sub-table algorithm.
In this alternative implementation, for the data stored in the sub-table, the data table indicated in the data query statement is a logical data table, that is, the data table viewed by the user, and the data in the logical data table is actually stored in a plurality of sub-tables of the database in a scattered manner. Therefore, after the data query statement is received, it is further required to determine to which sub-tables the data to be queried is actually stored according to a preset sub-table algorithm. In the stage of storing the sub-tables, the middleware of the database processes the sub-table fields included in the data to be stored by using a sub-table algorithm, determines the sub-table storage position to which the data to be stored is stored, and stores the data to be stored into the sub-table storage position by using the database. In the data query stage, the sub-table fields can be processed according to a preset sub-table algorithm to obtain the sub-table storage positions where the data to be queried are located. If the query condition of the data query statement includes the non-table-division field, the corresponding table-division field can be determined according to the non-table-division field and a preset mapping relation table, and then the table-division storage position can be determined according to the table-division field and a preset table-division algorithm.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods.
Fig. 4 shows a block diagram of a data storage device according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 4, the data storage device includes:
a first obtaining module 401 configured to obtain the received data storage statement;
an establishing module 402, configured to establish a mapping relationship between a sub-table field and a non-sub-table field when data to be stored in the data storage statement includes the sub-table field and at least one preset non-sub-table field;
a first determining module 403, configured to determine a sub-table storage location of the data to be stored according to the sub-table field.
In this embodiment, the data storage device may be located in the database middleware. The database middleware is positioned between the user and the database and is used for receiving the database operation statements sent by the user and forwarding the data operation statements to the database, and the database returns the operation results to the database middleware after executing the database operation statements; and the database middleware forwards the operation result to the user.
After receiving the data storage statement, the database middleware firstly analyzes the data storage statement and extracts a field included in the data to be stored and a field value corresponding to the field. The sub-table algorithm usually calculates the actual sub-table storage location according to the field value corresponding to a predetermined field in the database table.
For example, the database middleware receives the data storage statement as follows:
insert into user(userId,phone,name,city)values(12345,'13712345678','terry','shanghai')
the data storage statement represents storing a row of data in a data table with a table name "user", the row of data including the following fields "userId, phone, name, city", and the corresponding field values are "12345," '13712345678, ",' term," 'shanghai' ", respectively. In the sub-table storage manner, data in the table "user" may be stored in a plurality of sub-tables, that is, the data is stored in the table "user" as viewed by a user, and actually is stored in a plurality of sub-tables corresponding to the table "user" in the database in a dispersed manner.
When storing the sub-table, the sub-table position to which the row of data is actually stored is calculated for one of the fields, for example, the sub-table algorithm corresponds to the field "userld", that is, the sub-table field in the data storage statement is "userld", and the corresponding field value is "12345". A sub-table field may be understood as a field in a data table used to calculate a storage location of a sub-table, and a non-sub-table field may be one or more fields other than a sub-table field.
According to the existing table division algorithm, the database middleware directly calculates the table division storage position according to the field value "12345" of the table division field "userId" in the data storage statement, and sends the table division storage position and the data to be stored in the data storage statement to the database for storage. However, in the data stored in this way, only the field value corresponding to the sub-table field "userld" can be used as an index for querying subsequently, but the field values corresponding to other non-sub-table fields cannot be used as indexes for querying, and if the query condition of the data query statement includes the field values corresponding to the non-sub-table fields, the query result can be obtained only by querying all sub-tables, so that the query efficiency is low.
Therefore, in the embodiment of the present disclosure, when the to-be-stored data in the received data storage statement includes the table-partitioned field and the non-table-partitioned field, a mapping relationship is first established between the table-partitioned field and the non-table-partitioned field, and then a table-partitioned storage location to which the to-be-stored data is to be actually stored is determined according to the non-table field. The established mapping relation can be stored in a mapping table, so that after a data query statement is received subsequently, if the query condition of the data query statement comprises a non-sub-table field, the corresponding sub-table field can be found according to the mapping relation, the sub-table storage position where the data to be queried is stored is further determined according to the sub-table field, the data to be queried is obtained from the sub-table storage position, the data to be queried does not need to be obtained by querying all sub-tables, and the data query efficiency is greatly improved.
Still taking the above-mentioned data storage statement "insert inter user (userId, phone, name, city) values (12345, '13712345678', 'term', 'shanghai')", the database middleware may determine, according to a preset table-splitting algorithm, that the data to be stored in the data storage statement includes a table-splitting field "userId" and a non-table field "phone", so that a mapping relationship between the table-splitting field "userId ═ 12345" and the non-table field "phone ═ 13712345678" may be established. It should be noted that a mapping relationship between a preset sub-table field and a preset non-table field may be established, and the preset sub-table field may be determined according to actual storage data of the data table, which is not limited herein.
The mapping relationship may be stored in a mapping table established in advance, and may be represented as follows: user _ userId _ phone _ mapping _ tbl (phone varchar (255), userId int).
According to a preset sub-table algorithm, the sub-table storage position to which the line of data is actually stored can be calculated according to the field value "12345" of the sub-table field userId. For example, if the preset sub-table algorithm obtains the sub-table storage location by taking the modulus, the result obtained by taking the modulus of "userId" of 12345 "as 100 is 45, that is, the sub-table storage location where the data is to be stored is the 45 th sub-table. After that, the database middleware may send the data to be stored and the sub-table storage location to the database, so that the database stores the data to be stored in the 45 th sub-table.
In the embodiment of the disclosure, after a data storage statement of data to be stored in a table is received through a database middleware, when the data to be stored in the data storage statement includes a table dividing field and a non-table dividing field, a mapping relation between the table dividing field and the non-table dividing field is established, and a table dividing storage position of the data to be stored is determined according to the table dividing field. By the method, after the data storage statement is acquired by the database middleware, the mapping relation between the non-table-division fields and the table-division fields in the data storage statement is established, so that when the data is queried by the non-table-division fields, the table-division fields can be determined by the mapping relation, the positions of the tables to be queried are determined according to the table-division fields, query of all the tables is avoided, and query efficiency of the table-division storage data is improved.
In an optional implementation manner of this embodiment, the apparatus further includes:
the first sending module is configured to send the data to be stored and the sub-table storage position to a database so as to store the data to be stored in the sub-table storage position.
In the optional implementation manner, the database middleware establishes a mapping relationship between the sub-table field and the non-sub-table field, and after determining the sub-table storage location of the data to be stored according to the sub-table field, the data to be stored and the sub-table storage location may be sent to the database, so that the database stores the data to be stored into the sub-table corresponding to the sub-table storage location.
In an optional implementation manner of this embodiment, the first determining module 403 includes:
the first determining submodule is configured to determine the sub-table storage position of the data to be stored according to the sub-table field and a preset sub-table algorithm.
In this optional implementation manner, in order to store the data to be stored in the data storage statement sent by the user into the data table specified by the user, when the sub-table storage is adopted, it is further required to determine, according to a preset sub-table algorithm, to which sub-table corresponding to the data table specified by the user the data to be stored needs to be stored. Therefore, which field in the data table is a sub-table field can be preset, and the sub-table field included in the data to be stored currently is processed according to a preset sub-table algorithm when the sub-table is stored, so that the sub-table storage position to which the data to be stored is determined. The preset table splitting algorithm may include a calculation manner how to obtain the table splitting storage location according to the table splitting field, for example, the calculation manner may be a hash function, a modulo function, or the like, and then the table splitting storage location may be obtained by performing hash calculation or modulo calculation on the table splitting storage value in the data to be stored.
Fig. 5 shows a block diagram of a data query apparatus according to an embodiment of the present disclosure, which may be implemented as part of or all of an electronic device by software, hardware, or a combination of the two. As shown in fig. 5, the data query apparatus includes:
a second obtaining module 501 configured to obtain the received data query statement; the data to be queried of the data query statement is stored in a table-splitting mode;
a second determining module 502, configured to determine, when a query condition in the data query statement includes a preset non-table-division field, a table-division field having a preset mapping relationship with the non-table-division field;
a third determining module 503, configured to determine, according to the table-divided field, a table-divided storage location where the data to be queried by the data query statement is located.
In this embodiment, the data query device may be located on the database middleware, and the data targeted by the data query device may be stored by using the data storage device as described above. The specific storage details may be referred to in the above description of the data storage device and will not be described herein.
After the database middleware receives the data query statement, if the query condition comprises a preset non-table-division field, the preset table-division field is a field value which establishes a mapping relation with the table-division field when the data is stored.
For example, the database middleware receives the query statement as follows:
select*from user where phone＝'13712345678'
the data query statement is used to query the "phone" from the table "user" for data whose field value is '13712345678'. It is assumed that the data in the table "user" is stored based on the data storage manner described by way of example in the above-described data storage method embodiment. Then, it can be known that the sublist field in the table "user" is "userId", and "phone" is a non-sublist field, however, when storing data in the above embodiment, a mapping relationship between the non-table-division field "phone ═ 13712345678'" and the table-division field "userId ═ 12345" is established, therefore, if the sublist field corresponding to "phone" 13712345678' "in the query condition is" userId "12345" can be obtained through the above mapping relationship, the sublist storage location where the data to be queried is stored can be determined through the sublist field "userId" 12345 ", for example, modulo 100 is taken for" userId "12345, the result is 45, then it can be determined that the data to be queried is stored in the 45 th sub-table corresponding to the table "user", the database middleware may then send the sub-table storage location "45 th sub-table" and the data query statement to the database, so that the database can obtain the data to be queried by the data query statement from the 45 th sub-table.
By the method, when the query condition of the data query statement comprises the non-sub-table field, query data can be quickly obtained from the sub-tables of the database according to the mapping relation between the pre-established sub-table field and the non-sub-table field, and the speed of querying the data by the sub-tables is improved.
In an optional implementation manner of this embodiment, the apparatus further includes:
and the second sending module is configured to send the data query statement and the sub-table storage position to a database so as to obtain the data to be queried by the data query statement from the sub-table storage position of the database.
In this optional implementation manner, after the database middleware determines the sub-table storage location where the data to be queried by the data query statement is located, the data query statement and the sub-table storage location may be sent to the database, so that the database may query from the sub-table storage location according to the query condition in the data query statement to obtain a query result.
In an optional implementation manner of this embodiment, as shown in fig. 6, the second determining module 502 includes:
a first obtaining sub-module 601 configured to obtain a query condition in the data query statement;
a matching sub-module 602 configured to match non-table fields in the query condition with a pre-established mapping relationship table;
a second obtaining submodule 603 configured to obtain the sub-table fields having mapping relation with the non-table fields from the matching result.
In this optional implementation manner, if the query condition of the data query statement received by the database middleware includes a non-table-splitting field, the non-table-splitting field may be matched with a mapping relationship table established in advance, where the mapping relationship table is established in a data storage stage, and may specifically participate in the description of the data storage method, which is not limited herein.
If the matching result is obtained from the mapping relation table, the mapping relation is established between the non-table-division field and the table-division field in advance, so that the table-division field having the mapping relation with the non-table-division field can be determined from the matching result, and the table-division storage position of the data to be inquired can be determined according to the value corresponding to the table-division field.
In an optional implementation manner of this embodiment, the third determining module 503 includes:
and the second determining submodule is configured to determine the sub-table storage position according to the sub-table field and a preset sub-table algorithm.
In this alternative implementation, for the data stored in the sub-table, the data table indicated in the data query statement is a logical data table, that is, the data table viewed by the user, and the data in the logical data table is actually stored in a plurality of sub-tables of the database in a scattered manner. Therefore, after the data query statement is received, it is further required to determine to which sub-tables the data to be queried is actually stored according to a preset sub-table algorithm. In the stage of storing the sub-tables, the middleware of the database processes the sub-table fields included in the data to be stored by using a sub-table algorithm, determines the sub-table storage position to which the data to be stored is stored, and stores the data to be stored into the sub-table storage position by using the database. In the data query stage, the sub-table fields can be processed according to a preset sub-table algorithm to obtain the sub-table storage positions where the data to be queried are located. If the query condition of the data query statement includes the non-table-division field, the corresponding table-division field can be determined according to the non-table-division field and a preset mapping relation table, and then the table-division storage position can be determined according to the table-division field and a preset table-division algorithm.
The embodiment of the present disclosure also provides an electronic device, as shown in fig. 7, including at least one processor 701; and a memory 702 communicatively coupled to the at least one processor 701; wherein the memory 702 stores instructions executable by the at least one processor 701 to perform, by the at least one processor 701, the steps of:
acquiring a received data storage statement;
when the data to be stored in the data storage statement comprises a table division field and at least one preset non-table division field, establishing a mapping relation between the table division field and the non-table division field;
and determining the sub-table storage position of the data to be stored according to the sub-table field.
Wherein, still include:
and sending the data to be stored and the sub-table storage position to a database so as to store the data to be stored in the sub-table storage position.
Determining the sub-table storage position of the data to be stored according to the sub-table field, wherein the determining comprises:
and determining the sub-table storage position of the data to be stored according to the sub-table field and a preset sub-table algorithm.
In the electronic device provided in the embodiment of the present disclosure, the memory 702 further stores instructions executable by the at least one processor 701, and the instructions are executed by the at least one processor 701 to implement: acquiring a received data query statement; the data to be queried of the data query statement is stored in a table-splitting mode;
when the query condition in the data query statement comprises a preset non-table-division field, determining a table-division field having a preset mapping relation with the non-table-division field;
and determining the sub-table storage position of the data to be queried of the data query statement according to the sub-table field.
Wherein, still include:
and sending the data query statement and the sub-table storage position to a database so as to obtain the data to be queried of the data query statement from the sub-table storage position of the database.
When the query condition in the data query statement includes a preset non-table-splitting field, determining a table-splitting field having a preset mapping relationship with the non-table-splitting field, including:
acquiring a query condition in the data query statement;
matching the non-table fields in the query conditions with a pre-established mapping relation table;
obtaining the sublist field having a mapping relation with the non-tabular field from the matching result.
Determining a sub-table storage location where the data to be queried of the data query statement is located according to the sub-table field, including:
and determining the storage position of the sub-table according to the sub-table field and a preset sub-table algorithm. Specifically, the processor 701 and the memory 702 may be connected by a bus or by other means, and fig. 7 illustrates an example of connection by a bus. Memory 702, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The processor 701 executes various functional applications of the device and data processing by executing nonvolatile software programs, instructions, and modules stored in the memory 702, that is, implements the above-described method in the embodiments of the present disclosure.
The memory 702 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store historical data of shipping network traffic, and the like. Further, the memory 702 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the electronic device optionally includes a communications component 703 and the memory 702 optionally includes memory remotely located from the processor 701, which may be connected to an external device through the communications component 703. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more modules are stored in the memory 702, and when executed by the one or more processors 701, perform the above-described methods in the embodiments of the present disclosure.
The product can execute the method provided by the embodiment of the disclosure, has corresponding functional modules and beneficial effects of the execution method, and reference can be made to the method provided by the embodiment of the disclosure for technical details which are not described in detail in the embodiment.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present disclosure may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
As another aspect, the present disclosure also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the above-described embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the inventive concept. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Claims (10)
1. A method of storing data, comprising:
acquiring a received data storage statement;
when the data to be stored in the data storage statement comprises a table division field and at least one preset non-table division field, establishing a mapping relation between the table division field and the non-table division field;
and determining the sub-table storage position of the data to be stored according to the sub-table field.
2. The method of claim 1, further comprising:
and sending the data to be stored and the sub-table storage position to a database so as to store the data to be stored in the sub-table storage position.
3. The method according to claim 1 or 2, wherein determining the sub-table storage location of the data to be stored according to the sub-table field comprises:
and determining the sub-table storage position of the data to be stored according to the sub-table field and a preset sub-table algorithm.
4. A method for querying data, comprising:
acquiring a received data query statement; the data to be queried of the data query statement is stored in a table-splitting mode;
when the query condition in the data query statement comprises a preset non-table-division field, determining a table-division field having a preset mapping relation with the non-table-division field;
and determining the sub-table storage position of the data to be queried of the data query statement according to the sub-table field.
5. The method of claim 4, further comprising:
and sending the data query statement and the sub-table storage position to a database so as to obtain the data to be queried of the data query statement from the sub-table storage position of the database.
6. The method of claim 4 or 5, wherein when the query condition in the data query statement includes a preset non-table-divided field, determining a table-divided field having a preset mapping relation with the non-table-divided field comprises:
acquiring a query condition in the data query statement;
matching the non-table fields in the query conditions with a pre-established mapping relation table;
obtaining the sublist field having a mapping relation with the non-tabular field from the matching result.
7. A data storage device, comprising:
a first obtaining module configured to obtain the received data storage statement;
the establishing module is configured to establish a mapping relation between the table-divided fields and the non-table-divided fields when the data to be stored in the data storage statement comprises the table-divided fields and at least one preset non-table field;
the first determining module is configured to determine a sub-table storage position of the data to be stored according to the sub-table field.
8. A data query apparatus, comprising:
a second obtaining module configured to obtain the received data query statement; the data to be queried of the data query statement is stored in a table-splitting mode;
the second determination module is configured to determine a sub-table field having a preset mapping relation with a non-sub-table field when a query condition in the data query statement includes the preset non-sub-table field;
and the third determining module is configured to determine a sub-table storage position where the data to be queried of the data query statement is located according to the sub-table field.
9. An electronic device comprising a memory and at least one processor; wherein,
the memory is to store one or more computer instructions, wherein the one or more computer instructions are to be executed by the at least one processor to implement the method of any one of claims 1-6.
10. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by at least one processor, implement the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911013562.9A CN112699115A (en) | 2019-10-23 | 2019-10-23 | Data storage and query method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911013562.9A CN112699115A (en) | 2019-10-23 | 2019-10-23 | Data storage and query method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112699115A true CN112699115A (en) | 2021-04-23 |
Family
ID=75505228
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911013562.9A Pending CN112699115A (en) | 2019-10-23 | 2019-10-23 | Data storage and query method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699115A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255315A (en) * | 2021-07-19 | 2021-08-13 | 杭州天谷信息科技有限公司 | Method and system for configuring and generating evidence chain |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020193A (en) * | 2012-12-03 | 2013-04-03 | 北京奇虎科技有限公司 | Method and equipment for processing database operation request |
| CN104714972A (en) * | 2013-12-17 | 2015-06-17 | 中国银联股份有限公司 | Database sub-table establishing and searching method |
| CN106649435A (en) * | 2016-09-07 | 2017-05-10 | 努比亚技术有限公司 | Data query device and method of querying data |
| CN108628986A (en) * | 2018-04-27 | 2018-10-09 | 深圳市牛鼎丰科技有限公司 | Data query method, apparatus, computer equipment and storage medium |
| CN109344157A (en) * | 2018-09-20 | 2019-02-15 | 深圳市牛鼎丰科技有限公司 | Read and write abruption method, apparatus, computer equipment and storage medium |
-
2019
- 2019-10-23 CN CN201911013562.9A patent/CN112699115A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020193A (en) * | 2012-12-03 | 2013-04-03 | 北京奇虎科技有限公司 | Method and equipment for processing database operation request |
| CN104714972A (en) * | 2013-12-17 | 2015-06-17 | 中国银联股份有限公司 | Database sub-table establishing and searching method |
| CN106649435A (en) * | 2016-09-07 | 2017-05-10 | 努比亚技术有限公司 | Data query device and method of querying data |
| CN108628986A (en) * | 2018-04-27 | 2018-10-09 | 深圳市牛鼎丰科技有限公司 | Data query method, apparatus, computer equipment and storage medium |
| CN109344157A (en) * | 2018-09-20 | 2019-02-15 | 深圳市牛鼎丰科技有限公司 | Read and write abruption method, apparatus, computer equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255315A (en) * | 2021-07-19 | 2021-08-13 | 杭州天谷信息科技有限公司 | Method and system for configuring and generating evidence chain |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111522641B (en) | Task scheduling method, device, computer equipment and storage medium | |
| CN107872528B (en) | Message pushing method and device | |
| US9853906B2 (en) | Network prioritization based on node-level attributes | |
| CN110413845B (en) | Resource storage method and device based on Internet of things operating system | |
| US8812492B2 (en) | Automatic and dynamic design of cache groups | |
| US10979317B2 (en) | Service registration method and usage method, and related apparatus | |
| CN112598514B (en) | Cross-chain transaction management method, cross-chain platform and medium based on block chain | |
| CN102263701A (en) | Queue regulation method and device | |
| CN105119997A (en) | Data processing method of cloud computing system | |
| US20170374154A1 (en) | Generating a response to a client device in an internet of things domain | |
| CN112685175B (en) | Construction method and device of service topological graph and computer readable storage medium | |
| CN112699115A (en) | Data storage and query method and device, electronic equipment and storage medium | |
| CN104580428A (en) | Data routing method, data management device and distributed storage system | |
| CN111064776A (en) | Method and device for generating blocks in block chain and storage medium | |
| US20160150010A1 (en) | Information processing apparatus, data save method, and information processing system | |
| CN110597808B (en) | Distributed database table connection method, device, system, server and medium | |
| CN110955460B (en) | Service process starting method and device, electronic equipment and storage medium | |
| CN117082029A (en) | Access method and device based on line cache, electronic equipment and storage medium | |
| CN108681587B (en) | bitmap generation method, device, equipment and storage medium | |
| CN110798492B (en) | Data storage method and device and data processing system | |
| CN107193654B (en) | Resource operation method and device of distributed system and distributed system | |
| CN111078379B (en) | Task scheduling method, system, mobile terminal and storage medium | |
| US20200285641A1 (en) | Method, apparatus, system, server, and storage medium for connecting tables stored at distributed database | |
| WO2016082432A1 (en) | Data query method and device | |
| CN112954084A (en) | Edge computing processing method, network function example and edge service management and control center |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210423 |