CN112395293A - Warehouse and table dividing method, warehouse and table dividing device, warehouse and table dividing equipment and storage medium - Google Patents
Warehouse and table dividing method, warehouse and table dividing device, warehouse and table dividing equipment and storage medium Download PDFInfo
- Publication number
- CN112395293A CN112395293A CN202011363595.9A CN202011363595A CN112395293A CN 112395293 A CN112395293 A CN 112395293A CN 202011363595 A CN202011363595 A CN 202011363595A CN 112395293 A CN112395293 A CN 112395293A
- Authority
- CN
- China
- Prior art keywords
- data
- stored
- existing
- database
- weight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000000638 solvent extraction Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 abstract description 8
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a database-dividing and table-dividing method, which judges whether the existing database of a target storage system has associated data with the same data identification as the data to be stored according to the data identification of the data to be stored when receiving the data to be stored, if so, stores the data to be stored into the existing data table where the associated data is located according to the data identification to realize associated storage, and if not, stores the data to be stored into the existing data table according to a preset average distribution rule, thereby further realizing the associated storage of the data on the basis of realizing the average distribution of the data in the database-dividing and table-dividing process, and enabling a user to find the target data more quickly and conveniently when needing to check and modify the data. The invention also discloses a warehouse and table dividing device, a warehouse and table dividing device and a storage medium, which have the beneficial effects.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a database and table dividing method, a database and table dividing device, database and table dividing equipment and a storage medium.
Background
The database and table dividing is to divide the original independent database into a plurality of databases and divide the data large table into a plurality of data tables, so that the data amount of a single database and a single data table is reduced, and the aim of improving the performance of the database is fulfilled.
The existing database and table dividing method usually uses the principle of average distribution, and makes allocation rules according to the existing database and the existing data table, and uniformly distributes the data in each database and each data table. Although the database-dividing and table-dividing method realizes the even distribution of data in the database, the distribution of the data has no rules, so that some related data are likely to be distributed in different data tables or even different databases, and when the stored data needs to be checked and modified, all the databases and data tables need to be searched globally, which is very inconvenient.
Disclosure of Invention
The invention aims to provide a database and table dividing method, a database and table dividing device, database and table dividing equipment and a storage medium, which are used for further realizing the associated storage of data on the basis of realizing the average distribution of the data, so that a user can find target data more quickly and conveniently when needing to check and modify the data.
In order to solve the above technical problems, the present invention provides a database and table dividing method, comprising:
when data to be stored is received, acquiring a data identifier of the data to be stored;
if the existing database of the target storage system has the associated data with the same data identification as the data to be stored, storing the data to be stored into the existing data table where the associated data is located;
and if the associated data does not exist in the existing database, storing the data to be stored into an existing data table according to a preset average distribution rule.
Optionally, the obtaining of the data identifier of the data to be stored specifically includes:
and identifying the preset field of the data to be stored to obtain the data identification of the data to be stored.
Optionally, the storing the data to be stored into the existing data table according to the preset average distribution rule specifically includes:
determining the data type of the data to be stored according to the data identifier of the data to be stored, and determining the weight of the data to be stored according to the data type of the data to be stored according to a preset weight reference;
acquiring the weight sum of the data in the existing data table, and storing the data to be stored into the existing data table with the minimum weight sum;
the weight reference is established according to the principle that the more the quantity of the data type is, the more the weight of the data type is.
Optionally, storing the data to be stored in the existing data table with the smallest sum of the weights specifically includes:
if the existing data table with the minimum weight sum exists and only one existing data table with the minimum weight sum exists, storing the data to be stored into the existing data table with the minimum weight sum;
and if two or more existing data tables with the minimum weight sum exist, storing the data to be stored into the existing data table with the minimum weight sum, wherein the existing data table has the minimum data.
Optionally, the method further includes:
checking the quantity of data in each existing data table at regular time;
for any two data types, if the difference value between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is smaller than a first preset value, reducing the weight difference value between the data type with the larger weight and the data type with the smaller weight; if the difference value between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is larger than a second preset value, increasing the weight difference value between the data type with the larger weight and the data type with the smaller weight;
wherein the first preset value is smaller than the second preset value.
Optionally, the method further includes:
and storing the sum of the weights of the existing data tables.
Optionally, the method further includes:
if the existing data tables cannot be stored in the data to be stored, a new data table is created, and the table creation time is recorded as the starting time of the new data table for receiving the data to be stored;
and if the existing databases cannot create the new data table, creating a new database, and recording the database creation time as the starting time for receiving the data to be stored by the new database.
In order to solve the above technical problem, the present invention further provides a database and table dividing apparatus, including:
the device comprises a receiving unit, a storage unit and a processing unit, wherein the receiving unit is used for acquiring a data identifier of data to be stored when the data to be stored is received;
the distribution unit is used for storing the data to be stored into an existing data table where the associated data are located if the associated data with the same data identifier as the data to be stored exist; and if the associated data does not exist, storing the data to be stored into the existing data table according to a preset average distribution rule.
In order to solve the above technical problem, the present invention further provides a database and table dividing apparatus, including:
a memory for storing instructions, the instructions comprising any one of the steps of the method for sorting and listing;
a processor to execute the instructions.
In order to solve the above technical problem, the present invention further provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the database/table partitioning method according to any one of the above items.
The database-dividing and table-dividing method provided by the invention judges whether the existing database of the target storage system has the associated data with the same data identification as the data to be stored according to the data identification of the data to be stored when the data to be stored is received, if so, the data to be stored is stored into the existing data table where the associated data is located according to the data identification to realize the associated storage, and if not, the data to be stored is stored into the existing data table according to the preset average distribution rule, thereby further realizing the associated storage of the data on the basis of realizing the average distribution of the data in the database-dividing and table-dividing process, and enabling a user to find the target data more quickly and conveniently when needing to check and modify the data.
The invention also provides a database and table dividing device, a database and table dividing device and a storage medium, which have the beneficial effects and are not described again.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is a flowchart of a database and table partitioning method according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating an embodiment of step S104 in fig. 1 according to the present invention;
fig. 3 is a schematic structural diagram of a database and table partitioning device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a database and table dividing device according to an embodiment of the present invention.
Detailed Description
The core of the invention is to provide a database and table dividing method, a database and table dividing device and a storage medium, which are used for further realizing the associated storage of data on the basis of realizing the average distribution of the data, so that a user can find target data more quickly and conveniently when needing to check and modify the data.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of a database and table partitioning method according to an embodiment of the present invention.
As shown in fig. 1, the method for sorting and tabulating a library provided by the embodiment of the present invention includes:
s101: and when the data to be stored is received, acquiring a data identifier of the data to be stored.
S102: judging whether the existing database of the target storage system has associated data with the same data identifier as the data to be stored; if yes, go to step S103; if not, the process proceeds to step S104.
S103: and storing the data to be stored into the existing data table where the associated data is located.
S104: and storing the data to be stored into the existing data table according to a preset average distribution rule.
It should be noted that, in the embodiments of the present invention, the "data to be stored", "data" and "stored data" all refer to a piece of data occupying a row in the data table, and generally consist of a plurality of fields, and storage spaces occupied by different data are approximately the same.
In a specific implementation, when the storage system receives data to be stored, the content of the data to be stored is identified, and the data identifier of the data to be stored is determined. The "data identifier" refers to an identifier for dividing associated data, and may be a field or a plurality of fields, where a field at a predetermined position in a piece of data is preset as the data identifier of the data. A piece of data can also comprise a plurality of data identifications, and the data are finely classified according to the combination of the data identifications. Determining the data identifier of the data to be stored in step S101, specifically: and identifying a preset field of the data to be stored to obtain a data identifier of the data to be stored.
After determining the data identifier of the data to be stored, checking whether the existing data tables of the existing databases store data with the same data identifier, namely associated data with the data to be stored, if so, storing the data to be stored in the existing data tables where the associated data are located, and if not, storing the data to be stored in the existing data tables according to a preset average distribution rule. The preset average distribution rule may be to store the data to be stored into the data table with the least data number, or to perform a remainder operation on the number of the existing data tables included in the storage system according to the labels (such as order numbers) of the data to be stored, and to place the data to be stored into the existing data tables according to the remainder.
On this basis, if each existing data table is full, a new data table needs to be established under the condition that the existing database allows, or a new database is established under the condition that the existing database is full, then the database partitioning method provided by the embodiment of the present invention may further include:
if all the existing data tables cannot store the data to be stored, a new data table is created, and the table creation time is recorded as the initial time for the new data table to receive the data to be stored;
and if the existing databases cannot create the new data table, creating the new database, and recording the database creating time as the initial time for the new database to receive the data to be stored.
If the existing data table is not full, but the existing data table in which the associated data of the data to be stored is located is full, and the existing data table that is not full can store the data to be stored but is not beneficial to storing the data associated with the data to be stored, it can be considered that the existing data tables cannot store the data to be stored. When the existing data table is built, the table building time is recorded, when the existing data table is full or not full but cannot store the data to be stored, the end time is recorded, and the data which can be stored in the existing data table is the data between the table building time and the end time. Similarly, when the existing database is established, the database establishing time is recorded, when the existing database is full or not full but cannot store the data to be stored, the ending time is recorded, and the data which can be stored in the existing database is the data with the generation time between the database establishing time and the ending time. Through time management, the rough storage position of the data is convenient to determine according to the data generation time, and the time for data retrieval is reduced.
Since different types of data have different properties, such as frequency of generation, different average distribution rules may be established for different data types of data in order to achieve as even distribution of data as possible on the basis of associative storage.
The data type is a dividing basis of the upper level of the data identification, and can be divided according to the user requirement.
According to the database-dividing and table-dividing method provided by the embodiment of the invention, when the data to be stored is received, whether the existing database of the target storage system has the associated data with the same data identifier as the data to be stored is judged according to the data identifier of the data to be stored, if so, the data to be stored is stored into the existing data table where the associated data is located according to the data identifier to realize associated storage, and if not, the data to be stored is stored into the existing data table according to the preset average distribution rule, so that the associated storage of the data is further realized on the basis of realizing the average distribution of the data in the database-dividing and table-dividing process, and a user can find the target data more quickly and conveniently when needing to check and modify the data.
Fig. 2 is a flowchart illustrating a specific implementation manner of step S104 in fig. 1 according to an embodiment of the present invention.
In the prior art, in order to realize the even distribution of data into a data table and a database, a distribution rule is generally established according to an existing database and an existing data table. For example, for order data, a remainder table is taken according to order numbers, and if four data tables are shared, the order numbers of the data to be stored are taken according to four and are equally divided into four tables. When the existing data tables are full, and when data to be stored is received, the data tables need to be newly added (for example, eight data tables are added), at this time, the stored data and the data to be stored in each existing data table are together surplus according to eight, that is, the stored data need to be redistributed to realize the data average distribution. The same applies when a database is newly built. Therefore, the existing database partitioning method is based on the existing data table and the data average distribution mode established by the existing database, all data needs to be redistributed when the data table is newly built and the database is newly established, and data migration needs to be carried out on stored data in the distribution process, so that the method is very inconvenient; when a new data table or database is not needed, the database-dividing and table-dividing manner is, for example, to allocate the data table by using the order number to take the surplus of the total number of the data table, and since the order number is not always continuous, it is often not guaranteed that the data is evenly allocated to each database or each data table.
Therefore, on the basis of the above embodiments, the embodiments of the present invention provide a method for partitioning a library and a table, which can evenly distribute data without data migration. In the method for sorting and tabulating the database provided by the embodiment of the invention, the step S104: storing data to be stored into an existing data table according to a preset average distribution rule, which specifically comprises the following steps:
s201: and determining the data type of the data to be stored according to the data identifier of the data to be stored, and determining the weight of the data to be stored according to the data type of the data to be stored according to a preset weight reference.
S202: and acquiring the weight sum of the data in the existing data table, and storing the data to be stored into the existing data table with the minimum weight sum.
The weight reference is established according to the principle that the more the number of data of the data type is, the more the weight of the data type is.
In a specific implementation, the number of the stored data of each data type in other storage systems may be counted in advance, and the weight of each data type is determined according to the principle that the greater the number of the data type is, the greater the weight of the data type is, for example, the stored data of each type may be sorted from large to small, and then the weights are sequentially divided in a descending manner, or the specific value of the weight is determined according to a ratio.
For step S201, for the data to be stored without the associated data, the data type of the previous level is retrieved according to the data identifier, and then the weight corresponding to the data to be stored is obtained. For example, the data type includes "enterprise merchant", "personal merchant", etc., and the data identification includes "enterprise merchant a", "enterprise merchant B", "personal merchant C", etc. Assume that the weight of "business merchant" is set to 10 and the weight of "individual merchant" is set to 1. Recognizing that the data identifier of the received first piece of data to be stored is 'enterprise merchant A', recognizing the data type of the upper level to determine that the data type of the first piece of data to be stored is 'enterprise merchant', and further determining that the weight of the first piece of data to be stored is 10; similarly, after the data identifier of the received second to-be-stored data is identified as 'personal merchant C', determining that the weight of the second to-be-stored data is 1; and after the data identification of the received third piece of data to be stored is recognized as 'enterprise merchant B', determining the weight of the third piece of data to be stored as 10.
For step S202, the sum of the weights of the data already stored in the existing data tables, i.e. the sum of the weights of the existing data tables, is calculated, and the sum of the weights of the existing data tables is compared, so that the data to be stored is stored in the existing data table with the smallest sum of the weights. If a plurality of existing data tables with the minimum weight sum exist, the existing data tables can be divided according to the labels of the existing data tables, for example, data to be stored are stored into the existing data tables with the minimum labels preferentially; alternatively, the data to be stored may be stored in the existing data table with the smallest sum of the weights of the smallest number of data according to the number of data in each existing data table. Step S202 may specifically include:
if the existing data table with the minimum weight sum exists, storing the data to be stored into the existing data table with the minimum weight sum;
and if two or more existing data tables with the minimum weight sum exist, storing the data to be stored into the existing data table with the minimum weight sum, wherein the existing data table with the minimum weight sum has the minimum data.
In order to avoid calculating the sum of the weights of the existing data tables when the data to be stored is received each time, the database partitioning and table partitioning method provided by the embodiment of the invention further comprises the following steps: and storing the sum of the weights of the existing data tables. A table rule table can be established as shown in table 1, where table _ serial _ num is a table serial number, data _ volume is weight synthesis, and orders _ count is an order table statistical number.
Table 1 a table division rule table provided in an embodiment of the present invention
| table_serial_num | data_volume | orders_count |
| 0 | 10 | 10000 |
| 1 | 100 | 20000 |
Similarly, a library-splitting rule table may also be established for each existing database, as shown in table 2. Wherein db _ serial _ num is the sequence number of the sub-database, start _ time is the storage start time of the sub-database data, and end _ time is the storage end time of the sub-database data.
Table 2 a library rule table according to an embodiment of the present invention
| db_serial_num | start_time | end_time |
| 0 | 2018-10-11 | 2019-10-16 |
| 1 | 2019-10-17 | 2020-11-01 |
Note: the above tables 1 and 2 are merely examples of the table rule table and the library rule table, and are not the same examples as those assumed in the embodiment of the present invention.
Assuming that there are two existing data tables 0 and 1 before the first piece of data to be stored is received, and there is no stored data in both, the sum of the weights of the 0 table and the 1 table is 0. When a first piece of data to be stored, namely 'enterprise merchant A', is received, the first piece of data to be stored is stored into the table 0 according to the label, at the moment, the weight sum of the table 0 is changed into 10, and the weight sum of the table 1 is still 0. When a second piece of data to be stored, namely 'personal merchant C', is received, the second piece of data to be stored is stored into the table 1 according to the comparison of the weight sum, and the weight sum of the table 1 is changed into 1. When a third piece of data to be stored, namely 'enterprise merchant B', is received, the third piece of data to be stored is stored in the table 1 because the weight sum of the table 1 is still smaller than the weight sum of the table 0.
The weight reference established by the embodiment of the invention is substantially based on the associated storage, and the storage space of the associated data is reserved for the data in the existing data table in advance when the data of a certain data type is stored for the first time by predicting the quantity of the data of each data type, so that the data is better evenly distributed on the basis of the associated storage, and the query and the management are convenient.
On this basis, the library and table dividing method provided by the embodiment of the present invention may further include:
checking the quantity of data in each existing data table at regular time;
for any two data types, if the difference value between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is smaller than a first preset value, reducing the weight difference value between the data type with the larger weight and the data type with the smaller weight; if the difference value between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is larger than a second preset value, increasing the weight difference value between the data type with the larger weight and the data type with the smaller weight;
wherein the first preset value is smaller than the second preset value.
The preset weight standard may not meet the actual condition of the data quantity of each data type as time goes on, so the weight standard can be updated at regular time, and the updating work can be carried out at idle time every day.
On the basis of the above detailed description of each embodiment corresponding to the database-dividing and table-dividing method, the invention also discloses a database-dividing and table-dividing device, a database-dividing and table-dividing device and a storage medium corresponding to the above method.
Fig. 3 is a schematic structural diagram of a database and table partitioning device according to an embodiment of the present invention.
As shown in fig. 3, the database and table dividing device provided in the embodiment of the present invention includes:
a receiving unit 301, configured to obtain a data identifier of data to be stored when the data to be stored is received;
the allocating unit 302 is configured to, if there is associated data with the same data identifier as the data to be stored, store the data to be stored in an existing data table where the associated data is located; and if no associated data exists, storing the data to be stored into the existing data table according to a preset average distribution rule.
Further, the device for sorting and tabulating the libraries provided by the embodiment of the present invention may further include:
the checking unit is used for checking the quantity of the data in each existing data table at regular time;
the updating unit is used for reducing the weight difference between the data type with the larger weight and the data type with the smaller weight if the difference between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is smaller than a first preset value for any two data types; if the difference value between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is larger than a second preset value, increasing the weight difference value between the data type with the larger weight and the data type with the smaller weight;
wherein the first preset value is smaller than the second preset value.
The database and table dividing device provided by the embodiment of the invention can also comprise:
and the storage unit is used for storing the weight sum of all the existing data tables.
The database and table dividing device provided by the embodiment of the invention can also comprise:
the creating unit is used for creating a new data table if the existing data tables can not store the data to be stored, and recording the table creating time as the starting time for the new data table to receive the data to be stored; and if the existing databases cannot create the new data table, creating the new database, and recording the database creating time as the initial time for the new database to receive the data to be stored.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
Fig. 4 is a schematic structural diagram of a database and table dividing device according to an embodiment of the present invention.
As shown in fig. 4, the device for sorting and tabulating libraries provided by the embodiment of the present invention includes:
a memory 410 for storing instructions, the instructions comprising the steps of the method for sorting and listing according to any one of the above embodiments;
a processor 420 for executing the instructions.
Among other things, processor 420 may include one or more processing cores, such as a 3-core processor, an 8-core processor, and so forth. The processor 420 may be implemented in at least one hardware form of a Digital Signal Processing (DSP), a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), or a Programmable Logic Array (PLA). Processor 420 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a central Processing unit (cpu); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 420 may be integrated with an image processor GPU (graphics Processing unit) that is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, processor 420 may also include an Artificial Intelligence (AI) (artificial intelligence) processor for processing computational operations related to machine learning.
Memory 410 may include one or more storage media, which may be non-transitory. Memory 410 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 410 is at least used for storing the following computer program 411, wherein after the computer program 411 is loaded and executed by the processor 420, the relevant steps in the database and table partitioning method disclosed in any of the foregoing embodiments can be implemented. In addition, the resources stored by the memory 410 may also include an operating system 412, data 413, and the like, and the storage may be transient storage or permanent storage. Operating system 412 may be Windows, among others. The data 413 may include, but is not limited to, data involved in the above-described methods.
In some embodiments, the sub-library and sub-table device may further comprise a display 430, a power supply 440, a communication interface 450, an input/output interface 460, a sensor 470, and a communication bus 480.
Those skilled in the art will appreciate that the configuration shown in FIG. 4 does not constitute a limitation of a banking and spreadsheet facility and may include more or fewer components than those shown.
The database and table dividing device provided by the embodiment of the application comprises a memory and a processor, and the processor can realize the database and table dividing method when executing the program stored in the memory, and the effect is the same as that of the database and table dividing method.
It should be noted that the above-described embodiments of the apparatus and device are merely illustrative, for example, the division of modules is only one division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form. Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and performs all or part of the steps of the methods according to the embodiments of the present invention, or all or part of the technical solution.
To this end, an embodiment of the present invention further provides a storage medium, where the storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the method such as the database-based spreadsheet-based method.
The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory ROM (Read-Only Memory), a random Access Memory ram (random Access Memory), a magnetic disk, or an optical disk.
The computer program contained in the storage medium provided in this embodiment can implement the steps of the library/table splitting method described above when being executed by the processor, and the effect is the same as above.
The present invention provides a method, apparatus, device and storage medium for database and table allocation. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device, the equipment and the storage medium disclosed by the embodiment correspond to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A database and table dividing method is characterized by comprising the following steps:
when data to be stored is received, acquiring a data identifier of the data to be stored;
if the existing database of the target storage system has the associated data with the same data identification as the data to be stored, storing the data to be stored into the existing data table where the associated data is located;
and if the associated data does not exist in the existing database, storing the data to be stored into an existing data table according to a preset average distribution rule.
2. The database and table dividing method according to claim 1, wherein the obtaining of the data identifier of the data to be stored specifically comprises:
and identifying the preset field of the data to be stored to obtain the data identification of the data to be stored.
3. The database and table dividing method according to claim 1, wherein the storing the data to be stored into the existing data table according to the preset average distribution rule specifically comprises:
determining the data type of the data to be stored according to the data identifier of the data to be stored, and determining the weight of the data to be stored according to the data type of the data to be stored according to a preset weight reference;
acquiring the weight sum of the data in the existing data table, and storing the data to be stored into the existing data table with the minimum weight sum;
the weight reference is established according to the principle that the more the quantity of the data type is, the more the weight of the data type is.
4. The database and table dividing method according to claim 3, wherein storing the data to be stored into the existing data table with the minimum sum of the weights specifically comprises:
if the existing data table with the minimum weight sum exists and only one existing data table with the minimum weight sum exists, storing the data to be stored into the existing data table with the minimum weight sum;
and if two or more existing data tables with the minimum weight sum exist, storing the data to be stored into the existing data table with the minimum weight sum, wherein the existing data table has the minimum data.
5. The database and table dividing method according to claim 3, further comprising:
checking the quantity of data in each existing data table at regular time;
for any two data types, if the difference value between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is smaller than a first preset value, reducing the weight difference value between the data type with the larger weight and the data type with the smaller weight; if the difference value between the quantity of the data type with the larger weight and the quantity of the data type with the smaller weight is larger than a second preset value, increasing the weight difference value between the data type with the larger weight and the data type with the smaller weight;
wherein the first preset value is smaller than the second preset value.
6. The database and table dividing method according to claim 3, further comprising:
and storing the sum of the weights of the existing data tables.
7. The database and table partitioning method according to claim 1, further comprising:
if the existing data tables cannot be stored in the data to be stored, a new data table is created, and the table creation time is recorded as the starting time of the new data table for receiving the data to be stored;
and if the existing databases cannot create the new data table, creating a new database, and recording the database creation time as the starting time for receiving the data to be stored by the new database.
8. A device for sorting and tabulating a library, comprising:
the device comprises a receiving unit, a storage unit and a processing unit, wherein the receiving unit is used for acquiring a data identifier of data to be stored when the data to be stored is received;
the distribution unit is used for storing the data to be stored into an existing data table where the associated data are located if the associated data with the same data identifier as the data to be stored exist; and if the associated data does not exist, storing the data to be stored into the existing data table according to a preset average distribution rule.
9. A banking and spreadsheet distribution apparatus, comprising:
a memory for storing instructions, said instructions comprising the steps of the method of any of claims 1 to 7;
a processor to execute the instructions.
10. A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the banking and spreadsheet method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011363595.9A CN112395293B (en) | 2020-11-27 | 2020-11-27 | Database and table dividing method, database and table dividing device, database and table dividing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011363595.9A CN112395293B (en) | 2020-11-27 | 2020-11-27 | Database and table dividing method, database and table dividing device, database and table dividing equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112395293A true CN112395293A (en) | 2021-02-23 |
| CN112395293B CN112395293B (en) | 2024-03-01 |
Family
ID=74605325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011363595.9A Active CN112395293B (en) | 2020-11-27 | 2020-11-27 | Database and table dividing method, database and table dividing device, database and table dividing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112395293B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220691A (en) * | 2021-05-31 | 2021-08-06 | 北京互金新融科技有限公司 | Database table creating method and device for database table, and electronic equipment |
| CN113449042A (en) * | 2021-07-14 | 2021-09-28 | 中国银行股份有限公司 | Automatic data warehouse-dividing method and device |
| CN113590609A (en) * | 2021-06-22 | 2021-11-02 | 北京旷视科技有限公司 | Database partitioning method and device, storage medium and electronic equipment |
| CN113641670A (en) * | 2021-07-09 | 2021-11-12 | 北京百度网讯科技有限公司 | Data storage and data retrieval method and device, electronic equipment and storage medium |
| CN115509463A (en) * | 2022-11-15 | 2022-12-23 | 北京云成金融信息服务有限公司 | Balanced data storage method and system based on data middling station |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011081580A1 (en) * | 2009-12-29 | 2011-07-07 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and arrangement for data storage |
| US20160012235A1 (en) * | 2014-02-10 | 2016-01-14 | Vivo Security Inc. | Analysis and display of cybersecurity risks for enterprise data |
| CN106528579A (en) * | 2015-09-15 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Search method, device and system based on sharding structure databases |
| WO2017092470A1 (en) * | 2015-12-01 | 2017-06-08 | 中兴通讯股份有限公司 | Data storage method and device |
| CN106897409A (en) * | 2017-02-16 | 2017-06-27 | 北京致远互联软件股份有限公司 | Data point library storage method and device |
| US20170322974A1 (en) * | 2016-05-06 | 2017-11-09 | InsightSoftware.com International | Sql double counting resolver |
| CN108241934A (en) * | 2016-12-23 | 2018-07-03 | 北京京东尚科信息技术有限公司 | Data query method and apparatus |
| CN109815228A (en) * | 2018-12-14 | 2019-05-28 | 深圳壹账通智能科技有限公司 | Creation method, device, computer equipment and the readable storage medium storing program for executing of database table |
| CN109918442A (en) * | 2019-01-24 | 2019-06-21 | 中国联合网络通信集团有限公司 | A kind of processing method and processing device of data |
| CN109992488A (en) * | 2019-01-18 | 2019-07-09 | 珠海金山网络游戏科技有限公司 | A kind of statistical method based on MongoDB database |
| CN110287346A (en) * | 2019-06-28 | 2019-09-27 | 深圳云天励飞技术有限公司 | Date storage method, device, server and storage medium |
| CN111611249A (en) * | 2020-06-29 | 2020-09-01 | 平安银行股份有限公司 | Data management method, device, equipment and storage medium |
-
2020
- 2020-11-27 CN CN202011363595.9A patent/CN112395293B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011081580A1 (en) * | 2009-12-29 | 2011-07-07 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and arrangement for data storage |
| US20160012235A1 (en) * | 2014-02-10 | 2016-01-14 | Vivo Security Inc. | Analysis and display of cybersecurity risks for enterprise data |
| CN106528579A (en) * | 2015-09-15 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Search method, device and system based on sharding structure databases |
| WO2017092470A1 (en) * | 2015-12-01 | 2017-06-08 | 中兴通讯股份有限公司 | Data storage method and device |
| US20170322974A1 (en) * | 2016-05-06 | 2017-11-09 | InsightSoftware.com International | Sql double counting resolver |
| CN108241934A (en) * | 2016-12-23 | 2018-07-03 | 北京京东尚科信息技术有限公司 | Data query method and apparatus |
| CN106897409A (en) * | 2017-02-16 | 2017-06-27 | 北京致远互联软件股份有限公司 | Data point library storage method and device |
| CN109815228A (en) * | 2018-12-14 | 2019-05-28 | 深圳壹账通智能科技有限公司 | Creation method, device, computer equipment and the readable storage medium storing program for executing of database table |
| CN109992488A (en) * | 2019-01-18 | 2019-07-09 | 珠海金山网络游戏科技有限公司 | A kind of statistical method based on MongoDB database |
| CN109918442A (en) * | 2019-01-24 | 2019-06-21 | 中国联合网络通信集团有限公司 | A kind of processing method and processing device of data |
| CN110287346A (en) * | 2019-06-28 | 2019-09-27 | 深圳云天励飞技术有限公司 | Date storage method, device, server and storage medium |
| CN111611249A (en) * | 2020-06-29 | 2020-09-01 | 平安银行股份有限公司 | Data management method, device, equipment and storage medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220691A (en) * | 2021-05-31 | 2021-08-06 | 北京互金新融科技有限公司 | Database table creating method and device for database table, and electronic equipment |
| CN113220691B (en) * | 2021-05-31 | 2024-02-06 | 北京互金新融科技有限公司 | Database table creation method, database table creation device and electronic equipment |
| CN113590609A (en) * | 2021-06-22 | 2021-11-02 | 北京旷视科技有限公司 | Database partitioning method and device, storage medium and electronic equipment |
| CN113641670A (en) * | 2021-07-09 | 2021-11-12 | 北京百度网讯科技有限公司 | Data storage and data retrieval method and device, electronic equipment and storage medium |
| CN113641670B (en) * | 2021-07-09 | 2023-08-11 | 北京百度网讯科技有限公司 | Data storage and data retrieval method and device, electronic equipment and storage medium |
| CN113449042A (en) * | 2021-07-14 | 2021-09-28 | 中国银行股份有限公司 | Automatic data warehouse-dividing method and device |
| CN115509463A (en) * | 2022-11-15 | 2022-12-23 | 北京云成金融信息服务有限公司 | Balanced data storage method and system based on data middling station |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112395293B (en) | 2024-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112395293A (en) | Warehouse and table dividing method, warehouse and table dividing device, warehouse and table dividing equipment and storage medium | |
| EP3637280B1 (en) | Data storage method and device, and storage medium | |
| US7284022B2 (en) | Database management methods and equipment and database management program storage media | |
| EP2924594A1 (en) | Data encoding and corresponding data structure in a column-store database | |
| JP6242540B1 (en) | Data conversion system and data conversion method | |
| US11797487B2 (en) | Maintaining stable record identifiers in the presence of updated data records | |
| CN112925859A (en) | Data storage method and device | |
| CN111625561B (en) | Data query method and device | |
| CN104182295A (en) | Data backup method and data backup device | |
| CN112348420A (en) | Storage position information acquisition method and system, storage medium and electronic equipment | |
| CN111414422B (en) | Data distribution method, device, equipment and storage medium | |
| CN108197164A (en) | Business data storage method and device | |
| CN110688223B (en) | Data processing method and related product | |
| CN110929207B (en) | Data processing method, device and computer readable storage medium | |
| CN112052330B (en) | Application keyword distribution method and device | |
| CN113535727A (en) | Data output method and data output device of information system and electronic equipment | |
| US7430557B1 (en) | System and method for improving database reorganization time | |
| CN111709210A (en) | Method and device for generating component list, terminal equipment and storage medium | |
| CN112348298A (en) | Designer management method, designer management device, electronic terminal and storage medium | |
| CN112402955B (en) | Game log recording method and system | |
| Halbig et al. | Exploiting user-supplied Decompositions inside Heuristics | |
| CN115391052B (en) | Robot task processing method and device, electronic equipment and storage medium | |
| CN111158605B (en) | Method and device for optimizing disk storage policy of operating system and intelligent equipment | |
| CN115016735A (en) | Control method, device and medium for distributed cache system | |
| CN115878627A (en) | Database partitioning method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |