CN111061721B - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN111061721B CN111061721B CN201811200505.7A CN201811200505A CN111061721B CN 111061721 B CN111061721 B CN 111061721B CN 201811200505 A CN201811200505 A CN 201811200505A CN 111061721 B CN111061721 B CN 111061721B
- Authority
- CN
- China
- Prior art keywords
- attribute information
- database
- small table
- target data
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention provides a data processing method and a device, wherein the method comprises the following steps: acquiring attribute information of first target data to be stored in a database; determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises a large table and at least one small table, wherein the attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table; and storing the first target data into a small table corresponding to the attribute information of the first target data. The data processing method and the data processing device improve the data processing efficiency in the database.
Description
Technical Field
The present invention relates to the field of database technologies, and in particular, to a data processing method and apparatus.
Background
A log file is a log type of data in a database that is used to record database update operations. The log file is used for tracking and recording all modifications to database data, and the database can be protected through the log file of the database so as to execute data recovery or rollback after the database fails or is used for assisting a user in analyzing and solving the problems in the database.
In the prior art, a database generates new log files whenever there is an update of data in daily use, resulting in an increasing number of log files stored in a table in the database for storing log files during use of the database. The more and more log files are not beneficial to searching and processing the data such as the database log files, so that the processing efficiency of the database data is lower. Therefore, how to improve the efficiency of processing data in a database is a technical problem to be solved.
Disclosure of Invention
The invention provides a data processing method and a data processing device, which are used for improving the processing efficiency of data when a database processes the data.
The first aspect of the present invention provides a data processing method, including:
acquiring attribute information of first target data to be stored in a database;
determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises the large table and at least one small table, wherein attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table;
and storing the first target data into a small table corresponding to the attribute information of the first target data.
In an embodiment of the first aspect of the present invention, after storing the first target data in the small table corresponding to the attribute information, the method further includes:
acquiring attribute information of second target data to be processed in the database;
determining a small table corresponding to the attribute information of the second target data according to the large table of the database;
and acquiring the second target data from a small table corresponding to the attribute information of the second target data.
In an embodiment of the first aspect of the present invention, after the obtaining the second target data from the small table corresponding to the attribute information of the second target data, the method further includes:
and performing operations of adding, deleting, modifying and/or querying on the second target data.
In a first embodiment of the first aspect of the present invention, the method further includes:
acquiring attribute information of all data stored in the database;
determining the number of the at least one small table in the database according to the classification result of the attribute information of all the data;
and establishing the at least one small table in the database, and storing the corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table in the large table.
In an embodiment of the first aspect of the present invention, the determining, according to the large table of the database, a small table corresponding to attribute information of the first target data includes:
determining whether a small table corresponding to the attribute information of the first target data exists in the database;
if the first target data does not exist, a small table corresponding to the attribute information of the first target data is built in the database.
In a first embodiment of the first aspect of the present invention, the database is used for storing log data; the attribute information is the generation time of the log data.
A second aspect of the present invention provides a data processing apparatus comprising:
the acquisition module is used for acquiring attribute information of first target data to be stored in the database;
the determining module is used for determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises the large table and at least one small table, wherein attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table;
and the processing module is used for storing the first target data into a small table corresponding to the attribute information of the first target data.
In an embodiment of the second aspect of the present invention, the obtaining module is further configured to obtain attribute information of second target data to be processed in the database;
the determining module is further used for determining a small table corresponding to the attribute information of the second target data according to the large table of the database;
the processing module is further configured to obtain the second target data from a small table corresponding to the attribute information of the second target data.
In an embodiment of the second aspect of the present invention, the processing module is further configured to add, delete, modify, and/or query the second target data.
In an embodiment of the second aspect of the present invention, the obtaining module is further configured to obtain attribute information of all data stored in the database;
the determining module is further used for determining the number of the at least one small table in the database according to the classification result of the attribute information of all the data;
the determining module is further configured to establish the at least one small table in the database, and store a correspondence between the at least one small table and attribute information of data stored in the at least one small table in the large table.
In an embodiment of the second aspect of the present invention, the determining module is specifically configured to determine whether a small table corresponding to attribute information of the first target data exists in the database;
if not, the determining module is further configured to establish a small table corresponding to the attribute information of the first target data in the database.
In a third aspect, embodiments of the present invention provide a computer readable storage medium storing program code which, when executed, performs a data processing method according to any one of the first aspects of the present invention.
In a fourth aspect, an embodiment of the present invention provides a passenger information system PIS data transmission device, including: a processor and a memory; the memory is used for storing programs; the processor is configured to call the program stored in the memory to execute the data processing method according to any one of the first aspect of the present invention.
In summary, the present invention provides a data processing method and apparatus, where the method includes: acquiring attribute information of first target data to be stored in a database; determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises a large table and at least one small table, wherein the attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table; and storing the first target data into a small table corresponding to the attribute information of the first target data. According to the data processing method and device, the data of different attribute information are stored through the small table in the database, and the corresponding relation between the small table and the attribute information of the stored data is included in the large table. Therefore, the data are stored in different small tables, and the inquiry and subsequent processing of the data stored in the small tables can be realized through the large tables, so that the data processing efficiency in the database is improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is a flow chart of an embodiment of a data processing method of the present invention;
FIG. 2 is a flow chart of an embodiment of a data processing method according to the present invention;
FIG. 3 is a schematic diagram of a data processing apparatus according to an embodiment of the present invention.
Specific embodiments of the present disclosure have been shown by way of the above drawings and will be described in more detail below. These drawings and the written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the disclosed concepts to those skilled in the art by reference to specific embodiments. The technical scheme of the invention is described in detail below by specific examples. The following embodiments may be combined with each other and may not be described in detail in some embodiments for the same or similar concepts or processes.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented, for example, in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The technical scheme of the invention is described in detail below by specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
FIG. 1 is a flow chart of a data processing method according to an embodiment of the invention. As shown in fig. 1, the data processing method provided in this embodiment includes:
s101: and acquiring attribute information of the first target data to be stored in the database.
The execution body of the embodiment is an electronic device, for example, a computer, a server, a mobile phone, or the like, or a chip in the electronic device, which has a function of processing data in a database, so as to implement the processing of the data in the database in the embodiment.
Specifically, in S101, when the electronic device acquires first target data that needs to be stored in the database, attribute information of the first target data to be stored in the database is first acquired.
The electronic device may receive the first target data by receiving a user instruction before obtaining the attribute information, for example, the user sends the first target data and an instruction for storing the first target data in a designated position of the database; or, the data automatically generated in the database is used as the first target data and stored in a default storage position of the data.
For example: the first target data is a log file of the database, and log data is generated after each time of data addition, modification, deletion and other operations are performed in the database. As for the log data generated by each operation, the first target data in this embodiment may be used, and when the log data is used as the first target data, the attribute information of the first target data, i.e., the log data, needs to be acquired before the log data is stored in the database.
Alternatively, the attribute information of the first target data may include one or more of: generation time, modification time, data size, operating user, designated storage location, etc. For example, in the above example, when the first target data is a log file of the database, the attribute information of the log file may be a generation time of the log file.
S102: determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises a large table and at least one small table, wherein the attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table.
S103: and storing the first target data into a small table corresponding to the attribute information of the first target data.
Specifically, in S102, a small table corresponding to the attribute information of the first target data is determined from the large table of the database according to the attribute information of the first target data determined in S101, and the first target data is stored in the small table determined in S102 in S103. Wherein, at least the large table and at least one small table exist in the database, and each small table corresponds to the attribute information of different data. The large table is used as an index of the small table, and stores the corresponding relation between all the small tables and the attribute information of the data, and the corresponding small table can be determined through the corresponding relation in the large table by the attribute information of the data.
It should be noted that, the large table and the small table are concepts of abstract tables in the database, substantially correspond to different storage spaces in the database, the large table and the small table are part of the storage spaces of the database, and the sum of the storage spaces of all the large table and the small table is smaller than or equal to the size of the total storage space of the database.
Also taking the first target data as the log file in the foregoing example, the attribute information generation time is taken as an example, different small tables can be allocated in the database according to the generation time of different log files, for example, taking the time of 12 months as an example, 12 small tables are set in the database, and each small table is used for storing the log file generated in one month. Then in S102, all log data generated by the database are determined according to the generated time, and the corresponding month table in the 12 tables is stored in the corresponding table.
Alternatively, all the small tables and attribute information of data stored in the small tables may be stored in the large table, for example, the correspondence stored in the large table includes 12 small tables corresponding to "1 month-small table 1", "2 months-small table 2", … … for 12 months, where the correspondence may be names of the small tables, for example, small table 1, or the correspondence may also be a specific storage location of the small table, so that data generated by the corresponding month is stored in the corresponding small table according to the small table storage location.
For example: when the first target data acquired in S101 is a log file, and the attribute information of the first target data is that the generation time of the log file is 1 month and 1 day, the small table corresponding to 1 month and 1 day determined from the large table of the database in S102 is "small table 1", and the log file is stored in the small table 1 of the database in S103.
Alternatively, in the above embodiment, the tables generated in the database are identical, that is, refer to structures having identical definitions and tables, and differ only in attribute information of the stored data. Alternatively, the small tables in the database may also set different definitions and structures of the tables according to the stored attribute information.
Optionally, in the above embodiment, when the attribute information is a modification time, a data size, an operation user, a designated storage location, and the like of the first target data, it may also be respectively divided into different sub-tables of 12 months according to months of different modification times, into 2 sub-tables of big data and small data according to data sizes, into the same number of sub-tables as the number of users of the database according to operation of each user, or the attribute information carried by the first target data may designate a fixed area where the data exists, and then the sub-table corresponding to the fixed area is determined.
Optionally, the database described in this embodiment is a MySQL database, where the flexible feature of a MySQL database pluggable engine is used to implement the above embodiment based on the MySQL engine and the program for executing the above embodiment in combination.
In summary, the present embodiment provides a data processing method, by acquiring attribute information of first target data to be stored in a database; determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises a large table and at least one small table, wherein the attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table; and storing the first target data into a small table corresponding to the attribute information of the first target data. Therefore, the data processing method provided in this embodiment can store data of different attribute information through the small table in the database, and include the correspondence between the small table and the attribute information of the stored data in the large table. Therefore, the data are stored in different small tables, and the inquiry and subsequent processing of the data stored in the small tables can be realized through the large tables, so that the data processing efficiency in the database is improved.
FIG. 2 is a flow chart of an embodiment of a data processing method according to the present invention. The embodiment shown in fig. 2 is a step after S103 on the basis of the embodiment shown in fig. 1. It will be appreciated that fig. 1 is an operation of the database to store data, whereas in the embodiment shown in fig. 2, a flow of a lookup of data in the database is provided, the definition of large and small tables of the database being the same as in fig. 1, and may be the same as or different from the aforementioned first target data for the second target data being looked up. Specifically, the data processing method shown in fig. 2 includes:
s201: and acquiring attribute information of the second target data to be processed in the database.
Specifically, the method for acquiring the attribute information of the second target data in S201 of the embodiment is the same as the method for acquiring the attribute information of the first target data in S101, and will not be described again.
S202: and determining a small table corresponding to the attribute information of the second target data according to the large table of the database.
Specifically, the method for determining the attribute information of the second target data from the large table of the database in S202 in this embodiment is the same as the method for determining the attribute information of the first target data from the large table of the database in S102, and will not be described again.
S203: and acquiring the second target data from the small table corresponding to the attribute information of the second target data.
S204: and performing operations of adding, deleting, modifying and/or inquiring the second target data.
Specifically, in S203 and S204 of the present embodiment, according to the small table determined in S202, the second target data in the small table may be retrieved from the small table and finally determined, and the second target data may be extracted to perform operations of adding, deleting, modifying, and/or querying the second target data.
Therefore, with the embodiment as shown in fig. 2, it is possible to store data of different attribute information by the small table in the database, and include the correspondence relationship between the small table and the attribute information of the stored data thereof in the large table. Thus, data are stored in different small tables, and the inquiry and subsequent processing of the data stored in the small tables can be realized through the large tables. Compared with the prior art that a database stores all log files in one large table, for example, when the large table is needed to be searched, the small table where the data is located can be determined directly through the index of the large table, and the small table with smaller data volume is searched, so that the efficiency of data processing, particularly data query, in the database can be greatly improved.
Optionally, in the embodiment shown in fig. 1 and 2, the method of establishing the large table and the small table in the database in advance may be further included before S101 as follows. The method specifically comprises the following steps: acquiring attribute information of all data stored in a database; determining the number of at least one small table in the database according to the classification result of the attribute information of all the data; at least one small table is established in the database, and a correspondence relationship between the at least one small table and attribute information of data stored in the at least one small table is stored in the large table.
Specifically, in this embodiment, the small table to be built is determined according to attribute information of all data stored in the database. All data herein may be all data currently stored in the database, or may be all data that may be stored in the database. For example, log information of all the years may be stored in the database, and when the attribute information is the generation time of the log file, it may be determined that the attribute information of all the data in the database is 12 months, and thus it is determined that 12 sub-tables are established in the database, each for log files generated at different months. And establishes correspondence in the large table, for example, "1 month-small table 1", "2 month-small table 2", … ….
Alternatively, in the embodiment shown in fig. 1 and fig. 2, the step of determining the small table corresponding to the first target attribute information in S102 may specifically include: determining whether a small table corresponding to the attribute information of the first target data exists in the database; if the first target data does not exist, a small table corresponding to the attribute information of the first target data is built in the database.
Specifically, in the present embodiment, if the first target data is stored in the embodiment shown in fig. 1 and 2, there is no small table corresponding to the attribute information. A small table may be newly created when the first target data needs to be stored in the database. For example: when all log information of a certain year is possibly stored in the database and the attribute information is the generation time of the log file, if only a large table and a small table 1 exist in the database, the corresponding relation of '1 month-small table 1' is recorded in the large table. If the attribute information of the first target data acquired at this time is: the generation time is 2 months and 1 day, but the database does not have a small table for storing the data, the small table 2 can be newly built in the database for storing the log file with the generation time of 2 months, and the corresponding relation of '2 months-small table 2' is added in the large table.
More specifically, when the database is MySQL, in this embodiment, the mapping relationship of the small table may be stored as a large table by using a database file with suffix ". MRG", and the small table may be a database file with suffix ". MYD". And the data in the ". MYD" of the small table is searched, the small table is firstly required to be determined through the mapping relation stored in the ". MRG". Therefore, the development of complex path functions of front-end service codes in the database is reduced, and the improvement of the query speed under the condition of large data volume is realized. Under the condition of hundreds of GB data volume, the query speed can be controlled within millisecond level.
FIG. 3 is a schematic diagram of a data processing apparatus according to an embodiment of the present invention. As shown in fig. 3, the data processing apparatus provided in this embodiment includes: an acquisition module 301, a determination module 302 and a processing module 303.
The acquiring module 301 is configured to acquire attribute information of first target data to be stored in the database; the determining module 302 is configured to determine a small table corresponding to attribute information of the first target data according to the large table of the database; the database comprises a large table and at least one small table, wherein the attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table; the processing module 303 is configured to store the first target data in a small table corresponding to attribute information of the first target data.
The data processing apparatus provided in this embodiment may be used to execute the data processing method shown in fig. 1, and its implementation manner and principle are the same, and will not be described herein again.
Optionally, in the above embodiment, the obtaining module 301 is further configured to obtain attribute information of second target data to be processed in the database; the determining module 302 is further configured to determine a small table corresponding to the attribute information of the second target data according to the large table of the database; the processing module 303 is further configured to obtain second target data from a small table corresponding to attribute information of the second target data; the processing module 303 is further configured to perform operations of adding, deleting, modifying and/or querying the second target data.
The data processing apparatus provided in this embodiment may be used to execute the data processing method shown in fig. 2, and its implementation manner is the same as that of the principle, and will not be described herein again.
Optionally, in the above embodiment, the obtaining module 301 is further configured to obtain attribute information of all data stored in the database; the determining module 302 is further configured to determine the number of at least one small table in the database according to the classification result of the attribute information of all the data; the determining module 302 is further configured to establish at least one small table in the database, and store a correspondence between the at least one small table and attribute information of data stored in the at least one small table in the large table.
Optionally, in the foregoing embodiment, the determining module 302 is specifically configured to determine whether a small table corresponding to the attribute information of the first target data exists in the database; if not, the determining module is further configured to establish a small table corresponding to the attribute information of the first target data in the database.
Optionally, in the above embodiment, the database is used to store log data; the attribute information is the generation time of the log data.
The data processing device provided in each of the foregoing embodiments may be used to execute the data processing method shown in the foregoing corresponding embodiment, and its implementation manner and principle are the same, which is not described herein again.
It should be noted that, in the embodiments of the present application, the division of the modules is merely a logic function division, and other division manners may be implemented in actual practice. The functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution, in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
The present application also provides a computer-readable storage medium in which a program code is stored, which when executed, performs a data processing method as in any of the above embodiments.
The present application also provides a computer program product comprising program code for implementing a data processing method according to any of the above embodiments when the program code is executed by a processor.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.
The above description is only of the preferred embodiments of the present invention, and is not intended to limit the present invention in any way, but any simple modification, equivalent variation and modification made to the above embodiments according to the technical substance of the present invention still fall within the scope of the technical solution of the present invention.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (8)
1. A method of data processing, comprising:
acquiring attribute information of first target data to be stored in a database;
determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises the large table and at least one small table, wherein attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table;
storing the first target data into a small table corresponding to attribute information of the first target data;
further comprises:
acquiring attribute information of all data stored in the database;
determining the number of the at least one small table in the database according to the classification result of the attribute information of all the data;
and establishing the at least one small table in the database, and storing the corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table in the large table.
2. The method according to claim 1, wherein after storing the first target data in the small table corresponding to the attribute information, further comprises:
acquiring attribute information of second target data to be processed in the database;
determining a small table corresponding to the attribute information of the second target data according to the large table of the database;
and acquiring the second target data from a small table corresponding to the attribute information of the second target data.
3. The method according to claim 2, wherein after the second target data is obtained from the small table corresponding to the attribute information of the second target data, further comprising:
and performing operations of adding, deleting, modifying and/or querying on the second target data.
4. A method according to any one of claims 1-3, wherein determining a small table corresponding to the attribute information of the first target data according to the large table of the database comprises:
determining whether a small table corresponding to the attribute information of the first target data exists in the database;
if the first target data does not exist, a small table corresponding to the attribute information of the first target data is built in the database.
5. A method according to any one of claims 1-3, wherein the database is used to store log data; the attribute information is the generation time of the log data.
6. A data processing apparatus, characterized by comprising:
the acquisition module is used for acquiring attribute information of first target data to be stored in the database;
the determining module is used for determining a small table corresponding to the attribute information of the first target data according to the large table of the database; the database comprises the large table and at least one small table, wherein attribute information of data stored in each small table is different, and the large table comprises a corresponding relation between the at least one small table and the attribute information of the data stored in the at least one small table;
the processing module is used for storing the first target data into a small table corresponding to the attribute information of the first target data;
the acquisition module is also used for acquiring attribute information of all data stored in the database;
the determining module is further used for determining the number of the at least one small table in the database according to the classification result of the attribute information of all the data;
the determining module is further configured to establish the at least one small table in the database, and store a correspondence between the at least one small table and attribute information of data stored in the at least one small table in the large table.
7. The apparatus of claim 6, wherein the device comprises a plurality of sensors,
the acquisition module is also used for acquiring attribute information of second target data to be processed in the database;
the determining module is further used for determining a small table corresponding to the attribute information of the second target data according to the large table of the database;
the processing module is further used for acquiring the second target data from a small table corresponding to the attribute information of the second target data;
the processing module is further configured to perform operations of adding, deleting, modifying and/or querying on the second target data.
8. The apparatus according to claim 6 or 7, wherein,
the determining module is specifically configured to determine whether a small table corresponding to the attribute information of the first target data exists in the database;
if not, the determining module is further configured to establish a small table corresponding to the attribute information of the first target data in the database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811200505.7A CN111061721B (en) | 2018-10-16 | 2018-10-16 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811200505.7A CN111061721B (en) | 2018-10-16 | 2018-10-16 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111061721A CN111061721A (en) | 2020-04-24 |
| CN111061721B true CN111061721B (en) | 2023-07-25 |
Family
ID=70296281
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811200505.7A Active CN111061721B (en) | 2018-10-16 | 2018-10-16 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111061721B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021395A (en) * | 2016-05-12 | 2016-10-12 | 成都鼎智汇科技有限公司 | Drug information storage method in internet-of-things environment |
| CN106407172A (en) * | 2016-11-21 | 2017-02-15 | 成都鼎桥通信技术有限公司 | XML (Extensible Markup Language)-based report generating method |
| KR20170081557A (en) * | 2016-01-04 | 2017-07-12 | 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. | Method and device for updating object data in object storage system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6542893B1 (en) * | 2000-02-29 | 2003-04-01 | Unisys Corporation | Database sizer for preemptive multitasking operating system |
| US7689553B2 (en) * | 2006-03-30 | 2010-03-30 | International Business Machines Corporation | Execution cost reduction of sampled queries in a database |
| CN105989140B (en) * | 2015-02-27 | 2019-09-03 | 阿里巴巴集团控股有限公司 | A kind of data block processing method and equipment |
| CN105260399A (en) * | 2015-09-17 | 2016-01-20 | 西安未来国际信息股份有限公司 | Method for acquiring and retrieving distributed log |
| US9898614B1 (en) * | 2015-09-18 | 2018-02-20 | Amazon Technologies, Inc. | Implicit prioritization to rate-limit secondary index creation for an online table |
| JP2018005509A (en) * | 2016-06-30 | 2018-01-11 | キヤノン株式会社 | Information processing device, control method, and program |
| CN107239382A (en) * | 2017-06-23 | 2017-10-10 | 深圳市冬泉谷信息技术有限公司 | The log processing method and system of a kind of container application |
| CN107451034A (en) * | 2017-08-17 | 2017-12-08 | 浪潮软件股份有限公司 | A kind of big data cluster log management apparatus, method and system |
-
2018
- 2018-10-16 CN CN201811200505.7A patent/CN111061721B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20170081557A (en) * | 2016-01-04 | 2017-07-12 | 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. | Method and device for updating object data in object storage system |
| CN106021395A (en) * | 2016-05-12 | 2016-10-12 | 成都鼎智汇科技有限公司 | Drug information storage method in internet-of-things environment |
| CN106407172A (en) * | 2016-11-21 | 2017-02-15 | 成都鼎桥通信技术有限公司 | XML (Extensible Markup Language)-based report generating method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111061721A (en) | 2020-04-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20150363447A1 (en) | Minimizing index maintenance costs for database storage regions using hybrid zone maps and indices | |
| CN105760199B (en) | A kind of application resource loading method and its equipment | |
| CN107015985B (en) | Data storage and acquisition method and device | |
| CN109582470B (en) | Data processing method and data processing device | |
| CN107103011B (en) | Method and device for realizing terminal data search | |
| CN111259067B (en) | Method, device and equipment for realizing DAO interface based on Spring | |
| CN104113576A (en) | Method and device for updating client | |
| CN111339171B (en) | Data query method, device and equipment | |
| CN105468644B (en) | Method and equipment for querying in database | |
| CN104423982A (en) | Request processing method and device | |
| CN109388644B (en) | Data updating method and device | |
| CN103902565A (en) | Method and device for determining data access relation | |
| CN114328601A (en) | Data down-sampling and data query method, system and storage medium | |
| CN111061721B (en) | Data processing method and device | |
| CN112306957A (en) | Method and device for acquiring index node number, computing equipment and storage medium | |
| CN110019295B (en) | Database retrieval method, device, system and storage medium | |
| US11645283B2 (en) | Predictive query processing | |
| CN110968615A (en) | Data query method and device | |
| CN114528313A (en) | Data processing method and device and electronic equipment | |
| CN112699129A (en) | Data processing system, method and device | |
| CN108897873B (en) | Method and device for generating job file, storage medium and processor | |
| CN112749189A (en) | Data query method and device | |
| CN111563123A (en) | Live warehouse metadata real-time synchronization method | |
| CN117093597B (en) | Data processing method and device | |
| CN111368146A (en) | Path information query method and device, storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |