CN107798030B - Splitting method and device of data table - Google Patents
Splitting method and device of data table Download PDFInfo
- Publication number
- CN107798030B CN107798030B CN201710087369.4A CN201710087369A CN107798030B CN 107798030 B CN107798030 B CN 107798030B CN 201710087369 A CN201710087369 A CN 201710087369A CN 107798030 B CN107798030 B CN 107798030B
- Authority
- CN
- China
- Prior art keywords
- data table
- database
- sub
- data
- name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method and a device for splitting a data table, wherein the method comprises the following steps: acquiring a work task, wherein the work task carries a data table name and a field; displaying a configuration page, and receiving the data table name and the field input by a user in the configuration page; generating a script file for splitting the data table according to the data table name and the field; acquiring a corresponding data table according to the data table name, and splitting the data table by using the script file to obtain a plurality of sub-tables; and loading the split sub-tables and executing the work task by utilizing the sub-tables. By adopting the method, the time consumption for loading the data table can be effectively reduced, and the working efficiency is improved.
Description
Technical Field
The invention relates to the technical field of databases, in particular to a method and a device for splitting a data table.
Background
Currently, databases generally use data tables to store data. As the database usage time accumulates, the data in the data table will increase continuously. When the amount of data stored in the data table is too large, the time consumed for loading the data table increases, and the loading efficiency decreases. How to effectively improve the loading efficiency of the data table is a technical problem to be solved at present.
Disclosure of Invention
Therefore, it is necessary to provide a method and an apparatus for splitting a data table, which can effectively reduce the time consumed by loading the data table and improve the work efficiency, for solving the above technical problems.
A method of splitting a data table, the method comprising:
acquiring a work task, wherein the work task carries a data table name and a field;
displaying a configuration page, and receiving the data table name and the field input by a user in the configuration page;
generating a script file for splitting the data table according to the data table name and the field;
acquiring a corresponding data table according to the data table name, and splitting the data table by using the script file to obtain a plurality of sub-tables;
and loading the split sub-tables and executing the work task by utilizing the sub-tables.
In one embodiment, the data table includes delimiters, and the step of splitting the data table using the script file includes:
acquiring the corresponding relation between the separator and the field;
extracting data corresponding to the fields in the data table according to the corresponding relation;
and splitting the extracted data by utilizing the script file.
In one embodiment, the work task also carries a database identifier and a database user identifier; before the step of obtaining the corresponding data table according to the data table name, the method further comprises the following steps:
selecting a database corresponding to the database identifier;
inquiring whether a required data table name exists in a data table corresponding to the database user identification in the selected database;
if not, extracting the required data in the data table corresponding to the database user identification by using the field;
data tables required for the work task are created using the required data.
In one embodiment, before the step of splitting the data table by using the script file, the method further includes:
acquiring the number of characters of the name of the data table;
checking the data table name and the number of characters of the data table name respectively;
and if the name of the data table fails to be checked and/or the number of the characters of the name of the data table fails to be checked, generating error prompt information.
In one embodiment, the loading the split sub-tables, and the step of executing the work task by using the sub-tables includes:
acquiring the idle time of a database;
and loading the split sub-tables at the idle time of the database, and executing the work task by using the sub-tables.
An apparatus for splitting a data table, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a work task, and the work task carries a data table name and a field;
the display module is used for displaying the configuration page;
the receiving module is used for receiving the data table name and the field input by the user in the configuration page;
the splitting module is used for generating a script file for splitting the data table according to the name of the data table and the field; acquiring a corresponding data table according to the data table name, and splitting the data table by using the script file to obtain a plurality of sub-tables;
and the task execution module is used for loading the split sub-tables and executing the work task by utilizing the sub-tables.
In one embodiment, the data table includes separators, and the obtaining module is further configured to obtain correspondence between the separators and the fields; the splitting module is further used for extracting data corresponding to the fields in the data table according to the corresponding relation; and splitting the extracted data by utilizing the script file.
In one embodiment, the work task also carries a database identifier and a database user identifier; the device further comprises:
the data table creating module is used for selecting the database corresponding to the database identification; inquiring whether a required data table name exists in a data table corresponding to the database user identification in the selected database; if not, extracting the required data in the data table corresponding to the database user identification by using the field; data tables required for the work task are created using the required data.
In one embodiment, the obtaining module is further configured to obtain the number of characters of the data table name; the device further comprises:
the verification module is used for respectively verifying the data table name and the number of characters of the data table name; and if the name of the data table fails to be checked and/or the number of the characters of the name of the data table fails to be checked, generating error prompt information.
In one embodiment, the obtaining module is further configured to obtain an idle time of the database; the task execution module is further used for loading the split sub-tables in the idle time of the database and executing the work task by using the sub-tables.
According to the data table splitting method and device, when a work task is executed, the name, the field and the like of the data table carried in the work task are input into the configuration page, the script file for splitting the data table is generated according to the name and the field of the data table, the data table is split by the script file to obtain a plurality of sub-tables, and then the split sub-tables are loaded to execute the work task. Because the data table is divided into a plurality of sub-tables for loading, the problem that the loading consumes long time when the storage quantity in the data table is too large is effectively solved, and the working efficiency can be effectively improved.
Drawings
FIG. 1 is a flow diagram of a method for splitting a data table in one embodiment;
FIG. 2 is a schematic diagram of a configuration page in one embodiment;
FIG. 3 is a schematic diagram of a server in one embodiment;
FIG. 4 is a diagram illustrating a structure of a device for splitting a data table according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In an embodiment, as shown in fig. 1, a method for splitting a data table is provided, which is described by taking an example that the method is applied to a terminal, and specifically includes:
And the terminal acquires the work task. The work task carries task items, required data table names, fields, field types and the like. Furthermore, the number of databases is multiple, and the work tasks can also carry database identifiers, database user identifiers and the like. Thereby quickly locating the required data table through the database identification, the database user identification and the like.
And 104, displaying a configuration page, and receiving the data table name and the field input by the user in the configuration page.
And 106, generating a script file for splitting the data table according to the name and the field of the data table.
And 108, acquiring a corresponding data table according to the name of the data table, and splitting the data table by using the script file to obtain a plurality of sub-tables.
If the data amount in the data table is too large, the time consumed for the terminal to load the data table is increased, and therefore the efficiency of the terminal to execute the work task is reduced. In order to effectively improve the loading efficiency of the data table, the terminal may split the data table by using the second application program.
In order to facilitate the user to split different data tables according to different work tasks, the terminal can display a configuration page. The configuration page comprises a plurality of input boxes, and a user can respectively input required data table names, fields, field types and the like in the configuration page through the plurality of input boxes. Further, if there are multiple databases, the user may also enter a database identifier, a database user identifier, and the like in the configuration page. The configuration page is shown in fig. 2.
The configuration page may also include controls for creating script files. After the user finishes inputting, the user can click a control for creating the script file, and an instruction for creating the script file is generated through the control. The terminal generates a script file using the data table name, the field, and the field type according to an instruction to create the script file. Further, if there are multiple databases, the terminal may generate a corresponding script file using the database identifier, the database user identifier, the data table name, the field, and the field type.
And the terminal acquires the required data table from the database according to the name of the data table. If a plurality of data exist, the terminal can acquire the required data table according to the database identification, the database user identification and the data table name. The data table stores two-dimensional relational data. The data table may adopt various storage formats, for example, an excel format, a pdf format, a word format, and the like.
The terminal can split the data table by using the generated script file, so that a plurality of sub-tables can be reached. The terminal can split the data table according to the line number by using the script file. Specifically, before generating the script file, the user may input the number of rows of each sub-table in the configuration page according to the needs of the job task. This number of rows may also be referred to as the split number of rows. For example, the number of split rows may be ten thousand. And the terminal reads the acquired data table line by using the script file, and splits the data table according to the splitting line number to obtain a plurality of sub-tables.
The terminal can also split the data table according to time by using the script program. Specifically, before generating the script file, the user may input a time period corresponding to each sub-table in the configuration page according to the requirement of the work task. This time period may also be referred to as a split time period. For example, the split period is one week. And the second application program reads the time field of the acquired data table by using the script file, and divides the data belonging to the same time period into a sub-table, so that the splitting of the data table is completed. Further, if the data volume in the sub-table obtained after splitting according to the time exceeds a preset value, the terminal can further continue splitting the sub-table corresponding to the splitting time according to the splitting line number, so that a plurality of sub-tables are obtained.
The terminal can also split the data table according to the mechanism or the region by using the script program. Specifically, before generating the script file, the user may input the mechanism or region corresponding to each sub-table in the configuration page according to the requirement of the work task. The facility may be referred to as a split facility and the region may be referred to as a split region. And the terminal reads the mechanism field or the area field of the acquired data table by using the script file, and divides the data belonging to the same mechanism or area into a sub-table so as to finish the splitting of the data table. Further, if the data volume in the sub-table obtained after splitting according to the mechanism or the region exceeds a preset value, the terminal can further continue to split the sub-table corresponding to the mechanism or the region according to the splitting line number. Further, in order to conveniently execute a work task by using the mechanism sub-table or the region sub-table corresponding to a certain time period, the terminal may further split the mechanism sub-table or the region sub-table according to the split mechanism or the region sub-table, the split time, and the split line number, for example, a year of settlement is performed on policy data, and the mechanism sub-table or the region sub-table may be further split according to the split time and the split line number, so as to obtain the mechanism sub-table or the region sub-table of each month.
And step 110, loading the split sub-tables and executing the work task by using the sub-tables.
And the terminal loads the split sub-tables, reads data in the sub-tables and executes a work task by using the read data. For example, the first application calculates a reserve of the policy or the like using the data in the plurality of read child tables.
In this embodiment, when a work task is executed, the name and the field of a data table carried in the work task are input into a configuration page displayed by a terminal, the terminal generates a script file for splitting the data table according to the name and the field of the data table, the script file is used to split the data table to obtain a plurality of sub-tables, and then the split sub-tables are loaded to execute the work task. Because the data table is divided into a plurality of sub-tables for loading, the problem that the loading consumes long time when the storage quantity in the data table is too large is effectively solved, and the working efficiency can be effectively improved.
In one embodiment, the data table includes separators, and the step of splitting the data table by using the script file includes: acquiring the corresponding relation between the separator and the field by using a terminal; extracting data corresponding to the fields in the data table according to the corresponding relation; and splitting the extracted data by using the script file.
In this embodiment, two-dimensional relational data is stored in the data table. The data table may adopt various storage formats, for example, an excel format, a pdf format, a word format, and the like. For a pdf-formatted data table or a word-formatted data table, if the data in the data table is not separated by columns, but is separated by separators, the data table can be split by the correspondence of the separators and the columns (i.e., the separators and the fields). Specifically, the work task carries the corresponding relationship between the separators and the fields in the data table. For example, each delimiter corresponds to a field in the order of the delimiter. The terminal obtains the corresponding relation between the separator and the field, extracts the data corresponding to the field in the data table according to the corresponding relation, and splits the extracted data by using the script file. Specifically, the terminal may split the extracted data according to the splitting line number, the splitting time, the splitting mechanism, or the splitting area in the manner mentioned in the above embodiment. Furthermore, the terminal can also split the extracted data according to the splitting time and the splitting line number, or split the extracted data according to the splitting mechanism, the splitting time and the splitting line number. Thereby obtaining a plurality of sub-tables. Because the separator and the field have corresponding relation, the data table with various formats can be automatically split by utilizing the corresponding relation and the script file. Therefore, the data tables in various formats stored in the database can be conveniently split according to the requirements of the work tasks.
In one embodiment, the work task also carries a database identifier and a database user identifier; the step of obtaining the corresponding data table according to the name of the data table further comprises the following steps: selecting a database corresponding to the database identifier; inquiring whether a required data table name exists in a data table corresponding to the database user identification in the selected database; if not, extracting the required data in the data table corresponding to the database user identification by using the field; data tables required for the work task are created using the required data.
In this embodiment, before splitting the data table required by the work task, the terminal may first query whether the data table exists in the database, and if the data table exists, the data table may be directly split according to the manner provided in the above embodiment. If not, the terminal will create the required data table in the database. Specifically, the terminal selects a corresponding database according to a database identifier carried in the work task, searches a plurality of data tables corresponding to the database user identifier in the selected database, and judges whether the data tables required by the work task exist in the plurality of searched data tables. If the data does not exist, the terminal extracts corresponding data from the plurality of data tables corresponding to the database user identifier by using the fields and the field types carried in the job task, and the extracted data can also be called data required by the job task (also can be simply called required data). The terminal creates a data table required for the job task by fields and field types using the required data. So that the data table splitting can be performed in the manner provided in the above embodiments.
In one embodiment, before the step of splitting the data table by using the script file, the method further comprises the following steps: acquiring the number of characters of the name of the data table; checking the name of the data table and the number of characters of the name of the data table respectively; and if the name of the data table fails to be checked and/or the number of the characters of the name of the data table fails to be checked, generating error prompt information.
In this embodiment, in order to ensure that the splitting of the data table can be performed smoothly, the terminal may also check the name of the data table before the splitting of the data table. Specifically, the terminal obtains a data table name carried in the work task, and calculates the number of characters of the data table name. The terminal can check the name of the data table first and then check the number of characters of the name of the data table, the terminal can check the number of characters of the name of the data table first and then check the name of the data table, and the terminal can check the name of the data table and the number of characters of the name of the data table simultaneously.
Taking the example that the terminal can check the name of the data table first and then check the number of characters of the name of the data table, the terminal selects a corresponding database according to the database identification carried by the work task, and searches a plurality of names of the data table corresponding to the user identification of the database in the selected database. The terminal judges whether the data table name carried by the work task exists in the plurality of searched data table names, if so, the terminal further judges whether the data table name carried by the work task is unique, if not, the data table name is wrong, and the verification cannot pass. If the number of the characters of the data table name is unique, the terminal further judges whether the number of the characters of the data table name exceeds a preset number of the characters, and if the number of the characters of the data table name exceeds the preset number of the characters, the verification cannot pass. When the data table name and/or the number of characters of the data table name fail to pass the verification, the data table cannot be split, and the terminal can generate error prompt information so that a user can process the error prompt information in time.
In one embodiment, the step of loading the plurality of split sub-tables and executing the work task by using the plurality of sub-tables comprises: acquiring the idle time of a database; and loading the split sub-tables at the idle time of the database, and executing the work task by using the sub-tables.
In this embodiment, the work task may be monthly calculation, fund preparation calculation, or share insurance operation of the policy. The computational logic associated with the policy can be complex and the database provides support for the work tasks of multiple terminals. In order to effectively alleviate the resource consumption of the database, the work tasks with low requirements for real-time processing can be executed in the idle time of the database.
The idle time of a database refers to the time when the database performance is optimal. The database is deployed on a server. And the server runs the monitoring script, monitors the plurality of databases by using the monitoring script, and obtains the idle time of the databases according to the monitoring information. The idle time of the database may be the entire period of inactivity or may be a portion of the period of inactivity. The terminal can interact with the server to obtain the idle time of the database. And the terminal loads the split sub-tables in the idle time of the database and executes the work task by utilizing the loaded sub-tables.
Further, the terminal may utilize a plurality of model algorithms to calculate the amounts of the plurality of items while performing the work task. For example, the work task is a policy share, and the items to be calculated include a valid warranty, a risk warranty, and a reserve. The model algorithms for different risk categories may be different for the same project. The traditional way is to embed a model algorithm into a program corresponding to a certain project. There may be cross-overs in the calculations between different items, for example, in calculating risk reserves, there is also a need to calculate effective reserves. The same model algorithm may be embedded in programs of different projects, thereby resulting in repeated development of the same model algorithm in programs of different projects.
In order to effectively avoid repeated development of the model algorithm and reduce redundancy of the project program, the model algorithm can be stored independently and can be called through an interface when a certain model algorithm is used. Specifically, a second configuration table is established on the server, and the second configuration table includes fields such as a risky seed, an algorithm identifier, an interface and the like. The server searches out corresponding algorithm identification and interface in the second configuration table according to the dangerous species in the policy data, each dangerous species has a corresponding model algorithm, and the model algorithm can be directly called through the interface corresponding to the algorithm identification to calculate the corresponding money.
And the terminal sends a request for acquiring the second configuration table to the server to obtain the second configuration table returned by the server. The fields of the sub-table include the risk seeds. The work task also carries a number of items that need to be calculated. And in the idle time of the database, the terminal searches out the algorithm identification and the interface corresponding to the project from the second configuration table according to the dangerous seed, calls the model algorithm corresponding to the algorithm identification through the interface, and respectively calculates the money amounts of the projects in the work task according to the model algorithm and the loaded sub-tables.
In the process, the corresponding model algorithm does not need to be configured for each item, and the corresponding algorithm identification and the corresponding interface are retrieved through the second configuration table, so that the corresponding model algorithm can be directly called for calculation. Repeated development of project programs is effectively avoided, and program redundancy is effectively reduced.
In one embodiment, as shown in FIG. 3, a terminal 300 is provided that includes a processor 301, a non-volatile storage medium 302, a memory 303, a network interface 304, a display screen 305, and an input device 306 connected by a system bus. The non-volatile storage medium 302 of the server stores an operating system 3021 and a data table splitting apparatus 3022, which is used to reduce the time consumption for loading the data table and improve the work efficiency. The processor 301 of the terminal 300, for providing computing and control capabilities, is configured to perform a method of splitting a data table. The memory 303 of the terminal 300 provides an environment for the operation of the splitting apparatus of the data table in the non-volatile storage medium, and the memory 303 may store computer readable instructions, which when executed by the processor, may cause the processor to execute a splitting method of the data table. The network interface 304 of the server 300 is used for communicating with an external terminal or server through a network connection, such as sending a request for obtaining the second configuration table to the server. The display 305 of the terminal 300 may be a liquid crystal display or an electronic ink display, and the input device 306 may be a touch layer covered on the display 305, a button, a track ball or a touch pad arranged on a housing of the terminal 300, or an external keyboard, a touch pad or a mouse. Those skilled in the art will appreciate that the configuration shown in fig. 3 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation on the terminal to which the present application is applied, and that a particular terminal may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, as shown in fig. 4, there is provided a splitting apparatus for a data table, including: an obtaining module 402, a presentation module 404, a receiving module 406, a splitting module 408, and a task executing module 410, wherein:
an obtaining module 402, configured to obtain a work task, where the work task carries a data table name and a field.
A presentation module 404, configured to present the configuration page.
A receiving module 406, configured to receive the data table name and the field input by the user in the configuration page.
The splitting module 408 is configured to generate a script file for splitting the data table according to the name and the field of the data table; and acquiring a corresponding data table according to the name of the data table, and splitting the data table by using the script file to obtain a plurality of sub-tables.
And the task execution module 410 is configured to load the split sub-tables, and execute the work task by using the sub-tables.
In one embodiment, the data table includes separators, and the obtaining module 402 is further configured to obtain correspondence between the separators and the fields; the splitting module 408 is further configured to extract data corresponding to the field from the data table according to the corresponding relationship; and splitting the extracted data by using the script file.
In one embodiment, the work task also carries a database identifier and a database user identifier; the device also includes: a data table creation module 412 for selecting a database corresponding to the database identification; inquiring whether a required data table name exists in a data table corresponding to the database user identification in the selected database; if not, extracting the required data in the data table corresponding to the database user identification by using the field; data tables required for the work task are created using the required data.
In one embodiment, the obtaining module 402 is further configured to obtain the number of characters of the data table name; the device also includes: the checking module 414 is configured to check the data table name and the number of characters of the data table name respectively; and if the name of the data table fails to be checked and/or the number of the characters of the name of the data table fails to be checked, generating error prompt information.
In one embodiment, the obtaining module 402 is further configured to obtain the idle time of the database; the task execution module 410 is further configured to load the split sub-tables during idle time of the database, and execute the work task using the sub-tables.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, and the program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only show some embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of splitting a data table, the method comprising:
acquiring a work task, wherein the work task carries a data table name, a field and a plurality of items;
displaying a configuration page, and receiving a database identifier, a database user identifier, a data table name, a field and a splitting time period corresponding to each sub-table, which are input in the configuration page by a user;
generating a script file for splitting a data table according to the database identification, the database user identification, the data table name and the field;
acquiring a corresponding data table according to the database identification, the database user identification and the data table name, reading a time field in the data table by using the script file to split the data table, and dividing data belonging to the same time period into the same sub-table to obtain a plurality of sub-tables; the fields of the sub-table comprise dangerous seeds;
acquiring a second configuration table, retrieving an algorithm identifier and an interface corresponding to the item in the second configuration table according to the dangerous species, and calling a model algorithm corresponding to the algorithm identifier through the interface;
and loading the split sub-tables, and executing the calculation of a plurality of items in the work task by utilizing the sub-tables according to the model algorithm.
2. The method of claim 1, wherein the data table includes delimiters, and wherein splitting the data table using the script file comprises:
acquiring the corresponding relation between the separator and the field;
extracting data corresponding to the fields in the data table according to the corresponding relation;
and splitting the extracted data by utilizing the script file.
3. The method of claim 1, wherein the work task further carries a database identifier and a database user identifier; before the step of obtaining the corresponding data table according to the data table name, the method further comprises the following steps:
selecting a database corresponding to the database identifier;
inquiring whether a required data table name exists in a data table corresponding to the database user identification in the selected database;
if not, extracting the required data in the data table corresponding to the database user identification by using the field;
data tables required for the work task are created using the required data.
4. The method of claim 1, further comprising, prior to the step of splitting the data table using the script file:
acquiring the number of characters of the name of the data table;
checking the data table name and the number of characters of the data table name respectively;
and if the name of the data table fails to be checked and/or the number of the characters of the name of the data table fails to be checked, generating error prompt information.
5. The method of any of claims 1-4, wherein the step of loading the split plurality of sub-tables, and using the plurality of sub-tables to perform the computation of the plurality of items in the work task according to the model algorithm comprises:
acquiring the idle time of a database;
and loading the split sub-tables at the idle time of the database, and executing the work task by using the sub-tables.
6. An apparatus for splitting a data table, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a work task, and the work task carries a data table name, a field and a plurality of projects;
the display module is used for displaying the configuration page;
the receiving module is used for receiving a database identifier, a database user identifier, a data table name, a field and a splitting time period corresponding to each sub-table, which are input in the configuration page by a user;
the splitting module is used for generating a script file for splitting the data table according to the database identification, the database user identification, the data table name and the field; acquiring a corresponding data table according to the database identification, the database user identification and the data table name, reading a time field in the data table by using the script file to split the data table, and dividing data belonging to the same time period into the same sub-table to obtain a plurality of sub-tables; the fields of the sub-table comprise dangerous seeds;
the task execution module is used for acquiring a second configuration table, retrieving the algorithm identification and the interface corresponding to the project from the second configuration table according to the dangerous species, and calling the model algorithm corresponding to the algorithm identification through the interface; and loading the split sub-tables, and executing the calculation of a plurality of items in the work task by utilizing the sub-tables according to the model algorithm.
7. The apparatus according to claim 6, wherein the data table includes separators, and the obtaining module is further configured to obtain correspondence between the separators and the fields; the splitting module is further used for extracting data corresponding to the fields in the data table according to the corresponding relation; and splitting the extracted data by utilizing the script file.
8. The device of claim 6, wherein the work task further carries a database identifier and a database user identifier; the device further comprises:
the data table creating module is used for selecting the database corresponding to the database identification; inquiring whether a required data table name exists in a data table corresponding to the database user identification in the selected database; if not, extracting the required data in the data table corresponding to the database user identification by using the field; data tables required for the work task are created using the required data.
9. The apparatus of claim 6, wherein the obtaining module is further configured to obtain a number of characters of the data table name; the device further comprises:
the verification module is used for respectively verifying the data table name and the number of characters of the data table name; and if the name of the data table fails to be checked and/or the number of the characters of the name of the data table fails to be checked, generating error prompt information.
10. The apparatus according to any one of claims 6-9, wherein the obtaining module is further configured to obtain an idle time of the database; the task execution module is further used for loading the split sub-tables in the idle time of the database and executing the work task by using the sub-tables.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710087369.4A CN107798030B (en) | 2017-02-17 | 2017-02-17 | Splitting method and device of data table |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710087369.4A CN107798030B (en) | 2017-02-17 | 2017-02-17 | Splitting method and device of data table |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107798030A CN107798030A (en) | 2018-03-13 |
| CN107798030B true CN107798030B (en) | 2020-07-03 |
Family
ID=61530152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710087369.4A Active CN107798030B (en) | 2017-02-17 | 2017-02-17 | Splitting method and device of data table |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107798030B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595597A (en) * | 2018-04-19 | 2018-09-28 | 平安科技（深圳）有限公司 | Data lead-in method, device, computer equipment and storage medium |
| CN109102754B (en) * | 2018-06-20 | 2020-12-01 | 新华三大数据技术有限公司 | Data map generation method and device |
| CN109614622A (en) * | 2018-12-11 | 2019-04-12 | 北京锐安科技有限公司 | Valid data extracting method, device, storage medium and terminal |
| US20210019299A1 (en) * | 2019-07-17 | 2021-01-21 | Aveva Software, Llc | System and server comprising database schema for accessing and managing utilization and job data |
| CN112434078A (en) * | 2020-11-20 | 2021-03-02 | 广州奇享科技有限公司 | Boiler data processing method, device, equipment and storage medium |
| CN112767013A (en) * | 2021-01-05 | 2021-05-07 | 北京锐安科技有限公司 | Business report splitting method, device, server and storage medium |
| CN114119202B (en) * | 2021-11-29 | 2022-08-05 | 阿尔法时刻科技(深圳)有限公司 | Automatic configuration method and system for credit management platform |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598212A (en) * | 2013-10-30 | 2015-05-06 | 上海联影医疗科技有限公司 | Image processing method and device based on algorithm library |
| CN105677846A (en) * | 2016-01-06 | 2016-06-15 | 中国传媒大学 | Recommendation system and construction method thereof |
| CN106156186A (en) * | 2015-04-21 | 2016-11-23 | 阿里巴巴集团控股有限公司 | A kind of data model managing device, server and data processing method |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7089255B1 (en) * | 2001-03-30 | 2006-08-08 | Unisys Corporation | Intelligent step presentation |
| US6691132B2 (en) * | 2001-05-16 | 2004-02-10 | Reengineering Llc | Semantic encoding and compression of database tables |
| CN101566986A (en) * | 2008-04-21 | 2009-10-28 | 阿里巴巴集团控股有限公司 | Method and device for processing data in online business processing |
| CN102682044A (en) * | 2011-04-14 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Method for splitting tables for PHP MYSQL database |
| US9406020B2 (en) * | 2012-04-02 | 2016-08-02 | Taiger Spain Sl | System and method for natural language querying |
| CN104298695B (en) * | 2013-07-19 | 2020-06-16 | 腾讯科技（深圳）有限公司 | Data caching method and device and server |
| CN106168949B (en) * | 2016-05-03 | 2018-11-16 | 泰康保险集团股份有限公司 | The method and device that database is split |
-
2017
- 2017-02-17 CN CN201710087369.4A patent/CN107798030B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598212A (en) * | 2013-10-30 | 2015-05-06 | 上海联影医疗科技有限公司 | Image processing method and device based on algorithm library |
| CN106156186A (en) * | 2015-04-21 | 2016-11-23 | 阿里巴巴集团控股有限公司 | A kind of data model managing device, server and data processing method |
| CN105677846A (en) * | 2016-01-06 | 2016-06-15 | 中国传媒大学 | Recommendation system and construction method thereof |
Non-Patent Citations (2)
| Title |
|---|
| Dynamic Interface Selection in Portable Multi-interface Terminals;Sowmia Devi M.K. et al;《2007IEEE International Conference on Portable Information Devices》;20070604;第1-5页 * |
| 基于PaaS的云应用软件部署环境的元模型;刘欢欢 等;《计算机科学》;20151031;第42卷(第10期);第45-49,80页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107798030A (en) | 2018-03-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107798030B (en) | Splitting method and device of data table | |
| US11030166B2 (en) | Smart data transition to cloud | |
| JP5298117B2 (en) | Data merging in distributed computing | |
| CN110795455A (en) | Dependency relationship analysis method, electronic device, computer device and readable storage medium | |
| CN108492005B (en) | Project data processing method and device, computer equipment and storage medium | |
| CN110765750B (en) | Report data input method and terminal equipment | |
| US10497067B2 (en) | System for perfoming an extraction process on input data containing XBRL files using a combination of extraction criteria | |
| JP7015319B2 (en) | Data analysis support device, data analysis support method and data analysis support program | |
| CN111427577A (en) | Code processing method and device and server | |
| CN111680478B (en) | Report generation method, device, equipment and storage medium based on configuration software | |
| CN103235847B (en) | Based on the model dynamic adaptation method of matrix conversion | |
| CN112395182A (en) | Automatic testing method, device, equipment and computer readable storage medium | |
| US20200380037A1 (en) | Information Retrieval Precision Evaluation Method, System and Device and Computer-Readable Storage Medium | |
| CN111026574B (en) | Method and device for diagnosing elastiscearch cluster problem | |
| EP3657351B1 (en) | Smart data transition to cloud | |
| CN116595044A (en) | Optimization method, storage medium and equipment for database selectivity calculation | |
| CN112597162B (en) | Data set acquisition method, system, equipment and storage medium | |
| CN115774672A (en) | Test case generation method, test case generation device, server and storage medium | |
| CN116414594A (en) | Fault tree updating method, device, computer equipment and storage medium | |
| US20120005614A1 (en) | Account conversion of information technology systems | |
| CN113448985A (en) | API (application program interface) interface generation method, calling method and device and electronic equipment | |
| CN113051171A (en) | Interface test method, device, equipment and storage medium | |
| CN112668285A (en) | Method and device for generating fund daily report by combining RPA and AI and electronic equipment | |
| CN106844634B (en) | Database transaction optimization method and system | |
| CN113326042B (en) | Method, device, storage medium and processor for visual configuration salary |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |