CN112286934A

CN112286934A - Database table importing method, device, equipment and medium

Info

Publication number: CN112286934A
Application number: CN202011185991.7A
Authority: CN
Inventors: 林达坤
Original assignee: Ping An Trust Co Ltd
Current assignee: Ping An Trust Co Ltd
Priority date: 2020-10-29
Filing date: 2020-10-29
Publication date: 2021-01-29

Abstract

The invention relates to the field of data processing, and provides a method, a device, equipment and a medium for importing a database table, wherein the method comprises the following steps: obtaining a document to be imported and total project information; acquiring a first data row cell array and a first data column cell array by using an Apache poi technology, and simultaneously creating a database table; inquiring an item basic table, and acquiring a member list from the item basic table; identifying a first type through a cell data type model, acquiring data rules matched with each first type, and acquiring first attribute cells in a table to be imported; executing data processing operation before importing by using an Apache poi technology to obtain first data to be imported; and correspondingly importing the member identification codes, the first data column cells and the first data to be imported into a database table according to a column type storage mode. The invention realizes the import without fixing the header, and adopts the column type storage mode, thereby increasing the flexibility of data import and saving the workload of secondary development.

Description

Database table importing method, device, equipment and medium

Technical Field

The invention relates to the technical field of data processing of big data, in particular to a method, a device, equipment and a medium for importing a database table.

Background

At present, the electronic form is often imported into a fixed header appointed with a demander or a user, a system is developed according to the fixed header, the user exports a template from the system, then data is filled or edited locally according to the template (the header cannot be changed), the system reads data corresponding to each column row by identifying each column header during import, and finally the first database table to be imported is used, however, in an actual use scene, the user often feeds back some special scenes, for example, the form style cannot be unified, the header is different from the appointed fixed header (even if the header is only one new column), and the like, for example: in the salary distribution of the project, as one project comprises a plurality of sub-projects, and the roles of the members in one project in the sub-projects are different, the salary distribution is different, and in the project, the sub-projects can be added and deleted, the members can be added and removed, and a class of roles can be added or removed, so that the header can not be fixed in the process of importing the salary spreadsheet every day or every month. At this time, because the system cannot be compatible with the introduction of different headers, the requirements of the special scenes cannot be met, and further, headers corresponding to the electronic forms need to be separately developed for the special scenes one by one, so that the total amount of development labor is increased, and because the fixed headers cannot be changed, the maintainability after the electronic forms are generated is poor.

Disclosure of Invention

The invention provides a method and a device for importing a database table, computer equipment and a storage medium, which realize the import of an electronic form without a fixed header, and store the electronic form by adopting a column type storage mode, thereby increasing the flexibility of data import, saving the workload of secondary development due to non-uniform headers, greatly reducing the cost and improving the efficiency of data import.

A database table import method, comprising:

receiving a first import request, and acquiring documents to be imported and total project information in the first import request; the document to be imported comprises a table to be imported; the total project information comprises a total project name and an import table name;

acquiring a first data row cell array and a first data column cell array in the table to be imported by an Apache poi technology, and creating a database table corresponding to the import table name in a database; the first data row cell array comprises at least one first data row cell, and the first data column cell array comprises at least one first data column cell;

inquiring an item basic table matched with the total item name from the database, and acquiring a member list matched with the content of each first data row cell from the item basic table; the member list comprises member identification codes and basic information associated with the member identification codes;

identifying a first type matched with the content of each first data column cell through a cell data type model, acquiring a data rule matched with each first type, and acquiring a first attribute cell in the table to be imported; the first attribute cell is a cell corresponding to both one of the first data row cells and one of the first data column cells;

executing pre-import data processing operation on each first attribute cell by using an Apache poi technology to obtain first to-be-imported data corresponding to each first attribute cell; the pre-import data processing operation refers to verifying the first attribute cell according to the first type corresponding to the first attribute cell to obtain data to be processed, and then obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule and the member list;

and correspondingly importing each member identification code, each first data column unit cell and each first data to be imported into the database table in a column type storage mode to finish the first import request.

A database table importing apparatus, comprising:

the receiving module is used for receiving a first import request and acquiring documents to be imported and total project information in the first import request; the document to be imported comprises a table to be imported; the total project information comprises a total project name and an import table name;

the acquisition module is used for acquiring a first data row cell array and a first data column cell array in the table to be imported through an Apache poi technology, and meanwhile, creating a database table corresponding to the import table name in a database; the first data row cell array comprises at least one first data row cell, and the first data column cell array comprises at least one first data column cell;

the query module is used for querying an item basic table matched with the total item name from the database and acquiring a member list matched with the content of each first data row cell from the item basic table; the member list comprises member identification codes and basic information associated with the member identification codes;

the identification module is used for identifying a first type matched with the content of each first data column cell through a cell data type model, acquiring a data rule matched with each first type, and acquiring a first attribute cell in the table to be imported; the first attribute cell is a cell corresponding to both one of the first data row cells and one of the first data column cells;

the execution module is used for executing data processing operation before importing to each first attribute cell through an Apache poi technology to obtain first data to be imported corresponding to each first attribute cell; the pre-import data processing operation refers to verifying the first attribute cell according to the first type corresponding to the first attribute cell to obtain data to be processed, and then obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule and the member list;

and the importing module is used for correspondingly importing the member identification codes, the first data column cells and the first data to be imported into the database table in a column type storage mode to finish the first importing request.

A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the above database table importing method when executing the computer program.

A computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the above-described database table importing method.

According to the database table importing method, the database table importing device, the computer equipment and the storage medium, documents to be imported and total project information in the first import request are obtained; the document to be imported comprises a table to be imported; the total project information comprises a total project name and an import table name; acquiring a first data row cell array and a first data column cell array in the table to be imported by an Apache poi technology, and creating a database table corresponding to the import table name in a database; inquiring an item basic table matched with the total item name from the database, and acquiring a member list matched with the content of each first data row cell from the item basic table; identifying a first type matched with the content of each first data column cell through a cell data type model, acquiring a data rule matched with each first type, and acquiring a first attribute cell in the table to be imported; executing pre-import data processing operation on each first attribute cell by using an Apache poi technology to obtain first to-be-imported data corresponding to each first attribute cell; the pre-import data processing operation refers to verifying the first attribute cell according to the first type corresponding to the first attribute cell to obtain data to be processed, and then obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule and the member list; the member identification codes, the first data column cells and the first data to be imported are correspondingly imported into the database table according to a column type storage mode, so that the import of the electronic form without fixing the header is realized, the limitation of the position and the content of the header is broken, the form can be freely designed along with the increase/deletion of the items or members (roles), the effect of importing the electronic form into the database table without the limitation of the header is achieved, the column type storage mode is adopted for storage, the foundation without the limitation of the header is provided for the subsequent modification of the same database table, the flexibility of data import is improved, the workload of secondary development due to non-uniform headers is saved, the cost is greatly reduced, and the data import efficiency is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.

FIG. 1 is a diagram of an application environment of a database table importing method according to an embodiment of the present invention;

FIG. 2 is a flow diagram of a database table import method in an embodiment of the invention;

FIG. 3 is a flowchart of step S20 of the database table import method according to an embodiment of the present invention;

FIG. 4 is a flowchart of step S40 of the database table import method according to an embodiment of the present invention;

FIG. 5 is a flowchart of step S50 of the database table import method according to an embodiment of the present invention;

FIG. 6 is a flowchart of step S50 of the database table import method according to another embodiment of the present invention;

FIG. 7 is a schematic block diagram of a database table importing apparatus according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a computer device in an embodiment of the invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The database table importing method provided by the invention can be applied to the application environment shown in fig. 1, wherein a client (computer device) communicates with a server through a network. The client (computer device) includes, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, cameras, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers.

In an embodiment, as shown in fig. 2, a database table importing method is provided, and the technical solution thereof mainly includes the following steps S10-S60:

s10, receiving a first import request, and acquiring the document to be imported and the total project information in the first import request; the document to be imported comprises a table to be imported; the total item information includes a total item name and an import table name.

Understandably, the first import request is a request triggered when the document to be imported needs to be imported, the total item information is information related to the document to be imported, the total item information can be acquired after the total item information is input on an import interface of application software, the total item information includes a total item name and an import table name, the total item name is a unique identification name given to a total item, the import table name is a name given to an imported database table, the document to be imported is a document to be imported into the database, the file type of the document to be imported is not limited, and can be a document in a Word format or an Excel format, the document to be imported includes the table to be imported, the table to be imported is a table to be imported, for example, the table in the document to be imported in the Word format, or the table to be imported is a table of a workbook in the document to be imported in an Excel format, and the like.

S20, acquiring a first data row cell array and a first data column cell array in the table to be imported through an Apache poi technology, and creating a database table corresponding to the import table name in a database; the first data row cell array comprises at least one first data row cell, and the first data column cell array comprises at least one first data column cell.

Understandably, the Apache poi technology is a technology of a cross-platform Java API (Java Application Programming Interface) written in Java, and is mainly used for reading and writing documents of office software, and the Apache poi technology identifies file type information of the documents to be imported, where the file type information includes a file format and a file version, and the file format is a file type of the office software of the documents to be imported, such as: excel file type, Word file type, Powerpoint file type, etc., wherein the file version is a version in a file format, such as: version 95, version 07, etc.; inquiring Apache poi configuration components matched with the file type information from a configuration component management center, wherein the configuration component management center is a management center for managing all Apache poi configuration components, the configuration component management center comprises all Apache poi configuration components, the Apache poi configuration components are environment components configured according to file types of different office software, when a file of one office software needs to be operated, the Apache poi configuration components corresponding to the file are operated, so that a first data row cell array and a first data column cell array in the table to be imported are obtained, the first data row cell array is a cell set from a cell corresponding to a preset column of a first row in the table to be imported to a cell with an empty next cell in the row, and the first data column cell array is a cell set from the cell corresponding to the preset row of the first column in the table to be imported to the next cell with the empty cell array of the next column in the table to be imported Starting with a set of cells between the cells in the column for which the next cell is empty.

And creating a database table corresponding to the import table name in a database, wherein the database is a column database, for example, distributed databases such as Hbase, Vertica and Greenplus can all adopt a column storage mode, and the database table is a column-stored database created in the database.

In an embodiment, as shown in fig. 3, the step S20, namely, the obtaining, by using an Apache poi technique, a first data row cell array and a first data column cell array in the table to be imported includes:

s201, identifying the file type information of the document to be imported.

Understandably, the file type information includes a file format and a file version, and the file format is a file type of the office software to be imported with the document, such as: excel file type, Word file type, Powerpoint file type, etc., wherein the file version is a version in a file format, such as: version 95, version 07, etc.

S202, an Apache poi configuration component matched with the file type information is obtained from a configuration component management center, and the Apache poi configuration component is operated.

Understandably, the file format of the document to be imported is identified, then the file version of the document to be imported is identified, all the Apache poi configuration components matched with the file format and the file version are inquired, the configuration component management center comprises all the Apache poi configuration components, the Apache poi configuration components are environment components configured aiming at different file types of office software, and when the file of one office software needs to be operated, the Apache poi configuration components corresponding to the file are operated.

S203, obtaining the first data row cell array and the first data column cell array in the table to be imported.

Understandably, by the Apache poi technique, the first data row cell array containing at least one of the first data row cells is acquired, while the first data column cell data containing at least one of the second data column cells is acquired.

The invention realizes the purpose of identifying the file type information of the document to be imported; acquiring an Apache poi configuration component matched with the file type information from a configuration component management center, and operating the Apache poi configuration component; the method comprises the steps of obtaining a first data row cell array and a first data column cell array in a table to be imported, so that documents to be imported of different file types can be identified through an Apache poi technology, obtaining data in the table to be imported, improving diversity and convenience of data import, and improving satisfaction of users.

S30, searching an item basic table matched with the total item name from the database, and acquiring a member list matched with the content of each first data row cell from the item basic table; the member list includes member identification codes and basic information associated with the member identification codes.

Understandably, the database includes all the item basic tables corresponding to the total items one by one, the item basic table is a table of all basic information in one total item, for example, the item basic table includes members, gender, role, item proportion, and the like, the item basic table matching with the total item name is inquired from the database, the content of the first data row cell can be set according to requirements, for example, the content of the first data row cell is an item manager group, a quality inspection group, and the like, the member list matching with the content of each first data row cell is obtained, the member list includes the member identification code matching with the content of the first data row cell and the basic information, the member identification code is a unique identification code assigned to each member, and the basic information is information of a basic class related to the member, the matching process may be exact matching or fuzzy matching, for example, the content of the first data line cell is "item manager group", and a member list with a role of "item manager" is obtained.

S40, identifying a first type matched with the content of each first data column cell through a cell data type model, acquiring a data rule matched with each first type, and acquiring a first attribute cell in the table to be imported; the first attribute cell refers to a cell corresponding to both one of the first data row cells and one of the first data column cells.

Understandably, the cell data type model identifies the first type corresponding to each of the first data column cells, the cell data type model is a trained neural network model, the cell data type model enables automatic identification of the first type matching with the first data column cells, the first type is a data type identified according to contents of the first data column cells, the first type includes a value type, a character type, a decimal point type, and the like, the data rule is a rule defined according to the first type, and one of the first types is associated with one of the data rules, for example: the content in the first data column cell is the income sum of the XX item, the characteristics of the XX item, the income and the sum are identified, the first type is determined to be a decimal point type, the data rule corresponding to the first type is divided according to the proportion of each member in the XX item, the first type matched with the content of one first data column cell is identified, the data type of the first attribute cell in the same row with the first data column cell is indicated to be the first type, therefore, the fixed content of a header or a first column is not required to be fixed, the first type matched with the first data column cell is automatically identified through a cell data type model, and the traditional limitation of header fixation is broken.

The first attribute cell is a cell corresponding to a column which is the same as one first data row cell and a row which is the same as one first data column cell.

In an embodiment, as shown in fig. 4, the identifying, in step S40, a first type matching the content of each cell of the first data column through the cell data type model includes:

s401, inputting the content of the first data column cell into the cell data type model; the cell data type model is a neural network model based on a Word2Vec model.

Understandably, the cell data type model is a trained neural network model based on a Word2Vec model, that is, the network structure of the cell data type model is trained on the basis of the network structure of the Word2Vec model, and the cell data type model can realize automatic identification of the first type of model matched with the first data column cell by extracting data type features.

S402, extracting data type characteristics through the cell data type model.

Understandably, the data type features are features related to item types and embodied after a text is converted into word vectors, the process of extracting the data type features is that the cell data type model divides the content of the first data row cell into a plurality of unit words, the unit words are subjected to word vector conversion to obtain data containing a plurality of word vectors, a weighting matrix is obtained through weighting processing, and the data type features are extracted from the weighting matrix.

S403, identifying the first type corresponding to the first data column unit cell according to the data type characteristics.

Understandably, according to the extracted data type features, predicting probability values of the data types, acquiring the data type corresponding to the highest probability value in all the probability values, and determining the data type as the first type corresponding to the first data column unit cell.

The invention realizes that the data type characteristics are extracted through the cell data type model by inputting the content of the first data column cells into the cell data type model; according to the data type characteristics, the first type corresponding to the first data column unit cell is identified, so that the first type corresponding to the first data column unit cell is automatically identified through a unit cell data type model based on a Word2Vec model, the content of the fixed unit cell is not required, the content requirement of the first data column unit cell is predicted through a natural language identification technology, the first type corresponding to the first data column unit cell is automatically identified, the limitation of the header content is broken, the flexibility of data import is improved, and the identification accuracy is improved.

S50, performing pre-import data processing operation on each first attribute cell through an Apache poi technology to obtain first data to be imported corresponding to each first attribute cell; the pre-import data processing operation refers to verifying the first attribute cell according to the first type corresponding to the first attribute cell to obtain data to be processed, and obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule and the member list.

Understandably, by using the Apache poi technology, performing pre-import data processing operation on each first attribute cell to obtain first data to be imported corresponding to the first attribute cell, where the pre-import data processing operation refers to verifying the first attribute cell according to the first type corresponding to the first attribute cell, that is, according to the first type corresponding to the first data column cell corresponding to the first attribute cell, and the verification processing process is to convert the content in the first attribute cell into data conforming to the first type, determine the verified data as the data to be processed, and then according to the first type, may determine the data rule corresponding to the first attribute cell, where the data rule includes a processing field and a processing formula, and the member list corresponding to the first attribute cell is the first data line corresponding to the first attribute cell The member list matched with the content of the cell obtains cell information matched with the member identification code and the processing field from the item basic table according to the data to be processed, the data rule and the member list corresponding to the first attribute cell, wherein the basic information comprises a plurality of cell information, the cell information corresponding to the member identification code and the processing formula are input to be processed, and the first data to be imported is obtained.

In an embodiment, as shown in fig. 5, in the step S50, that is, the checking the first attribute cell according to the first type corresponding to the first attribute cell to obtain to-be-processed data includes:

s501, detecting whether the content in the first attribute cell meets the first type condition corresponding to the first attribute cell.

Understandably, the content in the first attribute cell is acquired, the acquired content in the first attribute cell is judged, and whether the acquired content meets the condition of the first type corresponding to the first attribute cell is judged.

And S502, if the content in the first attribute cell is detected to be not in accordance with the first type of condition, converting the content in the first attribute cell through a regular expression technology in the Apache poi technology to obtain the data to be processed in accordance with the first type of condition.

Understandably, when it is detected that the first attribute cell does not meet the first type of condition, regularizing, i.e., converting, by using a regular expression technique in an Apache poi technique, the content in the first attribute cell into a data type meeting the first type of condition, so as to obtain the data to be processed, for example: the content of the first attribute cell is a text type of '1364' of the text input, and the corresponding first type is a numerical value type, the text type '1364' is subjected to text-to-digital conversion, the text type '1364' is converted into the numerical value type of the numerical input '1364', and the numerical value type '1364' is determined as the data to be processed of the first attribute cell.

S503, if it is detected that the content in the first attribute cell meets the condition of the first type, determining the content in the first attribute cell as the data to be processed.

Understandably, when the first attribute cell is detected to meet the first type of condition, the content in the first attribute cell is not required to be processed, and is directly removed as the data to be processed.

The invention realizes the verification of each first attribute cell through the regular expression technology in the Apache poi technology to obtain the data to be processed which accords with the first type condition corresponding to the first attribute cell, thus the data in the table to be imported can be automatically verified to accord with the requirement of the data type, the secondary modification operation that the user inputs different types and inputs the data again is avoided, the modification time is saved, the threshold of data input is reduced, the flexibility of data import is improved, and the customer satisfaction degree of the user is improved.

In an embodiment, as shown in fig. 6, in the step S50, that is, obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule, and the member list, includes:

s504, acquiring the data rule matched with the first type corresponding to the first attribute cell; the data rule includes a processing field and a processing formula.

Understandably, the data rule including a processing field and a processing formula is obtained, the processing field is a field corresponding to basic information related to the data rule, and the processing formula is a formula related to the processing of the data to be processed and the processing field, for example: the first type is a "decimal point type", a processing field of the corresponding data rule is an "item duty", and a processing formula of the processing field of the corresponding data rule is "C = a × B", where C is the first data to be imported, a is the data to be processed, and B is the processing field ".

And S505, acquiring the cell information matched with the member identification code and the processing field from all the basic information.

Understandably, the cell information matched with the member identification code and the to-be-processed field in the member list is inquired from all the basic information, such as: the data to be processed of the first attribute cell is 10000.00, the content in the first data column unit corresponding to the first attribute cell is "the total income amount of the XX item", the first type corresponding to the first attribute cell is "the decimal point type", the processing field of the data rule is "the item ratio", that is, the cell information corresponding to the "item ratio" field is obtained.

S506, inputting the cell information corresponding to the member identification code and the data to be processed into the processing formula to obtain the first data to be imported.

Understandably, the acquired cell information and the to-be-processed data which are in one-to-one correspondence with the member identification codes are input into the processing formula corresponding to the acquired cell information and the to-be-processed data, and the first to-be-imported data is obtained through processing of the processing formula, for example: the to-be-processed data of the first attribute cell is 10000.00, one member identification code is 0001, the corresponding cell information of the "item proportion" is 10%, and the corresponding processing formula is "C = a × B", where C is the first to-be-imported data, a is the to-be-processed data, and B is the processing field ", and then the first to-be-imported data corresponding to the member identification code (0001) is 1000.00.

The invention realizes the acquisition of the data rule matched with the first type corresponding to the first attribute cell; the data rule comprises a processing field and a processing formula; acquiring cell information matched with the member identification code and the processing field from all the basic information; and inputting the cell information corresponding to the member identification code and the data to be processed into the processing formula to obtain the first data to be imported, so that the data rules (processing fields and processing formulas) corresponding to the first attribute cell are automatically obtained, the cell information matched with the processing fields is obtained, and the first data to be imported corresponding to the first attribute cell is obtained through the processing formula.

And S60, correspondingly importing each member identification code, each first data column cell and each first data to be imported into the database table in a column type storage mode, and completing the first import request.

Understandably, the Column-based storage (Column-based) manner is relative to the line-based storage, in a database based on Column-based storage, data is stored according to Column-based logical storage units, data in one Column exists in a continuous storage form in a storage medium, each member identification code, each first data Column cell and each first data to be imported are associated and imported into the database table, and thus, each Column is stored separately, data is an index, and the sequence of headers of tables to be imported is not required to be considered, so that the Column-based storage manner is adopted for storage, the flexibility of importing data is increased, and the Column-based storage manner is not limited to a specified number of table columns.

The method and the device realize the purpose that the document to be imported and the total project information in the first import request are obtained; the document to be imported comprises a table to be imported; the total project information comprises a total project name and an import table name; acquiring a first data row cell array and a first data column cell array in the table to be imported by an Apache poi technology, and creating a database table corresponding to the import table name in a database; inquiring an item basic table matched with the total item name from the database, and acquiring a member list matched with the content of each first data row cell from the item basic table; identifying a first type matched with the content of each first data column cell through a cell data type model, acquiring a data rule matched with each first type, and acquiring a first attribute cell in the table to be imported; executing pre-import data processing operation on each first attribute cell by using an Apache poi technology to obtain first to-be-imported data corresponding to each first attribute cell; the pre-import data processing operation refers to verifying the first attribute cell according to the first type corresponding to the first attribute cell to obtain data to be processed, and then obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule and the member list; the member identification codes, the first data column cells and the first data to be imported are correspondingly imported into the database table according to a column type storage mode, so that the import of the electronic form without fixing the header is realized, the limitation of the position and the content of the header is broken, the form can be freely designed along with the increase/deletion of the items or members (roles), the effect of importing the electronic form into the database table without the limitation of the header is achieved, the column type storage mode is adopted for storage, the foundation without the limitation of the header is provided for the subsequent modification of the same database table, the flexibility of data import is improved, the workload of secondary development due to non-uniform headers is saved, the cost is greatly reduced, and the data import efficiency is improved.

In an embodiment, after the step S60, that is, after the first import request is completed, the method includes:

s601, receiving a second import request, and acquiring the import document and the total project information added in the second import request; the add import document includes an add import table.

Understandably, after the first import request is completed, when the database table needs to be added, triggering the second import request, where the import request includes the add import document and the total item information, the add import document is a document that needs to be imported into the database table, the add import document includes the add import table, and the add import table is a data table that needs to be imported into the database table.

S602, acquiring a second row cell array and a second data column cell array in the added import table through an Apache poi technology, and simultaneously querying a database table corresponding to the import table name from the database; the second row of cell arrays includes at least one second row of cells and the second data column of cell arrays includes at least one second data column of cells.

Understandably, the Apache poi technology is a technology of a cross-platform Java API (Java Application Programming Interface) written in Java, and is mainly used for reading and writing documents of office software, and the Apache poi technology identifies file type information of the documents to be imported, where the file type information includes a file format and a file version, and the file format is a file type of the office software of the documents to be imported, such as: excel file type, Word file type, Powerpoint file type, etc., wherein the file version is a version in a file format, such as: version 95, version 07, etc.; inquiring Apache poi configuration components matched with the file type information from a configuration component management center, wherein the configuration component management center is a management center for managing all Apache poi configuration components, the configuration component management center comprises all Apache poi configuration components, the Apache poi configuration components are environment components configured according to file types of different office software, when a file of one office software needs to be operated, the Apache poi configuration components corresponding to the file are operated, so that a second row cell array and a second data column cell array in the added import table are obtained, the second row cell array is a cell set from a cell corresponding to a preset column of a first row in the added import table to a cell with an empty next cell in the row, and the second data column cell array is an open cell corresponding to a preset row of the first column in the added import table A set of cells between cells beginning with the next cell in the column being empty.

And querying the database table after creation corresponding to the import table name in the database.

S603, inquiring an item basic table matched with the total item name from a database, and acquiring a member list matched with the content of each second row of cells from the item basic table.

Understandably, the database includes all the item basic tables corresponding to the total items one by one, the item basic table is a table of all basic information in one total item, for example, the item basic table includes members, gender, role, item proportion, and the like, the item basic table matched with the total item name is inquired from the database, the content of the second row of cells can be set according to requirements, for example, the content of the second row of cells is an item manager group, a quality inspection group, and the like, the member list matched with the content of each second row of cells is obtained, the member list includes the member identification code matched with the content of the second row of cells and the basic information, the member identification code is a unique identification code assigned to each member, and the basic information is information of a basic class related to the member, the matching process may be exact matching or fuzzy matching, for example, the content of the second row of cells is "quality inspection group", and a member list with the role of "quality inspector" is obtained.

S604, identifying a second type matched with the content of each second data column cell through the cell data type model, acquiring a data rule matched with each second type, and acquiring a second attribute cell in the addition import table; the second attribute cell refers to a cell corresponding to both one of the second row cells and one of the second data column cells.

Understandably, the cell data type model identifies the second type corresponding to each of the second data column cells, the cell data type model is a trained neural network model, the cell data type model enables automatic identification of the second type matching the second data column cells, the second type is a data type identified according to contents of the second data column cells, the second type includes a value type, a character type, a decimal point type, and the like, the data rule is a rule defined according to the second type, and one of the second types is associated with one of the data rules, for example: the content in the second data column unit cell is 'XX project progress responsibility', the characteristics of 'XX project', 'progress' and 'responsibility' are identified, the second type is determined to be a character type, the data rule corresponding to the second type is that after each member adds responsibility content corresponding to the progress after aiming at the responsibility in the XX project, the second type matched with the content of one second data column unit cell is identified, and the data type of the second attribute unit cell in the same row as the second data column unit cell is indicated to be the second type.

The second attribute cell is a cell corresponding to a column which is the same as one second row header cell and a row which is the same as one second data column cell, and the second type may be the same as the first type or not.

S605, performing data processing operation before new import on each second attribute cell through an Apache poi technology to obtain second data to be imported corresponding to each second attribute cell; the data processing operation before new import refers to verifying the second attribute cell according to the second type corresponding to the second attribute cell to obtain added processing data, and then obtaining the second data to be imported according to the added processing data corresponding to the second attribute cell, the data rule and the member list.

Understandably, by using the Apache poi technology, performing pre-import data processing operation on each second attribute cell to obtain second data to be imported corresponding to the second attribute cell, where the pre-import data processing operation refers to verifying the second attribute cell according to the second type corresponding to the second attribute cell, that is, according to the second type corresponding to the second data column cell corresponding to the second attribute cell, and the verification processing process is to convert the content in the second attribute cell into data conforming to the second type, determine the verified data as the data to be processed, and then according to the second type, may determine the data rule corresponding to the second attribute cell, where the data rule includes a processing field and a processing formula, and the list member corresponding to the second attribute cell is the second data line corresponding to the second attribute cell And the member list matched with the content of the cell acquires cell information matched with the member identification code and the processing field from the item basic table according to the to-be-processed data, the data rule and the member list corresponding to the second attribute cell, wherein the basic information comprises a plurality of cell information, the cell information corresponding to the member identification code and the to-be-processed input processing formula are input into the processing formula, and the second to-be-imported data is acquired.

And S606, adding each second data column unit cell to the first empty field of the first column in the database table in a column type storage mode, and correspondingly inserting all the second data to be imported into the database table according to the member list matched with the content of each second row unit cell and all the second data to be imported.

Understandably, through the column type storage mode, each second data column unit cell is added at the tail of a first empty field of a first column in the original database table, namely, at the tail of a non-empty field of the first column, and a member list matched with the content of each second row unit cell and all second data to be imported are correspondingly inserted into the original database table, so that the second import request is completed, and the operation of adding data in the original database table is realized.

The method and the device realize the purpose that the import documents and the total project information are added in the second import request; acquiring a second row cell array and a second data column cell array in the added import table by using an Apache poi technology, and simultaneously querying a database table corresponding to the import table name from the database; inquiring an item basic table matched with the total item name from a database, and acquiring a member list matched with the content of each second row of cells from the item basic table; identifying a second type matched with the content of each second data column cell through the cell data type model, acquiring a data rule matched with each second type, and acquiring a second attribute cell in the addition import table; executing new pre-import data processing operation on each second attribute cell by using an Apache poi technology to obtain second data to be imported corresponding to each second attribute cell; according to a column type storage mode, each second data column unit cell is added to a first empty field of a first column in the database table, and the second data columns are correspondingly inserted into the database table according to a member list matched with the content of each second row unit cell and all the second data to be imported, so that the limitation of a history header is avoided under the condition of subsequently adding data, the database table import without header limitation can be realized, the data import flexibility and efficiency are improved, the workload of secondary development due to header non-uniformity is saved, the cost is greatly reduced, and the data import efficiency is improved.

In an embodiment, after the step S60, that is, after the completing the first import request, the method further includes:

and S70, receiving the export request, and acquiring the user export list and the import table name in the export request.

Understandably, when a user needs to check data imported into the database table, the export request is triggered, the export request comprises the user export list, and the user export list comprises a list of members needing to be exported from the database table.

S80, acquiring a member identification code matched with the user export list from the database table corresponding to the import table name, and determining the acquired member identification code as an export identification code.

Understandably, searching a member identification code which is completely matched with a member identification code in the user export list from the database table corresponding to the import table name, and marking the searched member identification code as the export identification code, wherein the export identification code is the member identification code of data needing to be exported.

S90, screening out data related to the export identification code from the database table, obtaining a line export table through row-column conversion, and exporting an export document in an Excel format through an Apache poi technology.

Understandably, screening out data associated with the export identification code from the database table, converting the data stored in a column form into a line form to obtain the line export table, creating a blank document in an Excel format through the Apache poi technology, writing the line export table into the blank document to obtain the export document, and displaying the export document to a user.

The invention realizes the purpose of obtaining the user export list and the import table name in the export request; acquiring a member identification code matched with the user export list from the database table, and determining the member identification code as an export identification code; the data relevant to the export identification code are screened out from the database table, a row-type export table is obtained through row-column conversion, and an export document in an Excel format is exported through an Apache poi technology, so that the aim that the database table stored in a row-type mode is converted into the export document in the Excel format through the Apache poi technology is achieved, a user can conveniently check the data in the database table, the experience satisfaction degree of the user is improved, and the convenience of data export is improved.

In an embodiment, a database table importing apparatus is provided, where the database table importing apparatus corresponds to the database table importing method in the foregoing embodiment one to one. As shown in fig. 7, the database table importing apparatus includes a receiving module 11, an obtaining module 12, a querying module 13, an identifying module 14, an executing module 15, and an importing module 16. The functional modules are explained in detail as follows:

the receiving module 11 is configured to receive a first import request, and acquire a document to be imported and total project information in the first import request; the document to be imported comprises a table to be imported; the total project information comprises a total project name and an import table name;

an obtaining module 12, configured to obtain, through an Apache poi technology, a first data row cell array and a first data column cell array in the table to be imported, and at the same time, create a database table corresponding to the import table name in a database; the first data row cell array comprises at least one first data row cell, and the first data column cell array comprises at least one first data column cell;

the query module 13 is configured to query an item base table matched with the total item name from the database, and obtain a member list matched with the content of each first data row cell from the item base table; the member list comprises member identification codes and basic information associated with the member identification codes;

the identification module 14 is configured to identify, through a cell data type model, a first type that matches the content of each first data column cell, obtain a data rule that matches each first type, and obtain a first attribute cell in the to-be-imported table; the first attribute cell is a cell corresponding to both one of the first data row cells and one of the first data column cells;

an execution module 15, configured to perform pre-import data processing operation on each first attribute cell through an Apache poi technique, to obtain first to-be-imported data corresponding to each first attribute cell; the pre-import data processing operation refers to verifying the first attribute cell according to the first type corresponding to the first attribute cell to obtain data to be processed, and then obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule and the member list;

and the importing module 16 is configured to import, in a columnar storage manner, each member identification code, each first data column cell, and each first data to be imported into the database table correspondingly, so as to complete the first import request.

The specific definition of the database table importing device can be referred to the definition of the database table importing method in the foregoing, and is not described in detail here. The modules in the database table importing device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a database table import method.

In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the database table importing method in the above embodiments when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the database table importing method in the above-described embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims

1. A database table importing method, comprising:

2. The database table import method of claim 1, wherein said completing the first import request comprises:

receiving a second import request, and acquiring the import document and the total project information added in the second import request; the add import document comprises an add import table;

acquiring a second row cell array and a second data column cell array in the added import table by using an Apache poi technology, and simultaneously querying a database table corresponding to the import table name from the database; the second row of cell arrays comprises at least one second row of cells and the second data column of cell arrays comprises at least one second data column of cells;

inquiring an item basic table matched with the total item name from a database, and acquiring a member list matched with the content of each second row of cells from the item basic table;

identifying a second type matched with the content of each second data column cell through the cell data type model, acquiring a data rule matched with each second type, and acquiring a second attribute cell in the addition import table; the second attribute cell is a cell corresponding to both one of the second row cells and one of the second data column cells;

executing new pre-import data processing operation on each second attribute cell by using an Apache poi technology to obtain second data to be imported corresponding to each second attribute cell; the data processing operation before new import refers to verifying the second attribute cell according to the second type corresponding to the second attribute cell to obtain added processing data, and then obtaining the second data to be imported according to the added processing data corresponding to the second attribute cell, the data rule and the member list;

and according to a column type storage mode, adding each second data column unit cell in a first empty field of a first column in the database table, and correspondingly inserting all the second data to be imported into the database table according to a member list matched with the content of each second row unit cell.

3. The database table import method of claim 1, wherein after the completing the first import request, further comprising:

receiving an export request, and acquiring a user export list and the import table name in the export request;

acquiring a member identification code matched with the user export list from the database table corresponding to the import table name, and determining the acquired member identification code as an export identification code;

and screening out data associated with the export identification code from the database table, obtaining a line export table through row-column conversion, and exporting an export document in an Excel format through an Apache poi technology.

4. The database table importing method of claim 1, wherein the obtaining the first data row cell array and the first data column cell array in the table to be imported by an Apache poi technique comprises:

identifying the file type information of the document to be imported;

acquiring an Apache poi configuration component matched with the file type information from a configuration component management center, and operating the Apache poi configuration component;

and acquiring the first data row cell array and the first data column cell array in the table to be imported.

5. The database table import method of claim 1, wherein the identifying, by the cell data type model, a first type that matches the contents of each of the first data column cells comprises:

inputting the contents of the first data column cells into the cell data type model; the cell data type model is a neural network model based on a Word2Vec model;

extracting data type features through the cell data type model;

and identifying the first type corresponding to the first data column unit cell according to the data type characteristics.

6. The database table importing method of claim 1, wherein the checking the first attribute cell according to the first type corresponding to the first attribute cell to obtain data to be processed includes:

detecting whether the content in the first attribute cell meets the first type of condition corresponding to the first attribute cell;

if the content in the first attribute cell is detected to be not in accordance with the first type of condition, converting the content in the first attribute cell through a regular expression technology in the Apache poi technology to obtain the data to be processed in accordance with the first type of condition;

and if the content in the first attribute cell is detected to meet the condition of the first type, determining the content in the first attribute cell as the data to be processed.

7. The database table importing method of claim 1, wherein the obtaining the first data to be imported according to the data to be processed corresponding to the first attribute cell, the data rule and the member list comprises:

acquiring the data rule matched with the first type corresponding to the first attribute cell; the data rule comprises a processing field and a processing formula;

acquiring cell information matched with the member identification code and the processing field from all the basic information;

and inputting the cell information corresponding to the member identification code and the data to be processed into the processing formula to obtain the first data to be imported.

8. A database table importing apparatus, comprising:

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the database table importing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the database table importing method according to any one of claims 1 to 7.