CN114691602A

CN114691602A - Data import method and device, computer equipment and storage medium

Info

Publication number: CN114691602A
Application number: CN202011564864.8A
Authority: CN
Inventors: 罗晨; 苑立英; 杨李; 顾树华; 李峥; 王文聪
Original assignee: China Life Reinsurance Co ltd; China Reinsurance Group Co ltd
Current assignee: China Life Reinsurance Co ltd; China Reinsurance Group Co ltd
Priority date: 2020-12-25
Filing date: 2020-12-25
Publication date: 2022-07-01

Abstract

The application relates to a data import method, a data import device, computer equipment and a storage medium. The method comprises the following steps: responding to the uploaded file list, and reading and displaying the characteristic information of each file in the file list in a classified mode according to different types of files in the file list; responding to a command for generating an import task, and generating a corresponding file import task for a target file, wherein the target file is determined based on the characteristic information label of each file; and executing the file import task, reading the data column attribute information in the target file corresponding to the file import task, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relation between the preset data column attribute information and the attribute field of the data table. By adopting the method, files in various different formats can be uniformly imported into the database.

Description

Data import method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a data importing method and apparatus, a computer device, and a storage medium.

Background

With the development of information technology, more and more scenes are available for data exchange in a file form, and when data exchange is performed among different systems, file data needs to be formatted and then uniformly imported into a database, so that standardized data can be conveniently inquired and obtained in the database for statistical analysis. In order to ensure the smooth transmission and entry of the file between the systems of the two parties, the format type, the coding rule, the field content and the like of the file need to be agreed in advance before the two parties exchange data, and the receiver receives and analyzes the file data with the specified format, the coding and the field sequence according to the agreed entry rule.

However, since the formats of the files are various and the sizes of the files are different, the database with preset entry rules cannot support various types of file data, cannot uniformly process various file formats, and if dirty data or abnormal data (for example, multiple or few rows of data) exists in the files, all data import fails.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a data importing method, apparatus, computer device and storage medium for solving the above technical problems.

A method of data import, the method comprising:

responding to the uploaded file list, and reading and displaying the characteristic information of each file in the file list in a classified mode according to different types of files in the file list;

responding to a command for generating an import task, and generating a corresponding file import task for a target file, wherein the target file is determined based on the characteristic information labels of the files;

and executing the file import task, reading data column attribute information in a target file corresponding to the file import task, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to the corresponding relation between preset data column attribute information and the attribute field of the data table.

In one embodiment, the reading and displaying feature information of each file in the file list in a classified manner according to different types of files in the file list in response to the uploaded file list includes:

responding to a file list uploaded by a user, and reading the file format of each file in the file list;

if the file with the file format being the compressed file format exists in the file list, decompressing the file with the compressed file format to obtain a file list with the decompressed file format;

and reading and displaying the characteristic information of each file in the file list of the decompressed file format in a classified mode.

In one embodiment, after the generating an import task instruction and generating a corresponding file import task for the target file, the method further includes:

generating a data table attribute label according to the acquired data table attribute demand information;

matching the data table attribute tag with the file import task to obtain a file import task carrying the data table attribute tag as a file import task to be processed;

generating and displaying a file import task list to be processed according to a task identification sequence carried by the file import task to be processed;

the executing the file import task, reading data column attribute information in a target file corresponding to the file import task, and importing data in each data column of the target file into a database data table according to a corresponding relationship between preset data column attribute information and data table attribute fields, wherein the data column attribute information comprises:

in the file import task list to be processed, sequentially executing the file import tasks to be processed according to the task identification sequence;

and for each file import task to be processed, reading data column attribute information in a target file corresponding to the file import task to be processed, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to a corresponding relation between preset data column attribute information and data table attribute fields.

In one embodiment, the reading, for each to-be-processed file import task, data column attribute information in a target file corresponding to the to-be-processed file import task, and importing, according to a preset correspondence between the data column attribute information and a data table attribute field, data in each data column of the target file into an attribute field corresponding to a data table of a database includes:

for each file import task to be processed, performing label configuration check on the file import task to be processed;

and if the attribute tag of the data table matched with the file import task to be processed meets a preset tag configuration rule, reading the attribute information of the data column in the target file corresponding to the file import task to be processed, and importing the data in each data column of the target file into the attribute field corresponding to the data table of the database according to the corresponding relation between the preset attribute information of the data column and the attribute field of the data table.

In one embodiment, the executing the file import task, reading data column attribute information in a target file corresponding to the file import task, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to a corresponding relationship between preset data column attribute information and a data table attribute field includes:

executing the file import task, and preprocessing data contained in each data column in the target file to obtain preprocessed standard data;

reading attribute information of each data column in the target file, and calling a corresponding relation between the attribute information of the data column pre-stored in a database and attribute fields of a data table according to the attribute information of each data column;

and importing the standard data corresponding to the attribute information of each data column into the attribute field of the data table corresponding to the database according to the corresponding relation.

In one embodiment, the executing the file importing task and preprocessing data included in each data column in the target file to obtain preprocessed standard data includes:

responding to the determined target correction type and the corresponding target correction rule, and performing data correction on the data in the target file to obtain first target file data after data correction;

and responding to the determined data cleaning rule, performing data cleaning on the first target file data to obtain second target file data, and taking the second target file data as the preprocessed standard data.

In one embodiment, the reading attribute information of each data column in the target file, and calling, according to the attribute information of each data column, a corresponding relationship between the attribute information of the data column pre-stored in the database and the attribute field of the data table includes:

reading and displaying attribute information of each data column in the target file;

responding to the determined data import rule and a target data table, and configuring the corresponding relation between the data column attribute information of the target file and the data table attribute field of the target data table according to the data import rule;

and checking the corresponding relation, and if the corresponding relation meets a preset checking rule, calling the corresponding relation after the checking is passed, and importing the corresponding relation as a standard.

In one embodiment, the method further comprises:

and if the corresponding relation does not meet the preset check rule, responding to a received user configuration modification instruction, and modifying the corresponding relation to enable the corresponding relation to meet the preset check rule.

A data import apparatus, the apparatus comprising:

the reading display module is used for responding to the uploaded file list, and reading and displaying the characteristic information of each file in the file list in a classified mode according to different types of files in the file list;

the task generating module is used for responding to a task generating and importing instruction and generating a corresponding file importing task for a target file, wherein the target file is determined based on the characteristic information labels of the files;

and the execution import module is used for executing the file import task, reading the data column attribute information in the target file corresponding to the file import task, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relation between the preset data column attribute information and the attribute field of the data table.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

and executing the file import task, reading data column attribute information in a target file corresponding to the file import task, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to the corresponding relationship between preset data column attribute information and data table attribute fields.

The data import method, the data import device, the computer equipment and the storage medium respond to the uploaded file list, and read and display the characteristic information of each file in the file list in a classified mode according to different types of files in the file list; responding to a command for generating an import task, and generating a corresponding file import task for a target file, wherein the target file is determined based on the characteristic information labels of the files; and executing the file import task, reading data column attribute information in a target file corresponding to the file import task, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to the corresponding relationship between preset data column attribute information and data table attribute fields. By adopting the method, the file data of different types and different formats can be uniformly imported into the database.

Drawings

FIG. 1 is a diagram of an application environment of a data import method in one embodiment;

FIG. 2 is a flowchart illustrating a data import method according to an embodiment;

FIG. 3 is a diagram of a file list display interface uploaded in one embodiment;

FIG. 4 is a flowchart illustrating the steps of decompressing compressed files in a file list in one embodiment;

FIG. 5 is a schematic diagram illustrating a process for generating a file import task list in one embodiment;

FIG. 6 is a diagram illustrating a display interface for a file import task list in one embodiment;

FIG. 7 is a flowchart illustrating a process of sequentially handling import of file import tasks in one embodiment;

FIG. 8 is a flowchart illustrating the step of checking attribute tags of a file import task configuration in one embodiment;

FIG. 9 is a flowchart illustrating a method for performing file import tasks, according to one embodiment;

FIG. 10 is a diagram illustrating a display interface showing the correspondence between attribute information for a data column and attribute fields of a data table, according to one embodiment;

FIG. 11 is a schematic flow chart diagram illustrating the steps for preprocessing different types of data in one embodiment;

FIG. 12 is a diagram illustrating an import rule configuration display interface for a file import task, according to an embodiment;

FIG. 13 is a flowchart illustrating steps of configuring and verifying correspondence of file import tasks in one embodiment;

FIG. 14 is a block diagram showing the configuration of a data import apparatus according to one embodiment;

FIG. 15 is a diagram showing an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The data importing method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The server 104 responds to a file list uploaded by a user through the terminal 102, and according to different types of files in the file list, reads and displays the characteristic information of each file in the file list in a classified mode; then, responding to a task generating and importing instruction triggered by a user, and generating a corresponding file importing task from a target file, wherein the target file is determined based on the characteristic information labels of the files; and executing the file import task, reading the data column attribute information in the target file corresponding to the file import task, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relation between the preset data column attribute information and the attribute field of the data table.

The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, and tablet computers, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers. Alternatively, the database server may be a server independent from the server 104, and therefore, after the server 104 completes the target file reading and mapping configuration operation (obtains the corresponding relationship between the attribute information of the data column and the attribute field of the data table), the server 104 may execute a file import task to import the target file into the database server.

In one embodiment, as shown in fig. 2, a data importing method is provided, which is described by taking the method as an example applied to the server 104 in fig. 1, and includes the following steps:

step 201, responding to the uploaded file list, and according to different types of files in the file list, reading and displaying feature information of each file in the file list in a classified manner.

In implementation, a user accesses a web server through a client device, candidate import files are uploaded to the web server, and the web server (referred to as a computer device for short) responds to a file list (including a set of all candidate import files) uploaded by the user, reads the candidate import files in types according to different types of files in the file list, and displays characteristic information of each file in the file list. As shown in fig. 3, in the import file list, a user may check a target file, where different types of files may be, but are not limited to, an Excel type, a txt type, an xml type, and the like, and feature information of each file in the file list includes: file name, file size, file path, file type, etc.

Step 202, in response to the instruction of generating the import task, generating a corresponding file import task for the target file, where the target file is determined based on the characteristic information labels of the files.

In implementation, the target file is determined based on the characteristic information of each file, that is, a user performs selection on the target file according to the characteristic information of each file in a file list displayed on a display interface of the client, determines the target file, and clicks a task button for generating an import file according to the determined target file, and the background of the computer device responds to an instruction for generating the import task to generate a corresponding file import task for the target file.

Step 203, executing the file import task, reading the data column attribute information in the target file corresponding to the file import task, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relationship between the preset data column attribute information and the data table attribute field.

In implementation, the computer device executes a file import task, reads data column attribute information in a target file corresponding to the file import task, and imports data in each data column of the target file into an attribute field corresponding to a database data table according to a preset corresponding relationship between the data column attribute information and an attribute field of the data table.

In the data importing method, in response to an uploaded file list, according to different types of files in the file list, characteristic information of each file in the file list is read and displayed in a classified mode; responding to a command for generating an import task, and generating a corresponding file import task for a target file, wherein the target file is determined based on the characteristic information label of each file; and executing the file import task, reading the data column attribute information in the target file corresponding to the file import task, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relation between the preset data column attribute information and the attribute field of the data table. By adopting the method, the file data with different types and different formats can be uniformly imported into the database.

In one embodiment, as shown in fig. 4, the specific processing procedure of step 201 is as follows:

in step 2011, the file format of each file in the file list is read in response to the file list uploaded by the user.

In implementation, after the user uploads the candidate import file through the client device, the computer device reads file format information of each file in the file list in response to the file list uploaded by the user, that is, determines a file format (or called a file type) of the file according to a file name field suffix.

Step 2012, if the file list has a file with a file format of a compressed file format, the file with the compressed file format is decompressed to obtain a file list with a decompressed file format.

In implementation, if a file with a compressed file format exists in the file list, the computer device first decompresses the file with the compressed file format to obtain a file list with a decompressed file format.

And 2013, reading and displaying the characteristic information of each file in the file list in the decompressed file format in a classified mode.

In implementation, the computer device performs classified reading (for example, reading a txt-format file as one category and reading an Excel-format file as one category) on all the decompressed files in the file list, reads characteristic information of each file, and then displays the characteristic information in a display interface in a manner of category arrangement.

In this embodiment, whether a compressed format file exists in the file list is determined by preliminarily determining the file format type uploaded by the user, and then decompression processing is performed on the compressed format file, so that the files in the file list can be previewed, and the characteristic information of each file can be displayed on the display interface.

In one embodiment, as shown in fig. 5, after step 202, the method further comprises:

step 202(a), generating a data table attribute label according to the acquired data table attribute demand information.

In implementation, the computer device generates a data table attribute tag according to the acquired data table attribute demand information. Specifically, different data tables are stored in the database, each data table is configured according to specific service requirements, when attribute information in the data table needs to be processed and supplemented for the task of this time, the attribute requirement information of the data table input by a user is obtained, and a corresponding data table attribute tag is generated according to the attribute requirement information of the data table.

Specifically, in one import task, in addition to importing the data content in the target file into the data table of the database, the data table needs to have other attribute data (for example, a bill attribute, a contract attribute, a client name attribute, and the like) besides the target file data according to the business requirements of the company, and then corresponding data table attribute tags are generated for the attributes.

And step 202(b), matching the attribute tag of the data table with the file import task to obtain the file import task carrying the attribute tag of the data table, and using the file import task as the file import task to be processed.

In implementation, the computer device matches the generated attribute tag of the data table with a file import task (i.e. the attribute tag is marked on the corresponding file import task), and then obtains the file import task carrying the attribute tag of the data table, which is used as a file import task to be processed.

And step 202(c), generating and displaying a file import task list to be processed according to a task identification sequence carried by the file import task to be processed.

In implementation, the computer device sorts according to the task identifier carried by each to-be-processed file import task, generates a file import task list, and displays the file import task list on a display interface. Specifically, as shown in fig. 6, in the file import task list, a task identifier (also referred to as a task ID) may be character string data randomly matched by a computer device, and the computer device may further perform process monitoring on the file import task, so that information such as a task type, a task start time, an execution duration, a task state, and a current node, and information displayed by a specific file import task list are also displayed in the file import task list, which is not limited in the embodiment of the present application.

On this basis, as shown in fig. 7, the processing procedure of step 203 is:

and 701, sequentially executing the file import tasks to be processed in the file import task list to be processed according to the task identification sequence.

In implementation, the computer device obtains a list of file import tasks to be processed, and executes each file import task to be processed in sequence according to the sequence of task identifiers in the list.

Step 702, for each file import task to be processed, reading the data column attribute information in the target file corresponding to the file import task to be processed, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relationship between the preset data column attribute information and the attribute field of the data table.

In implementation, for each file import task to be processed, reading data column attribute information in a target file corresponding to the file import task to be processed, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to a corresponding relationship between preset data column attribute information and a data table attribute field.

In one embodiment, as shown in fig. 8, the specific processing procedure of step 2032 is as follows:

step 801, for each file import task to be processed, performing tag configuration check on the file import task to be processed.

In implementation, the computer device performs tag configuration check on each to-be-processed file import task, and specifically performs tag configuration check on each to-be-processed file import task according to a preset configuration rule of a data table attribute tag that needs to be matched with each file import task.

Step 802, if the attribute tag of the data table matched with the file import task to be processed meets the preset tag configuration rule, reading the attribute information of the data column in the target file corresponding to the file import task to be processed, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relationship between the preset attribute information of the data column and the attribute field of the data table.

In implementation, if the attribute tag of the data table matched with the file import task to be processed meets a preset tag configuration rule, the computer device imports data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relationship between the preset data column attribute information and the attribute field of the data table.

Specifically, after detecting that the file import task satisfies the preset tag configuration rule, the computer device imports the target file corresponding to the file import task into the database data table according to the file import process described in step 702, which is not described in detail in this embodiment of the present application.

In one embodiment, as shown in fig. 9, the specific processing procedure of step 203 is as follows:

step 2031, executing a file import task, and preprocessing the data contained in each data column in the target file to obtain preprocessed standard data.

In implementation, when the computer device executes a file import task, data of each data column of a target file in the file import task needs to be preprocessed to ensure that file data of different types and different formats have a uniform specification, so that preprocessed standard data are obtained.

Step 2032, reading the attribute information of each data column in the target file, and calling the corresponding relationship between the attribute information of the data column pre-stored in the database and the attribute field of the data table according to the attribute information of each data column.

In implementation, the computer device reads the attribute information of each data column in the target file, and calls the corresponding relation between the attribute information of the data column and the attribute field of the data table, which are stored in the database in advance, according to the attribute information of each data column. Specifically, as shown in fig. 10, according to the attribute "policy number" of the data column in the target file, it is found that the corresponding relationship between the attribute information of the data column and the fields of the data table is: policy number 1.

Step 2033, according to the corresponding relationship, importing the standard data corresponding to the attribute information of each data column into the attribute field of the data table corresponding to the database.

In implementation, the computer device imports the preprocessed standard data corresponding to the attribute information of each data column in the target file into the attribute fields of the data table corresponding to the database according to the corresponding relationship.

In one embodiment, as shown in fig. 11, the specific processing procedure of step 2031 is as follows:

step 1101, in response to the determined target correction type and the corresponding target correction rule, performing data correction on data in the target file to obtain first target file data after data correction.

In implementation, a user determines a target correction type, specifically a data correction rule configuration interface, through a click trigger instruction performed on a client device, as shown in fig. 12, a configuration entry, a corresponding check item, and a configuration type are displayed on the data correction rule configuration interface, and for a target correction type of a user click mark, a computer device performs data correction on data in a target file in response to a target correction rule of which the determined target correction type corresponds to the target correction type, so as to obtain first target file data after data correction. Wherein the target correction type includes: general revision types (e.g., data font type, font size, font color, etc.), date-specific revision types (e.g., display format of date data), and data-specific revision types (e.g., span revision of data).

Step 1102, in response to the determined data cleaning rule, performing data cleaning on the first target file data to obtain second target file data, and taking the second target file data as the preprocessed standard data.

In implementation, a user triggers to enter a data cleaning interface by clicking (or called selecting) an instruction, a data cleaning rule is determined (configured) on the cleaning interface, data cleaning is carried out on the first target file data in response to the determined data cleaning rule, missing data or error data in the first target file data are removed, second target file data are obtained, and the second target file data are used as standard data after preprocessing.

Optionally, if there is also data with non-obvious abnormality (i.e., abnormal data that cannot be corrected and cleaned) in the preprocessed second target file data, when the second target file data corresponding to each data column attribute information is imported into the data table attribute field corresponding to the database, the non-obvious abnormal data is thrown out (i.e., the mark is skipped over first) to import the rest of normal data, and then the thrown-out non-obvious abnormal data is separately stored in a certain error data file according to the mark, so that the error data is analyzed at a later stage.

In one embodiment, as shown in fig. 13, the specific processing procedure of step 2032 is as follows:

step 1301, reading and displaying the attribute information of each data column in the target file.

In implementation, the computer device reads and displays the attribute information of each data column in the target file.

Step 1302, in response to the determined data import rule and the target data table, configuring a corresponding relationship between the data column attribute information of the target file and the data table attribute field of the target data table according to the data import rule.

In the implementation, a user determines a data import rule through checking, and the computer device configures the corresponding relation between the data column attribute information of the target file and the data table attribute field of the target data table according to the determined data import rule and the target data table and according to the data import rule.

And step 1303, checking the corresponding relation, and calling the corresponding relation after the verification is passed as a standard to import the corresponding relation if the corresponding relation meets a preset verification rule.

In implementation, the computer device checks the corresponding relationship between the obtained data column attribute information and the data table attribute field, and if the corresponding relationship meets a preset check rule, calls the corresponding relationship after the check is passed, and uses the corresponding relationship as a standard import corresponding relationship for importing the data in the target file. Meanwhile, when data import is carried out according to a standard import corresponding relation, in a corresponding relation display interface, a certain piece of data in the target file is successfully imported according to the corresponding relation, the data record is displayed as a green background, if the certain piece of data in the target file does not completely correspond to the attribute field of the data table (or a completely matched corresponding relation does not exist), the data record is displayed as a yellow background in the corresponding relation display interface, and if the certain piece of data in the target file does not completely correspond to the attribute field of the data table, the data record is displayed as a red background.

Optionally, if the corresponding relationship does not satisfy the preset check rule, the corresponding relationship is modified in response to the received user configuration modification instruction, so that the corresponding relationship satisfies the preset check rule.

Specifically, if the correspondence does not satisfy the preset check rule, the user may modify and configure the correspondence between the data array attribute information and the data table attribute field by dragging a table or other trigger operations (i.e., the user modifies the configuration instruction) in the data import rule configuration interface, and at the same time, the user triggers the computer device to identify and end the modification and configuration task by clicking a "configuration complete" button.

It should be understood that although the various steps in the flowcharts of fig. 2, 4-5, 7-9, 11, 13 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 4-5, 7-9, 11, 13 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least some of the other steps.

In one embodiment, as shown in fig. 14, there is provided a data import apparatus 1400, including: a read display module 1410, a generate task module 1420, and an execute import module 1430, wherein:

the reading display module 1410 is configured to respond to the uploaded file list, and according to different types of files in the file list, classify and read and display feature information of each file in the file list;

a task generating module 1420, configured to generate a corresponding file import task for the target file in response to the instruction for generating the import task, where the target file is determined based on the feature information labels of the files;

and the execution import module 1430 is configured to execute the file import task, read data column attribute information in the target file corresponding to the file import task, and import data in each data column of the target file into an attribute field corresponding to the database data table according to a preset correspondence between the data column attribute information and the attribute field of the data table.

In one embodiment, the reading display module 1410 is specifically configured to, in response to a file list uploaded by a user, read a file format of each file in the file list;

and reading and displaying the characteristic information of each file in the file list in the decompressed file format according to the type.

In one embodiment, the apparatus 1400 further comprises:

matching the attribute tag of the data table with a file import task to obtain the file import task carrying the attribute tag of the data table as a file import task to be processed;

generating and displaying a file import task list to be processed according to a task identification sequence carried by a file import task to be processed;

the execution import module 1430 is specifically configured to sequentially execute the file import tasks to be processed in the file import task list to be processed according to the task identification sequence;

and for each file import task to be processed, reading the data column attribute information in the target file corresponding to the file import task to be processed, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relation between the preset data column attribute information and the attribute field of the data table.

In an embodiment, the import execution module 1430 is specifically configured to, for each file import task to be processed, perform tag configuration check on the file import task to be processed;

and if the attribute tags of the data table matched with the file import task to be processed meet the preset tag configuration rule, reading the attribute information of the data columns in the target file corresponding to the file import task to be processed, and importing the data in each data column of the target file into the attribute fields corresponding to the data table of the database according to the corresponding relation between the preset attribute information of the data columns and the attribute fields of the data table.

In an embodiment, the import execution module 1430 is further specifically configured to execute a file import task, and pre-process data included in each data column in the target file to obtain pre-processed standard data;

reading attribute information of each data column in a target file, and calling a corresponding relation between the attribute information of the data column pre-stored in a database and attribute fields of a data table according to the attribute information of each data column;

In an embodiment, the execution import module 1430 is further specifically configured to perform data modification on data in the target file in response to the determined target modification type and the corresponding target modification rule, so as to obtain first target file data after data modification;

In one embodiment, the execution import module 1430 is further specifically configured to read and display attribute information of each data column in the target file;

responding to the determined data import rule and the target data table, and configuring the corresponding relation between the data column attribute information of the target file and the data table attribute field of the target data table according to the data import rule;

For specific limitations of the data importing apparatus, reference may be made to the above limitations of the data importing method, and details are not described here. The modules in the data importing apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 15. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing file data to be imported. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data import method.

Those skilled in the art will appreciate that the architecture shown in fig. 15 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:

in response to the uploaded file list, according to different types of files in the file list, reading and displaying the characteristic information of each file in the file list in a classified mode;

responding to a command for generating an import task, and generating a corresponding file import task for a target file, wherein the target file is determined based on the characteristic information label of each file;

and executing the file import task, reading the data column attribute information in the target file corresponding to the file import task, and importing the data in each data column of the target file into the attribute field corresponding to the database data table according to the corresponding relation between the preset data column attribute information and the attribute field of the data table.

In one embodiment, the processor, when executing the computer program, further performs the steps of:

and generating and displaying a file import task list to be processed according to a task identification sequence carried by the file import task to be processed.

in a file import task list to be processed, sequentially executing file import tasks to be processed according to a task identification sequence;

executing a file import task, and preprocessing data contained in each data column in a target file to obtain preprocessed standard data;

responding to the determined target correction type and the corresponding target correction rule, and performing data correction on data in the target file to obtain first target file data after data correction;

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

and executing a file import task, reading data column attribute information in a target file corresponding to the file import task, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to the corresponding relationship between the preset data column attribute information and the attribute field of the data table.

In one embodiment, the computer program when executed by the processor further performs the steps of:

and for each file import task to be processed, reading data column attribute information in a target file corresponding to the file import task to be processed, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to the corresponding relation between the preset data column attribute information and the data table attribute field.

responding to the determined target correction type and the corresponding target correction rule, and performing data correction on data in the target file to obtain first target file data after the data correction;

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method for importing data, the method comprising:

2. The method according to claim 1, wherein the step of reading and displaying the characteristic information of each file in the file list according to different types of files in the file list in a classified manner in response to the uploaded file list comprises:

3. The method of claim 1, wherein after the generating a corresponding file import task for the target file in response to the generate import task instruction, the method further comprises:

and for each file import task to be processed, reading data column attribute information in a target file corresponding to the file import task to be processed, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to a corresponding relation between preset data column attribute information and a data table attribute field.

4. The method according to claim 3, wherein for each of the to-be-processed file import tasks, reading data column attribute information in a target file corresponding to the to-be-processed file import task, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to a preset correspondence between the data column attribute information and a data table attribute field, includes:

5. The method according to claim 1, wherein the executing the file import task, reading data column attribute information in a target file corresponding to the file import task, and importing data in each data column of the target file into an attribute field corresponding to a database data table according to a preset correspondence between the data column attribute information and an attribute field of the data table includes:

6. The method according to claim 5, wherein the executing the file import task to preprocess data included in each data column in the target file to obtain preprocessed standard data includes:

7. The method according to claim 5, wherein the reading of the attribute information of each data column in the target file, and the calling of the corresponding relationship between the attribute information of the data column pre-stored in the database and the attribute field of the data table according to the attribute information of each data column comprises:

8. The method of claim 7, further comprising:

9. A data import apparatus, characterized in that the apparatus comprises:

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.