CN110069449B - File processing method, device, computer equipment and storage medium - Google Patents
File processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110069449B CN110069449B CN201910212526.9A CN201910212526A CN110069449B CN 110069449 B CN110069449 B CN 110069449B CN 201910212526 A CN201910212526 A CN 201910212526A CN 110069449 B CN110069449 B CN 110069449B
- Authority
- CN
- China
- Prior art keywords
- file
- data
- target
- template
- name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present application relates to the field of data processing, and in particular, to a method, an apparatus, a computer device, and a storage medium for processing a file. The method comprises the following steps: receiving the file and file type information uploaded by the terminal through the data management system; extracting a first file name from a target file, judging whether the first file name is matched with file type information, and uniformly configuring the first file name by the data management system; when the first file name is matched with the file type information, acquiring a data reading strategy corresponding to the file type information; reading target data from the target file according to the data reading strategy, and performing data verification on the target data; and when the data check is successful, adding the target data into a data table corresponding to the file type information. By adopting the method, the data processing efficiency of the target file can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and apparatus for processing a file, a computer device, and a storage medium.
Background
With the continuous development of the information age, at present, all companies and institutions have special data management systems for storing and managing working data, and a large amount of data generated in daily work needs to be uploaded to the data management systems.
At present, when analyzing and processing data files such as excel uploaded by a user, data in the uploaded file needs to be read row by row and identified, a large amount of resource space needs to be wasted, and the data processing speed is slow because the data content contained in the uploaded file cannot be predicted.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a file processing method, apparatus, computer device, and storage medium that improve the data processing efficiency of uploading files.
A method of object file processing, the method comprising:
receiving target files and file type information uploaded by a terminal through a data management system;
extracting a first file name from the target file, and judging whether the first file name is matched with the file type information or not, wherein the first file name is uniformly configured by the data management system;
when the first file name is matched with the file type information, acquiring a data reading strategy corresponding to the first service identifier;
reading target data from the target file according to the data reading strategy, and performing data verification on the target data;
And when the data check is successful, adding the target data into a data table corresponding to the file type information.
In one embodiment, before the receiving terminal uploads the target file and the file type information through the data management system, the method includes:
receiving a template downloading request sent by a terminal, wherein the template downloading request carries a first service identifier and a first user identifier;
searching a file template corresponding to the first service identifier;
generating a second file name according to the first service identifier and the first user identifier;
and generating a file template according to the file template and the second file name, and returning the file template to the terminal.
In one embodiment, generating a file template from the file template and the second file name includes:
searching a department number corresponding to the first user identifier;
searching a limitation data class corresponding to the department number;
and processing the file template according to the limiting data class, and generating a file template according to the processed file template and the second file name.
In one embodiment, adding the target data to the data table corresponding to the file type information includes:
Extracting a second service identifier and a second user identifier from the file type information;
searching a service data table corresponding to the second service identifier;
acquiring the uploading time of the target file;
and adding the target data to a position corresponding to the second user identifier and the uploading time in the service data table.
In one embodiment, reading target data from the target file according to the data reading policy, and performing data verification on the target data, including:
acquiring file data storage information from the data reading strategy;
searching a first file field with a data type of a system type from the target data according to the file data storage information, and reading a first file field name and a first file field value of the first file field;
searching system data corresponding to the field name of the first file;
data matching is carried out on the first file field values and the system data one by one;
and when all the first file field values are successfully matched with the system data, the data verification is successful.
In one embodiment, reading target data from the target file according to the data reading policy, and performing data verification on the target data, including:
Acquiring file data storage information from the data reading strategy;
searching a second file field with a data type of non-heavy type from the target data according to the file data storage information, and reading a second file field value of the second file field;
judging whether repeated numerical values exist in the second file field value or not;
and when the fact that the repeated numerical value does not exist in the second file field value is judged, the data verification is successful.
An object file processing apparatus, the apparatus comprising:
the data receiving module is used for receiving the target file and file type information uploaded by the terminal through the data management system;
the file matching module is used for extracting a first file name from the target file, judging whether the first file name is matched with the file type information or not, and uniformly configuring the first file name by the data management system;
the rule acquisition module is used for acquiring a data reading strategy corresponding to the first service identifier when the first file name is judged to be matched with the file type information;
the data reading module is used for reading target data from the target file according to the data reading strategy and carrying out data verification on the target data;
And the data adding module is used for adding the target data into a data table corresponding to the file type information when the data verification is successful.
In one embodiment, the apparatus further comprises:
the request receiving module is used for receiving a template downloading request sent by the terminal, wherein the template downloading request carries a first service identifier and a first user identifier;
the template searching module is used for searching a file template corresponding to the first service identifier;
the file name generation module is used for generating a second file name according to the first service identifier and the first user identifier;
and the template generation module is used for generating a file template according to the file template and the second file name and returning the file template to the terminal.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method described above when the processor executes the computer program.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method.
According to the file processing method, the device, the computer equipment and the storage medium, the file type information is uploaded at the same time when the file is uploaded, so that whether the uploaded file is correct or not can be judged according to the file type information, the phenomenon that error data is continuously read when the error file is uploaded is avoided, when the uploaded file is accurate, a preset data reading strategy corresponding to the file type information is obtained, each data storage information in the uploaded file can be determined according to the data reading strategy, the uploaded data can be directly read from the target file according to the data reading strategy, and processing operations such as identification and integration are not needed to the data content in the file, so that the data reading efficiency of the uploaded file is effectively improved.
Drawings
FIG. 1 is an application scenario diagram of a file processing method in one embodiment;
FIG. 2 is a flow diagram of a method of processing files in one embodiment;
FIG. 3 is a flow chart of a method of processing files according to another embodiment;
FIG. 4 is a block diagram of a file processing device in one embodiment;
fig. 5 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The method for processing the uploading file can be applied to an application environment shown in figure 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 sends the target file uploaded by the user and the file type information selected by the user to the server 104 through the data management system, and the server 104 extracts a first file name from the received target file, and judges whether the first file name is matched with the file type information or not; when the first file name is judged to be matched with the file type information, a data reading strategy which corresponds to the file type information and contains file data storage information is obtained; reading target data from the target file according to the data reading strategy, and performing data verification on the target data; and when the data check is successful, adding the target data into a data table corresponding to the file type information.
The terminal 102 may be, but not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, and the server 104 may be implemented by a stand-alone server or a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, a file processing method is provided, and the method is applied to the server 104 in fig. 1 for illustration, and the method includes the following steps:
step 210, receiving the target file and file type information uploaded by the terminal through the data management system.
The data management system is used for uniformly managing and maintaining business data of users, a large amount of business data can be generated in daily business work, and when newly added business data or business data are changed, the users can upload the newly added or changed business data to the data management system in the form of data files, for example, the business data can be uploaded at regular time every day, and the business data can also be uploaded when data updating demands exist.
The user can upload the target file through a data management system on the terminal, the user can select the service type of the service to which the uploaded file belongs through a system page when uploading the target file, the terminal generates file type information according to the service type selected by the user, the target file and the file type information are sent to the server, and the server receives the target file and the file type information sent by the terminal.
Specifically, the file type of the target file may be excel, txt, csv or the like. The file type information may include service type information, such as a service identifier for identifying a service type, and may also include user information, such as a user identifier, a department identifier of a department to which the user belongs, and so on. The terminal can acquire the user information of the current login user and generate file type information according to the service type information and the user information.
And 220, extracting a first file name from the target file, and judging whether the first file name is matched with the file type information, wherein the first file name is uniformly configured by the data management system.
The server extracts a first file name of the target file. The file name of the target file uploaded by the data management system of the terminal is generated by the data management system according to unified configuration of the service type selected by the user, the user information and the like, the file name at least comprises a service identifier of the service type, and the service identifier is arranged at a preset character position in the file name.
The server extracts the service identifier from the preset character position of the first file name, and extracts the service type information from the received file type, where the service type information may be the service identifier, or may be other information indicating the service type, for example, other types of information such as a service name, a service code, and the like,
If the information is other information representing the service type, the mapping relationship between the information and the service identifier may be stored in the server in advance.
The server judges whether the extracted service identification is matched with the service type information or not, so that whether the target file of the user target is correct or not is judged according to the file name of the target file and the service type of the file selected by the user, and unnecessary data reading work is avoided when the user target is wrong. When the server determines that the first file name is not matched with the file type information, the server generates a file target error prompt and returns the file target error prompt to the terminal, and the terminal displays the file target error prompt so as to enable a user to re-target the file or modify the service type of the file and ensure the accuracy of the target file.
In step 230, when the first file name matches the file type information, a data reading policy corresponding to the file type information is obtained.
When the server determines that the first file name is matched with the file type information, the server extracts the service type from the file type information and searches a data reading strategy corresponding to the service type. In this embodiment, the data storage information in the target file corresponding to each service type is determined in advance, and the server sets the data reading policy of the data in the file according to the data storage information of the file.
The file data storage information may include field names of file fields included in the file, location information of columns where the respective file fields are located, data format information of file field values, data titles of the file, and the like. The location information may include a column number of a column in which the field is located, an arrangement order of the field, etc., and the data format information may include a data length of the field value, a data type such as text, a character string, etc.
The data reading policy may determine a reading order of each file field according to the file data storage information, for example, an arrangement order of file field columns may be used as a reading order, and determine a data verification method of each file field, for example, data verification is performed according to field content, field attribute, data format of a field, and the like corresponding to each file field, where the data reading policy may further include other logic policies related to reading.
And step 240, reading target data from the target file according to the data reading strategy, and performing data verification on the target data.
The server reads the file field values line by line from the target file according to the data reading strategy to form a plurality of data records, performs data verification on the file field values according to the data format of each file field, and judges whether the data length, the data type and other formats of the file field values accord with preset data formats. If the field values of the files in each record accord with the preset data format, the data record is successfully checked; if the file field value which does not accord with the preset data format exists in the data record, the verification of the record data fails.
Further, when the server finishes reading all the data records, an uploading error report can be generated according to the data records with failed verification, the uploading error report can include the line number of the error record in the target file, and the error reasons such as 'the length is not in accordance with the requirement', 'the data type is wrong', and the server returns the generated uploading error report to the terminal.
And 250, adding the target data to a data table corresponding to the file type information when the data verification is successful.
When the field values of all the file fields in the target file are successfully checked, the server searches a data table corresponding to the file type information. The data table may be stored in a database of a data management platform of the server, and classified according to file types, and the server generates a plurality of data tables according to classification categories of the files, for example, the data tables may be marked according to service types, or after classified according to service types, the data tables may be subdivided according to user information, for example, departments, groups, and the like where the users are located. After the server searches the corresponding data table from the database of the data management platform, the read data record is added into the corresponding data table, so that the data reading work of the target file is completed.
In another embodiment, when a data record of verification failure exists in the target file, the server adds the data record of verification success to a corresponding data table, and records information such as the name of the target file, the line number of the error record, the uploading time and the like, so that the modified target file number is uploaded at the terminal, the server can read the modified data record from the re-uploaded file according to the recorded information of the error data record and re-verify the modified data record, and when the verification is successful, the modified data record is added to a data position corresponding to the uploading time in the corresponding data table.
In the method, when a user uploads the file, the file type information is required to be uploaded at the same time, the server can judge whether the file uploaded by the user is correct according to the received file type information, the user is prevented from uploading wrong file and continuously reading wrong data, after the fact that the file is uploaded accurately is judged, a preset data reading strategy corresponding to the file type information is obtained, each data storage information in the uploaded file can be predicted according to the data reading strategy, target data can be read from the uploaded file directly according to the data reading strategy, and processing operations such as identification, integration and the like are not required to be carried out on the data content in the file, so that the data reading efficiency of the uploaded target file is effectively improved.
In one embodiment, as shown in fig. 3, before the server receives the target file and the file type information uploaded by the terminal through the data management system, the method may further include the following steps:
step 202, receiving a template downloading request sent by a terminal, wherein the template downloading request carries a first service identifier and a first user identifier.
Before uploading a file to a terminal, a user needs to use a file template specified by a data management system and enter data by using the file template. The data management system of the terminal can provide a template downloading function, a user selects the service type of a file template to be downloaded on a page, generates a template downloading request carrying a first service identifier of the service type selected by the user and a user identifier of a login user after selecting a file downloading function button, the terminal sends the template downloading request to a server, and the server reads the first service identifier and the first user identifier from the request after receiving the template downloading request.
Step 204, searching a file template corresponding to the first service identifier.
A plurality of file templates are preset in the server, each service type corresponds to one file template, and data fields needing to be input, arrangement information of the fields, titles of the files and the like are defined in the file templates. Furthermore, the data format of each field column can be set in the file template, and the format of the input data can be automatically checked when the user inputs the data, so that the data format checking work during data reading is saved. The file template is marked according to the service identification corresponding to the service type, and the server searches the file template corresponding to the first service identification.
And 206, generating a second file name according to the first service identifier and the first user identifier.
In this embodiment, the naming rule of the file template is also preset, and the naming rule includes two parts of fixed information of the service identifier and the user identifier, and the two parts of information cannot be changed after the user downloads the file, so that the user can conveniently and automatically check whether the uploaded file is correct or not according to the file name when uploading the file. The server generates a second file name according to the first service identifier and the first user identifier.
And step 208, generating a file template according to the file template and the second file name, and returning the file template to the terminal.
And the server names the file templates according to the second file names, and returns the named file templates to the terminal. In other embodiments, the file template may also be stored on the terminal, with the file template being obtained directly from the terminal locally.
In one embodiment, generating the file template from the file template and the second file name may include: searching for a department number corresponding to the first user identifier; searching a limitation data class corresponding to the department number; and processing the file template according to the limiting data class, and generating the file template according to the processed file template and the second file name.
The data requirements of different departments may differ for the same service, and not all data fields are required by some departments, which are constraint-like data. The file template corresponding to each business department contains data fields required by all departments, and for departments with special data requirements, limitation type data in the file template needs to be processed.
The server stores the limitation class data of each department of each service in advance, and can store the mapping relation between the department number and each limitation field in the limitation class data. The server searches the department number corresponding to the first user identification, searches the limitation class data corresponding to the department number, obtains each limitation field in the limitation class data, performs hidden invisible setting or deletion setting on the corresponding field column in the file template according to the limitation field, and names the processed file template according to the second file name.
In this embodiment, the same file template can be implemented to meet the service data requirements of multiple service departments by setting constraint class data of each department.
In one embodiment, adding the target data to the data table corresponding to the file type information may include: extracting a second service identifier and a second user identifier from the file type information; searching a service data table corresponding to the second service identifier; acquiring uploading time of a target file; and adding the target data to a position corresponding to the second user identification and the uploading time in the service data table.
The server extracts a second service identifier and a second user identifier of the uploaded file from the file type information. It should be noted that the "second" is used herein to distinguish from the "first" in the above embodiment, where the target file uploaded by the user may be a file in which data is recorded through a file template downloaded in the above embodiment, or may be a file in which data is recorded through another file template, that is, the first service identifier may be the same as or different from the second service identifier, and the first user identifier and the second user identifier may be different from each other.
The data belonging to the same service type is summarized into a data table, the server searches the data table corresponding to the first service identifier, the data table can be divided into a plurality of data sub-tables according to the service department, the data sub-tables can be arranged according to the user identifier, the data of the same user are stored together, and the data are ordered according to the time sequence. The server searches a data sub-table corresponding to the department to which the user identifier belongs, the data in the data sub-table are marked according to the uploading time of the file, and the uploading data are marked according to the uploading time. And the server acquires the uploading time of the uploading file, if the uploading time is the first uploading time of the file, the server adds the uploading data to the position corresponding to the uploading time in the data sub-table.
In one embodiment, reading target data from a target file according to a data reading policy, and performing data verification on the target data may include: acquiring file data storage information from a data reading strategy; searching a first file field with a data type of a system type from target data according to file data storage information, and reading a first file field name and a first file field value of the first file field; searching system data corresponding to the field name of the first file; carrying out data matching on the first file field values and system data one by one; and when all the first file field values are successfully matched with the system data, the data verification is successful.
The data reading policy includes file data storage information, where the file data storage information may include a field name of a file field included in the file, location information of a column where each file field is located, data format information of a file field value, such as a data type and a data header of the file. The server acquires file data storage information in the data reading strategy.
The data types of the file fields in the uploaded files are classified according to the numerical characteristics of the field values and can be classified into common types, system types, non-heavy types and the like, and if no special numerical requirement exists, the data types can be classified into the common types. The server configures the data types of the data fields in the file template in advance, acquires the data types of the file fields, and judges whether the data types of the system types exist or not. The data of the system type is fixed system data in the data management system, such as product names, product codes and the like, and because the products in the business are determined, the product names and the product codes are also determined, the data are stored in the database in advance, and when the data are updated, such as when the products are newly added, the corresponding system data in the database are correspondingly updated.
The server searches whether a file field with the data type being the system type exists in target data of the file according to file data storage information, when a first file field with the system type exists, the server acquires a first file field name and a first file field value of the file field, searches system data corresponding to the first file field name, matches the first file field value with the searched system data one by one, if consistent system data is found, the matching is successful, and when all the file field values are successfully matched, the file field is successfully checked. For example, a product name field exists in the uploading file, the server compares the corresponding field value with the product name data in the system data, and when the product name exceeds the range of the system data, the user is indicated to possibly input data in error, and the data verification fails.
In one embodiment, reading target data from a target file according to a data reading policy, and performing data verification on the target data may include: acquiring file data storage information from a data reading strategy; searching a second file field with a data type of non-heavy type from the target data according to the file data storage information, and reading a second file field value of the second file field; judging whether repeated numerical values exist in the field value of the second file; and when the fact that the repeated numerical value does not exist in the second file field value is judged, the data verification is successful.
The data reading policy includes file data storage information, where the file data storage information may include a field name of a file field included in the file, location information of a column where each file field is located, data format information of a file field value, such as a data type and a data header of the file. The server acquires file data storage information in the data reading strategy.
The field values of the non-heavy file fields should not have mutually repeated numerical values, such as the mobile phone number field of the client, and the situation that two mobile phone numbers of the client are identical cannot exist in the data input by the user. The server acquires the data type of each file field and judges whether the data type of non-heavy type exists. The server searches whether a file field with a data type of non-heavy type exists in target data of the file according to file data storage information, when a second file field with a non-heavy type exists, the server acquires all second file field values in the second file field, compares the second file field values with each other, judges whether repeated field values exist, and if the repeated field values do not exist in the second file field values, the second file field data is successfully checked; otherwise, the verification of the file field data fails.
It should be noted that the same file field may correspond to multiple data types, for example, may be a system type or a non-duplicate type, and in this case, it is necessary to perform verification according to a data verification method corresponding to the system type or a data verification method corresponding to the non-duplicate type.
It should be understood that, although the steps in the flowcharts of fig. 2-3 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2-3 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily occur sequentially, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or steps.
In one embodiment, as shown in fig. 4, there is provided a target file processing apparatus including: a data receiving module 410, a file matching module 420, a rule obtaining module 430, a data reading module 440, and a data adding module 450, wherein:
The data receiving module 410 is configured to receive the target file and the file type information uploaded by the terminal through the data management system.
And the file matching module 420 is configured to extract a first file name from the target file, and determine whether the first file name is matched with the file type information, where the first file name is uniformly configured by the data management system.
The rule obtaining module 430 is configured to obtain a data reading policy corresponding to the first service identifier when it is determined that the first file name matches the file type information.
The data reading module 440 is configured to read target data from a target file according to a data reading policy, and perform data verification on the target data.
And the data adding module 450 is configured to add the target data to the data table corresponding to the file type information when the data verification is successful.
In one embodiment, the file processing apparatus may further include:
the request receiving module is used for receiving a template downloading request sent by the terminal, wherein the template downloading request carries a first service identifier and a first user identifier.
And the template searching module is used for searching the file template corresponding to the first service identifier.
And the file name generation module is used for generating a second file name according to the first service identifier and the first user identifier.
And the template generating module is used for generating a file template according to the file template and the second file name and returning the file template to the terminal.
In one embodiment, the template generation module may further include:
and the number searching unit is used for searching the department number corresponding to the first user identifier.
And the restriction class searching unit is used for searching restriction data classes corresponding to the department numbers.
And the template processing unit is used for processing the file template according to the limiting data class and generating the file template according to the processed file template and the second file name.
In one embodiment, the data addition module 450 may include:
and the identifier extraction unit is used for extracting the second service identifier and the second user identifier from the file type information.
And the data table searching unit is used for searching the service data table corresponding to the second service identifier.
And the time acquisition unit is used for acquiring the uploading time of the target file.
And the adding unit is used for adding the target data to the position corresponding to the second user identifier and the uploading time in the service data table.
In one embodiment, the data reading module 440 may include:
an information reading unit for acquiring file data storage information from the data reading strategy;
And the field data acquisition unit is used for searching a first file field with the data type of the system type from the target data according to the file data storage information, and reading a first file field name and a first file field value of the first file field.
And the system data searching unit is used for searching the system data corresponding to the field name of the first file.
And the data matching unit is used for carrying out data matching on the first file field values and the system data one by one.
And the first verification unit is used for successfully verifying the data when all the first file field values are successfully matched with the system data.
In one embodiment, the data reading module 440 may include:
an information reading unit for acquiring file data storage information from the data reading strategy;
and the field value acquisition unit is used for searching a second file field with a data type of non-heavy type from the target data according to the file data storage information and reading a second file field value of the second file field.
And the non-duplicate judgment unit is used for judging whether duplicate values exist in the field value of the second file.
And the second checking unit is used for checking the data successfully when the repeated numerical value does not exist in the field value of the second file.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing the data related to the uploading file processing. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a method of processing an upload file.
It will be appreciated by those skilled in the art that the structure shown in FIG. 5 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory storing a computer program and a processor that when executing the computer program performs the steps of: receiving a target file and file type information uploaded through a data management system; extracting a first file name from a target file, and judging whether the first file name is matched with file type information or not, wherein the first file name is uniformly configured by the data management system; when the first file name is matched with the file type information, acquiring a data reading strategy corresponding to the file type information; reading target data from the target file according to the data reading strategy, and performing data verification on the target data; and when the data check is successful, adding the target data into a data table corresponding to the file type information.
In one embodiment, the processor when executing the computer program further performs the steps of: receiving a template downloading request sent by a terminal, wherein the template downloading request carries a first service identifier and a first user identifier; searching a file template corresponding to the first service identifier; generating a second file name according to the first service identifier and the first user identifier; and generating a file template according to the file template and the second file name, and returning the file template to the terminal.
In one embodiment, the processor, when executing the computer program, further performs the step of generating a file template from the file template and the second file name: searching for a department number corresponding to the first user identifier; searching a limitation data class corresponding to the department number; and processing the file template according to the limiting data class, and generating the file template according to the processed file template and the second file name.
In one embodiment, the processor, when executing the step of adding the target data to the data table corresponding to the file type information, is further configured to: extracting a second service identifier and a second user identifier from the file type information; searching a service data table corresponding to the second service identifier; acquiring uploading time of a target file; and adding the target data to a position corresponding to the second user identification and the target time in the service data table.
In one embodiment, when the processor executes the computer program to implement the steps of reading the target data from the target file according to the data reading policy, and performing data verification on the target data, the method is further used for: acquiring file data storage information from a data reading strategy; searching a first file field with a data type of a system type from target data according to file data storage information, and reading a first file field name and a first file field value of the first file field; searching system data corresponding to the field name of the first file; carrying out data matching on the first file field values and system data one by one; and when all the first file field values are successfully matched with the system data, the data verification is successful.
In one embodiment, the step of reading the target data from the target file according to the data reading policy and performing data verification on the target data is further performed when the processor executes the computer program: acquiring file data storage information from a data reading strategy; searching a second file field with a data type of non-heavy type from the target data according to the file data storage information, and reading a second file field value of the second file field; judging whether repeated numerical values exist in the field value of the second file; and when the fact that the repeated numerical value does not exist in the second file field value is judged, the data verification is successful.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of: receiving a target file and file type information uploaded through a data management system; extracting a first file name from a target file, and judging whether the first file name is matched with file type information or not, wherein the first file name is uniformly configured by the data management system; when the first file name is matched with the file type information, acquiring a data reading strategy corresponding to the file type information; reading target data from the target file according to the data reading strategy, and performing data verification on the target data; and when the data check is successful, adding the target data into a data table corresponding to the file type information.
In one embodiment, the computer program when executed by the processor further performs the steps of: receiving a template downloading request sent by a terminal, wherein the template downloading request carries a first service identifier and a first user identifier; searching a file template corresponding to the first service identifier; generating a second file name according to the first service identifier and the first user identifier; and generating a file template according to the file template and the second file name, and returning the file template to the terminal.
In one embodiment, the computer program when executed by the processor is further configured to: searching for a department number corresponding to the first user identifier; searching a limitation data class corresponding to the department number; and processing the file template according to the limiting data class, and generating the file template according to the processed file template and the second file name.
In one embodiment, the computer program when executed by the processor performs the step of adding the target data to a data table corresponding to the file type information further for: extracting a second service identifier and a second user identifier from the file type information; searching a service data table corresponding to the second service identifier; acquiring target time of a target file; and adding the target data to a position corresponding to the second user identification and the target time in the service data table.
In one embodiment, the computer program when executed by the processor is further configured to implement the steps of reading target data from the target file according to a data reading policy, and performing data verification on the target data: acquiring file data storage information from a data reading strategy; searching a first file field with a data type of a system type from target data according to file data storage information, and reading a first file field name and a first file field value of the first file field; searching system data corresponding to the field name of the first file; carrying out data matching on the first file field values and system data one by one; and when all the first file field values are successfully matched with the system data, the data verification is successful.
In one embodiment, the computer program when executed by the processor is further configured to implement the steps of reading target data from the target file according to a data reading policy, and performing data verification on the target data: acquiring file data storage information from a data reading strategy; searching a second file field with a data type of non-heavy type from the target data according to the file data storage information, and reading a second file field value of the second file field; judging whether repeated numerical values exist in the field value of the second file; and when the fact that the repeated numerical value does not exist in the second file field value is judged, the data verification is successful.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (10)
1. A method of file processing, the method comprising:
receiving target files and file type information uploaded by a terminal through a data management system; the file type information comprises service type information;
extracting a first file name from the target file, and judging whether the first file name is matched with the file type information or not, wherein the first file name is uniformly configured by the data management system;
When the first file name is matched with the file type information, acquiring a data reading strategy corresponding to the file type information;
reading target data from the target file according to the data reading strategy, and performing data verification on the target data;
when the data verification is successful, adding the target data into a data table corresponding to the file type information;
the reading the target data from the target file according to the data reading strategy, and performing data verification on the target data, including:
acquiring file data storage information from the data reading strategy;
searching a first file field with a data type of a system type from the target data according to the file data storage information, and reading a first file field name and a first file field value of the first file field;
searching system data corresponding to the field name of the first file;
data matching is carried out on the first file field values and the system data one by one;
when all the first file field values are successfully matched with the system data, the data verification is successful;
or alternatively
Acquiring file data storage information from the data reading strategy;
Searching a second file field with a data type of non-heavy type from the target data according to the file data storage information, and reading a second file field value of the second file field;
judging whether repeated numerical values exist in the second file field value or not;
and when the fact that the repeated numerical value does not exist in the second file field value is judged, the data verification is successful.
2. The method according to claim 1, wherein before the receiving terminal uploads the target file and the file type information through the data management system, the method comprises:
receiving a template downloading request sent by a terminal, wherein the template downloading request carries a first service identifier and a first user identifier;
searching a file template corresponding to the first service identifier;
generating a second file name according to the first service identifier and the first user identifier;
and generating a file template according to the file template and the second file name, and returning the file template to the terminal.
3. The method of claim 2, wherein the generating a file template from the file template and the second file name comprises:
searching a department number corresponding to the first user identifier;
Searching a limitation data class corresponding to the department number;
and processing the file template according to the limiting data class, and generating a file template according to the processed file template and the second file name.
4. The method according to claim 1, wherein adding the target data to a data table corresponding to the file type information includes:
extracting a second service identifier and a second user identifier from the file type information;
searching a service data table corresponding to the second service identifier;
acquiring the uploading time of the target file;
and adding the target data to a position corresponding to the second user identifier and the uploading time in the service data table.
5. An object file processing apparatus, characterized in that the apparatus comprises:
the data receiving module is used for receiving the target file and file type information uploaded by the terminal through the data management system; the file type information comprises service type information;
the file matching module is used for extracting a first file name from the target file, judging whether the first file name is matched with the file type information or not, and uniformly configuring the first file name by the data management system;
The rule acquisition module is used for acquiring a data reading strategy corresponding to the file type information when the first file name is judged to be matched with the file type information;
the data reading module is used for reading target data from the target file according to the data reading strategy and carrying out data verification on the target data;
the data adding module is used for adding the target data into a data table corresponding to the file type information when the data verification is successful;
the data reading module is further used for acquiring file data storage information from the data reading strategy; searching a first file field with a data type of a system type from the target data according to the file data storage information, and reading a first file field name and a first file field value of the first file field; searching system data corresponding to the field name of the first file; data matching is carried out on the first file field values and the system data one by one; when all the first file field values are successfully matched with the system data, the data verification is successful; or, the file data storage information is obtained from the data reading strategy; searching a second file field with a data type of non-heavy type from the target data according to the file data storage information, and reading a second file field value of the second file field; judging whether repeated numerical values exist in the second file field value or not; and when the fact that the repeated numerical value does not exist in the second file field value is judged, the data verification is successful.
6. The apparatus of claim 5, wherein the apparatus further comprises:
the request receiving module is used for receiving a template downloading request sent by the terminal, wherein the template downloading request carries a first service identifier and a first user identifier;
the template searching module is used for searching a file template corresponding to the first service identifier;
the file name generation module is used for generating a second file name according to the first service identifier and the first user identifier;
and the template generation module is used for generating a file template according to the file template and the second file name and returning the file template to the terminal.
7. The apparatus of claim 6, the template generation module comprising:
the number searching unit is used for searching department numbers corresponding to the first user identifiers;
a limitation type searching unit, configured to search a limitation data type corresponding to the department number;
and the template processing unit is used for processing the file template according to the limiting data class and generating a file template according to the processed file template and the second file name.
8. The apparatus of claim 5, wherein the data adding module comprises
The identification extraction unit is used for extracting a second service identification and a second user identification from the file type information;
the data table searching unit is used for searching a service data table corresponding to the second service identifier;
the time acquisition unit is used for acquiring the uploading time of the target file;
and the adding unit is used for adding the target data to the position corresponding to the second user identifier and the uploading time in the service data table.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 4 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 4.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910212526.9A CN110069449B (en) | 2019-03-20 | 2019-03-20 | File processing method, device, computer equipment and storage medium |
| PCT/CN2019/117942 WO2020186786A1 (en) | 2019-03-20 | 2019-11-13 | File processing method and apparatus, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910212526.9A CN110069449B (en) | 2019-03-20 | 2019-03-20 | File processing method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110069449A CN110069449A (en) | 2019-07-30 |
| CN110069449B true CN110069449B (en) | 2023-09-26 |
Family
ID=67366412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910212526.9A Active CN110069449B (en) | 2019-03-20 | 2019-03-20 | File processing method, device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110069449B (en) |
| WO (1) | WO2020186786A1 (en) |
Families Citing this family (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110069449B (en) * | 2019-03-20 | 2023-09-26 | 平安科技（深圳）有限公司 | File processing method, device, computer equipment and storage medium |
| CN112395266B (en) * | 2019-08-19 | 2022-12-13 | 中移(苏州)软件技术有限公司 | Data file processing method, first server and computer storage medium |
| CN110806979B (en) * | 2019-10-31 | 2023-04-07 | 望海康信(北京)科技股份公司 | Interface return value checking method, device, equipment and storage medium |
| CN111144334B (en) * | 2019-12-27 | 2023-09-26 | 北京天融信网络安全技术有限公司 | File matching method and device, electronic equipment and storage medium |
| CN111291051B (en) * | 2020-01-22 | 2023-06-16 | 中国民航信息网络股份有限公司 | Freight rate data processing method and device, storage medium and electronic equipment |
| CN111274350B (en) * | 2020-02-03 | 2023-06-23 | 广州极尚网络技术有限公司 | Data processing method, device, computer equipment and storage medium |
| CN111654522B (en) * | 2020-04-27 | 2023-02-03 | 平安证券股份有限公司 | File synchronization method, file synchronization server and storage medium |
| CN111666259B (en) * | 2020-06-06 | 2023-05-23 | 智同道合(苏州)信息技术服务有限公司 | Document management method, management system, readable storage medium, and electronic device |
| CN111782517B (en) * | 2020-06-23 | 2021-06-04 | 浙江保融科技股份有限公司 | Automatic file checking method |
| CN111935546B (en) * | 2020-07-23 | 2022-02-25 | 深圳市鼎盛光电有限公司 | Program importing method and device, video playing equipment and storage medium |
| CN111966636A (en) * | 2020-08-20 | 2020-11-20 | 中国农业银行股份有限公司 | Data file fusion method and device |
| CN112509653A (en) * | 2020-10-29 | 2021-03-16 | 望海康信(北京)科技股份公司 | Medical record data processing method and system, corresponding equipment and storage medium |
| CN112463737A (en) * | 2020-11-17 | 2021-03-09 | 中科金审(北京)科技有限公司 | System and method for rapidly acquiring data aiming at multi-format data intelligent matching template |
| CN112380167A (en) * | 2020-11-17 | 2021-02-19 | 深圳市和讯华谷信息技术有限公司 | Batch data verification method and device, computer equipment and storage medium |
| CN112612505B (en) * | 2020-12-04 | 2024-01-30 | 北京思特奇信息技术股份有限公司 | Universal file processing method and system based on jdk flow processing |
| CN112882995A (en) * | 2021-02-24 | 2021-06-01 | 深圳壹账通智能科技有限公司 | Script automatic generation method and device, electronic equipment and storage medium |
| CN112950370A (en) * | 2021-02-25 | 2021-06-11 | 未鲲(上海)科技服务有限公司 | Service processing method, device, equipment and storage medium |
| CN112988868B (en) * | 2021-03-17 | 2022-12-13 | 首钢京唐钢铁联合有限责任公司 | Steel laboratory data processing method and system |
| CN113297222A (en) * | 2021-05-25 | 2021-08-24 | 北京京东振世信息技术有限公司 | Report data acquisition method and device, electronic equipment and storage medium |
| CN113438290A (en) * | 2021-06-22 | 2021-09-24 | 康键信息技术（深圳）有限公司 | Method, device and equipment for detecting repeated request and storage medium |
| CN113505580A (en) * | 2021-07-26 | 2021-10-15 | 京东科技控股股份有限公司 | Method and device for analyzing table file |
| CN113590554A (en) * | 2021-08-10 | 2021-11-02 | 中国工商银行股份有限公司 | File processing method and device, electronic equipment and storage medium |
| CN113672567B (en) * | 2021-10-22 | 2023-07-07 | 南京数睿数据科技有限公司 | File construction method, device, electronic equipment and computer readable medium |
| CN114449063B (en) * | 2022-01-17 | 2024-03-26 | 蚂蚁区块链科技(上海)有限公司 | Message processing method, device and equipment |
| CN114401147B (en) * | 2022-01-20 | 2024-02-20 | 山西晟视汇智科技有限公司 | New energy power station communication message comparison method and system based on abstract algorithm |
| CN114896468B (en) * | 2022-04-24 | 2024-02-02 | 北京月新时代科技股份有限公司 | File type matching method and data intelligent input method based on neural network |
| CN114595199B (en) * | 2022-05-10 | 2022-09-02 | 太平金融科技服务（上海）有限公司 | File analysis method and device, computer equipment and storage medium |
| CN116644035B (en) * | 2023-07-21 | 2024-01-26 | 中邮消费金融有限公司 | File batch warehousing method, device, equipment and storage medium |
| CN117376033A (en) * | 2023-12-06 | 2024-01-09 | 浙江网商银行股份有限公司 | File processing method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339438A (en) * | 2016-08-22 | 2017-01-18 | 深圳鼎智通讯股份有限公司 | Internal storage display method in file manager |
| CN107330096A (en) * | 2017-07-05 | 2017-11-07 | 恒生电子股份有限公司 | Document handling method and device, computer-readable recording medium and electronic equipment |
| CN107908396A (en) * | 2017-10-17 | 2018-04-13 | 广东广业开元科技有限公司 | A kind of chart data Online Processing System and method based on browser |
| CN108804336A (en) * | 2018-06-15 | 2018-11-13 | 深圳壹账通智能科技有限公司 | Interactive system test method, device, computer equipment and storage medium |
| CN109325030A (en) * | 2018-09-03 | 2019-02-12 | 平安国际融资租赁有限公司 | Message processing method, device, computer equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08314784A (en) * | 1995-05-19 | 1996-11-29 | Toshiba Corp | File management device |
| CN101231647B (en) * | 2007-01-26 | 2011-02-02 | 鸿富锦精密工业（深圳）有限公司 | File management system and method |
| CN110069449B (en) * | 2019-03-20 | 2023-09-26 | 平安科技（深圳）有限公司 | File processing method, device, computer equipment and storage medium |
-
2019
- 2019-03-20 CN CN201910212526.9A patent/CN110069449B/en active Active
- 2019-11-13 WO PCT/CN2019/117942 patent/WO2020186786A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339438A (en) * | 2016-08-22 | 2017-01-18 | 深圳鼎智通讯股份有限公司 | Internal storage display method in file manager |
| CN107330096A (en) * | 2017-07-05 | 2017-11-07 | 恒生电子股份有限公司 | Document handling method and device, computer-readable recording medium and electronic equipment |
| CN107908396A (en) * | 2017-10-17 | 2018-04-13 | 广东广业开元科技有限公司 | A kind of chart data Online Processing System and method based on browser |
| CN108804336A (en) * | 2018-06-15 | 2018-11-13 | 深圳壹账通智能科技有限公司 | Interactive system test method, device, computer equipment and storage medium |
| CN109325030A (en) * | 2018-09-03 | 2019-02-12 | 平安国际融资租赁有限公司 | Message processing method, device, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 大数据时代下的动态可配置数据采集系统的研究与设计;丁俊;郑辉;;计算机应用与软件(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020186786A1 (en) | 2020-09-24 |
| CN110069449A (en) | 2019-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110069449B (en) | File processing method, device, computer equipment and storage medium | |
| CN110276002B (en) | Search application data processing method and device, computer equipment and storage medium | |
| CN108874926B (en) | Mass data query method, device, computer equipment and storage medium | |
| CN109408746B (en) | Image information query method, image information query device, computer equipment and storage medium | |
| CN109558404B (en) | Data storage method, device, computer equipment and storage medium | |
| CN109446068B (en) | Interface test method, device, computer equipment and storage medium | |
| CN110290212B (en) | Service call recording method, device, computer equipment and storage medium | |
| CN111177302B (en) | Service bill processing method, device, computer equipment and storage medium | |
| CN111027295B (en) | Data processing method, device, computer equipment and storage medium | |
| CN111090788B (en) | json file comparison method and device, storage medium and computer equipment | |
| CN110287192B (en) | Search application data processing method and device, computer equipment and storage medium | |
| CN109902272B (en) | Electronic form data processing method, electronic form data processing device, computer equipment and storage medium | |
| CN110321284B (en) | Test data entry method, device, computer equipment and storage medium | |
| CN109508964B (en) | Project establishing method, device, computer equipment and storage medium | |
| CN111191079B (en) | Document content acquisition method, device, equipment and storage medium | |
| CN109885299B (en) | Template processing method, device, computer equipment and storage medium in model development | |
| CN108509424A (en) | Institutional information processing method, device, computer equipment and storage medium | |
| CN112286934A (en) | Database table importing method, device, equipment and medium | |
| CN109783457B (en) | CGI interface management method, device, computer equipment and storage medium | |
| CN111209061B (en) | User information filling method, device, computer equipment and storage medium | |
| CN110659297A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN109325042B (en) | Processing template acquisition method, form processing method, device, equipment and medium | |
| CN112328631A (en) | Production fault analysis method and device, electronic equipment and storage medium | |
| CN111083054B (en) | Route configuration processing method and device, computer equipment and storage medium | |
| CN111984659B (en) | Data updating method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |