CN112395266B - Data file processing method, first server and computer storage medium - Google Patents
Data file processing method, first server and computer storage medium Download PDFInfo
- Publication number
- CN112395266B CN112395266B CN201910765633.4A CN201910765633A CN112395266B CN 112395266 B CN112395266 B CN 112395266B CN 201910765633 A CN201910765633 A CN 201910765633A CN 112395266 B CN112395266 B CN 112395266B
- Authority
- CN
- China
- Prior art keywords
- file
- data
- data file
- server
- interface machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a data file processing method, which comprises the following steps: acquiring a first data file to be uploaded in real time; distributing a first data file to an interface machine corresponding to the first data file; checking the first data file through the interface machine; and if the first data file is successfully verified, sending a verification success receipt to the second server, and storing the first data file in a database of the first server. The embodiment of the invention also discloses a first server and a computer storage medium. The invention solves the problem of low acquisition efficiency when the data files are acquired by the related technology, improves the acquisition efficiency of the data files and ensures the accuracy of the acquired data files.
Description
Technical Field
The present invention relates to data file processing in the field of computer technologies, and in particular, to a data file processing method, a first server, and a computer storage medium.
Background
At present, a great deal of data files generated by the office of each branch company in a large-scale website and an enterprise every day have more and more remarkable importance on how to collect a great deal of data in a centralized way. In the relative technology, a periodic scheduling-configuration-data storage (ETL) technology is generally adopted to realize data acquisition. However, in the data acquisition method in the relative technology, the data file of the next period can be acquired only after the data file corresponding to the current period is successfully verified and uploaded to a data warehouse tool (Hive), so that the acquisition efficiency is low.
Disclosure of Invention
In order to solve the above problem, embodiments of the present invention desirably provide a data file processing method, a first server, and a computer storage medium, so as to solve the problem in the related art that the acquisition efficiency is low when acquiring a data file, improve the acquisition efficiency of the data file, and ensure the accuracy of the acquired data file.
The technical scheme of the invention is realized as follows:
a method of data file processing, the method comprising:
acquiring a first data file to be uploaded in real time;
distributing the first data file to an interface machine corresponding to the first data file;
verifying the first data file through the interface machine;
and if the first data file is successfully verified, sending a verification success receipt to a second server, and storing the first data file to a database of the first server.
Optionally, the acquiring, in real time, a first data file to be uploaded includes:
collecting a data compression file to be uploaded in real time;
and analyzing the data compression file to obtain a first data file.
Optionally, the checking the first data file by the interface machine includes:
and verifying the first data file through the interface machine based on the verification instruction file of the first data file in the data compression file.
Optionally, the verifying the first data file through the interface machine based on the verification instruction file of the first data file in the data compression file includes:
acquiring file attribute parameters of the first data file and file summary information of the first data file through the interface machine;
detecting whether the file attribute parameters of the first data file are matched with preset file attribute parameters and whether the file abstract information of the first data file is matched with preset file abstract information through the interface machine; wherein the verification specification file of the first data file comprises: the preset file attribute parameters and the preset file summary information are obtained;
and if the file attribute parameters of the first data file are matched with the preset file attribute parameters and the file abstract information of the first data file is matched with the preset file abstract information, determining that the first data file is successfully verified through the interface machine.
Optionally, the allocating the first data file to the interface machine corresponding to the first data file includes:
determining a file type of the first data file;
and distributing the first data file to an interface machine corresponding to the file type of the first data file based on the file type of the first data file.
Optionally, the method further includes:
if the first data file fails to be verified, acquiring first target data which fails to be verified in the first data file;
sending the first target data to the second server;
receiving second target data sent by the second server, and verifying the second target data through the interface machine; the second target data is obtained by revising the first target data by the second server.
Optionally, if the first data file is successfully verified, sending a verification success receipt to the second server, and storing the first data file in the database of the first server includes:
if the first data file is successfully verified, verifying a data field included in the first data file through the interface machine;
and if the data field is verified successfully, sending a verification success receipt to a second server and storing the first data file to the first server database.
Optionally, if the first data file is successfully verified, verifying, by the interface machine, a data field included in the first data file, including:
if the first data file is successfully verified, acquiring field attribute parameters of the data fields and field abstract information of the data fields through the interface machine;
detecting whether the data field attribute parameters are matched with preset field attribute parameters and whether the field abstract information of the data field is matched with preset field abstract information through the interface machine;
and if the field attribute parameters of the data are matched with the preset field attribute parameters and the field abstract information of the data field is matched with the preset field abstract information, determining that the data field is successfully verified through the interface machine and generating a field verification success receipt.
Optionally, the method further includes:
if the field verification of the data field fails, acquiring third target data corresponding to the field with the failed field verification;
sending the third target data to a second server;
receiving fourth target data sent by a second server, and verifying the fourth target data through the interface machine; wherein the fourth target data is obtained by revising the third target data by the second server.
A first server, the first server comprising: a processor, a memory, and a communication bus, comprising:
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is used for executing the program of the data file processing method in the memory so as to realize the following steps:
acquiring a first data file to be uploaded in real time;
distributing the first data file to an interface machine corresponding to the first data file;
verifying the first data file through the interface machine;
and if the first data file is successfully verified, sending a verification success receipt to a second server, and storing the first data file to a database of the first server.
A computer storage medium storing one or more programs executable by one or more processors to implement the steps of the data file processing method described above.
The data file processing method, the first server and the computer storage medium provided by the embodiment of the invention comprise the steps of collecting a first data file to be uploaded in real time; distributing a first data file to an interface machine corresponding to the first data file; the first data file is verified through the interface machine; if the first data file is successfully verified, a verification success receipt is sent to the second server, and the first data file is stored in the database of the first server, so that the first data file is verified based on the first server with the interface machine, the verification success receipt can be returned to the second server after the verification is successful, the next period of data file is collected, and meanwhile, the first data file is uploaded to the database of the first server, and the next period of data file is collected without waiting for the completion of the uploading of the first data file in the relative technology.
Drawings
Fig. 1 is a schematic flow chart of a data file processing method according to various embodiments of the present invention;
FIG. 2 is a schematic flow chart illustrating another data file processing method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart illustrating a further data file processing method according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a data file processing method according to another embodiment of the present invention;
fig. 5 is a schematic structural diagram of a first server according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating another data file processing method according to another embodiment of the present invention;
fig. 7 is a schematic diagram of a file verification process of a data file according to an embodiment of the present invention;
fig. 8 is a schematic view of a field checking process of a data file according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of another first server according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
An embodiment of the present invention provides a data file processing method, which is applied to a first server, and as shown in fig. 1, the method includes the following steps:
In the embodiment of the invention, the first server collects the first data file in real time, wherein the first server is a first server with an interface machine, and the first data can be data files generated by a large website every day, data files generated by a Web system at offices of various branch companies in a large enterprise every day, and the like.
And 102, distributing the first data file to an interface machine corresponding to the first data file.
In the embodiment of the invention, the first server is provided with a plurality of interface machine modules, and the plurality of interface machine modules can process data in parallel aiming at different data types.
Illustratively, the daily sales volume data file of the A department of the X company is distributed to an interface machine A module corresponding to the sales data of the A department, the office public asset data file of the B department of the X company is distributed to an interface machine B module corresponding to the asset data of the B department, and the A module of the interface machine and the B module of the interface machine process the data in real time after receiving the corresponding data.
And 103, checking the first data file through the interface machine.
In the embodiment of the invention, a first server with an interface machine verifies a first data file sent by a second server based on the interface machine specification, a sender and a receiver of the file verify the data based on the agreed specification before the first server receives the data sent by the sender, the interface machine sends the data which is verified successfully to the first server, and the data which is verified unsuccessfully is returned to the sender of the file. After receiving the file which fails to be verified, the sender of the file modifies the data which fails to be verified and then uploads the modified data to the interface machine again, and the interface machine verifies the data again.
And step 104, if the first data file is successfully verified, sending a verification success receipt to the second server, and storing the first data file in a database of the first server.
In the embodiment of the invention, the interface machine module in the first server performs file verification on the first data file which is successfully verified, generates the file verification receipt according to the file verification result of the first data file, and determines whether the verification work of the first data file is finished or not based on the specific content of the file verification receipt result. If the interface machine module determines that the first data file is successfully verified, a first data file verification success receipt is returned to the second server, and the first data file verification is completed. Further, the interface machine receives the verification work of the next group of data files and can load the first data files which are verified. That is to say, in the embodiment of the present invention, the verification and the loading of the data file are operated independently and do not affect each other, the second server does not need to wait for the completion of the loading of the data, and the uploading can be completed as long as the verification is passed, so that the waiting time is saved.
In the embodiment of the invention, after the interface machine of the first server completes the verification work on the first data file, the first server interface machine module sends the result of successful verification to the second server, and the interface machine of the first service uploads the data of successful verification to the first server database. It should be noted that, in the embodiment of the present invention, the first server database includes a distributed cluster database (Hive).
In the embodiment of the invention, the interface machine module of the first server respectively checks different types of data files sent by the sender, extracts the first data file which is checked and loads the first data file into the data memory, that is, extracts the first data file which is successfully checked on the interface machine and successfully checked on the field and loads the first data file into the data memory. The proposal provides a data file processing method, which realizes an independent data checking tool and decouples two processes of loading data of a data checking and data warehousing technology (Extract-Transform-Load, ETL) to a data warehouse tool (Hive); meanwhile, data uploading does not need to follow a strict front-back period relation, data of different time periods in a single system and data of different time periods in the single system can be uploaded in parallel without mutual influence, and the acquisition efficiency is improved.
The data file processing method provided by the embodiment of the invention comprises the steps of collecting a first data file to be uploaded in real time; distributing a first data file to an interface machine corresponding to the first data file; the first data file is verified through the interface machine; if the first data file is successfully verified, a verification success receipt is sent to the second server, and the first data file is stored in the database of the first server, so that the first data file is verified based on the first server with the interface machine, the verification success receipt can be returned to the second server after the verification is successful, the next period of data file is collected, and meanwhile, the first data file is uploaded to the database of the first server, and the next period of data file is collected without waiting for the completion of the uploading of the first data file in the relative technology.
Based on the foregoing embodiments, an embodiment of the present invention provides a data file processing method applied to a first server, and as shown in fig. 2, the method includes the following steps:
In the embodiment of the invention, the first server acquires the compressed packet of the first data file in real time, wherein the compressed packet comprises the data file packet of the first data file and the verification instruction file of the first data file, which are agreed between the second server and the first server.
Illustratively, the verification specification file comprises a designated naming mode, a file format, an interface type, a file packager and the like of the data file.
In the embodiment of the invention, a first server interface machine server with an interface machine decompresses a received compressed file, so that the obtained file comprises a first data file and a first data file verification instruction file; the first server of the interface machine comprises at least one interface machine, the data file type and the interface machine have a corresponding relation, and meanwhile, different types of data files can be decomposed based on different compressed files, that is, the interface machine can distribute the data files to the corresponding interface machine for data processing according to different data types, so that the problem of inconvenient operation caused by data classification again is solved.
Illustratively, the first server interface machine decompresses the data file to obtain a first data file and a first data file verification specification file ending in verf, which is a code for data file decompression as shown below.
-rwx--x—x 1oaadmin sftp 107Sep 12 00:59
bj_a_20180911_OAGW_PC_LOGIN_00.verf
-rwx--x—x 1oaadmin sftp 532697Sep 12 00:59
done_success_bj_a_20180911_OAGW_PC_LOGIN_00_001.csv
Illustratively, after the first server receives a compressed file (zip) to be uploaded sent by the second server, the interface engine of the first server decompresses the first data file, and the first data verification instruction file and the first data file. In the schematic diagram, the data file ending with the verf code is a first data verification description file, and the data file with successfully field verification is a file beginning with the done _ success code.
In this embodiment of the present invention, the step 203 of allocating the first data file to the interface machine corresponding to the first data file includes the following steps:
step 203a, determining the file type of the first data file.
In the embodiment of the invention, a first server with an interface machine receives a data file sent by a second server, wherein the second server carries the file type of the data file in the sent first data file.
Step 203b, based on the file type of the first data file, allocating the first data file to the interface machine corresponding to the file type of the first data file.
In the embodiment of the invention, the first server is provided with a plurality of interface machine modules, each interface machine can correspondingly process one type of data, the plurality of interface machines of the first server are connected in parallel, and the plurality of interface machines of the first server have the capacity of the same step data verification work. Therefore, when the interface machine module of the first server receives a plurality of different types of data at the same time, the different types of data can be processed at the same time.
For example, the interface machine module of the first server allocates the first data file to the interface machine module corresponding to the file type based on the file type of the first data file, and when receiving the first data file, the interface machine module of the first server starts to perform verification processing on the first file data.
And 204, verifying the first data file through the interface machine based on the verification instruction file of the first data file in the data compression file.
In this embodiment of the present invention, in step 204, based on the verification instruction file of the first data file in the data compression file, the first data file is verified through the interface machine, which may be implemented through the following steps:
step 204a, acquiring file attribute parameters of the first data file and file summary information of the first data file through the interface machine.
In the embodiment of the invention, after receiving a first data file sent by a second server, a first server interface machine acquires file attribute parameters of the first data file and file summary information of the first data file. Here, the attribute parameters of the file include: file name, file size, file time, and the number of data recorded in the file. The summary information of the file refers to that the data file is calculated based on a specific summary algorithm to obtain the file summary information of the first data file. The specific summarization algorithm here includes: message Digest Algorithm (MD 5) and Secure Hash Algorithm-1 (SHA-1) algorithms and a number of variations thereof.
And 204b, detecting whether the file attribute parameters of the first data file are matched with the preset file attribute parameters and whether the file abstract information of the first data file is matched with the preset file abstract information through the interface machine.
Wherein, the verification instruction file of the first data file comprises: presetting file attribute parameters and presetting file abstract information.
In the embodiment of the invention, the preset file attribute parameters and the preset file abstract information are carried in the verification instruction file of the first data file. The preset file attribute is an attribute parameter which exists when the second server transmits the compressed data file to be transmitted, and is a file-related attribute parameter when the first data file is not transmitted.
And 204c, if the file attribute parameters of the first data file are matched with the preset file attribute parameters and the file abstract information of the first data file is matched with the preset file abstract information, determining that the first data file is successfully verified through the interface machine.
In the embodiment of the present invention, after receiving a first data file, a first server having an interface engine acquires a file attribute parameter of the first data file based on the received first data file, that is, an interface engine module of the first server acquires a file name of the first data file, a file size of the first data file, a generation time of the first data file, and a data amount recorded in the first data file after receiving the first data file. The interface machine module is further used for calculating file summary information of the first data file based on the acquired first data file.
Illustratively, a first server with an interface machine acquires a file name of a first data file, a file size of the first data file, generation time of the first data file, data volume recorded in the first data file, and file summary information of the first data file. Comparing the file name of the first data file, the file size of the first data file, the generation time of the first data file, the data volume recorded in the first data file and the file abstract information of the first data file with the preset file attribute parameters and the preset file abstract information in the verification description file of the first data file one by one, and checking whether the comparison result is consistent. And the first server with the interface machine determines that the first data file with the consistent comparison result is the data file which is successfully verified.
In the embodiment of the invention, the first server determines that the first data file which is successfully verified generates the verification receipt, writes the file verification result of the first data file into the verification receipt, and returns the file verification result to the second server. And when the first server sends the file verification result to the second server, the interface machine module uploads the first data file successfully verified to the database of the first server.
For example, after the first server determines that the first data file is successfully verified, a file verification receipt beginning with the prefix f is generated, and the file verification result is written into the receipt. Here, the check result is represented by a different code. For example, when the file verification results are consistent, the code is 00, and when the verification results are inconsistent, different codes are written in the receipt according to the coding meaning of the verification results of the file verification, and the verification results are returned to the corresponding second server. And finally, adding a success prefix to the file name of the first data file with the file verification success.
Illustratively, the meaning of each code of the verification result of the file verification is shown in the following table:
| serial number | Check result code | Description of verification result |
| 1 | 00 | Success of the verification |
| 2 | 01 | The name of the interface file does not conform to the rules |
| 3 | 02 | Interface data file absence |
| 4 | 03 | The interface data file cannot be opened |
| 5 | 05 | File size mismatch |
| 6 | 06 | Inconsistency of number of file records |
| 7 | 07 | Date of file data is not in line |
| 8 | 08 | Data file data date is illegal |
| 9 | 09 | Data file interface unit English name illegal |
| 10 | 10 | Data file sizes exceeding 2,000,000,000bytes |
| 11 | 11 | Interface data file repeat upload |
| 12 | 12 | Date of data file not matching expected date |
| 13 | 13 | Checking file recording format errors |
| 14 | 14 | Calculating that the MD5 value is not equal to the MD5 value of the check file |
| 15 | 92 | Verifying that the data date of the instruction file does not match the expected date |
| 17 | 94 | Checking that the file interface unit English name is illegal |
| 18 | 95 | Check description file record illegal terminator (non-carriage return line change) |
| 19 | 96 | Checking that the data date of the instruction file is illegal |
| 20 | 97 | Verification statement file cannot be opened |
Based on the foregoing embodiment, in other embodiments of the present invention, the data file processing method may further include the following steps:
In the embodiment of the invention, the interface machine of the first server performs file verification on the first data file, and if the interface machine of the first server determines that partial data in the first data file fails to be verified, first target data corresponding to the data with failed file verification is obtained.
And step 207, sending the first target data to a second server.
In the embodiment of the invention, the interface machine of the first server acquires the first target data which fails in file verification and then sends the first target data to the second server, and the second server further processes the first target data.
And 208, receiving second target data sent by the second server, and verifying the second target data through the interface machine.
And the second target data is obtained by revising the first target data by the second server.
In the embodiment of the invention, if the interface module of the first server determines that the file attribute parameters of the first data file are not matched with the preset file attribute parameters and/or the file summary information of the first data file is not matched with the preset file summary information, it is determined that the file verification of the first data file fails. And the interface machine acquires unmatched first target data in the first data file and sends the unmatched first target data to the second server. At this time, when the second server receives the first target data which fails to be verified, the second server revises the first target data to obtain revised second target data, the second server sends the second target data to the interface machine of the first server, and the interface machine of the first server verifies the second target data. Therefore, the first server interface machine revises the data which fails to be checked in the first data file through the second server and then checks the data again, and the correctness of the first data file checking of the first server interface machine is guaranteed.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
The data file processing method provided by the embodiment of the invention comprises the steps of collecting a first data file to be uploaded in real time; distributing a first data file to an interface machine corresponding to the first data file; the first data file is verified through the interface machine; if the first data file is successfully verified, a verification success receipt is sent to the second server, and the first data file is stored in the database of the first server, so that the first data file is verified based on the first server with the interface machine, the verification success receipt can be returned to the second server after the verification is successful, the next period of data file collection is carried out, the first data file is uploaded to the database of the first server, and the next period of data file collection is not required to be carried out after the first data file is uploaded to a distributed cluster database (Hive) as in the relative technology.
Based on the foregoing embodiments, an embodiment of the present invention provides a data file processing method, which is applied to a first server having an interface machine, and as shown in fig. 3, the method includes the following steps:
and 301, acquiring a first data file to be uploaded in real time.
Wherein, the verification instruction file of the first data file comprises: presetting file attribute parameters and presetting file summary information.
And step 306, if the first data file is successfully verified, verifying the data fields included in the first data file through the interface machine.
In the embodiment of the invention, after the monitoring module of the first server with the interface machine detects that the file verification of the first data file is successful, the first data file which is verified successfully is further read, and the field of the data in the first data file is verified according to the field verification rule in the specification in the interface machine.
In this embodiment of the present invention, if the data field included in the first data file is successfully verified in step 307, sending a verification success receipt to the second server and storing the first data file in the first server database, which may be implemented by the following steps:
step 307a, if the first data file is successfully verified, acquiring field attribute parameters of the data fields and field summary information of the data fields through the interface machine.
In the embodiment of the invention, after receiving the first data file with the successfully verified file, the first server interface machine acquires the field attribute parameters of the data included in the first data file with the successfully verified file and the field abstract information of the data included in the first data file. Here, the attribute parameters of the field include: the number of fields in the file, the type of field, the field format, and the field delimiter. The summary information of the fields refers to summary information of the fields of the data included in the first data file obtained by performing summary calculation on each of the fields divided in the data file based on a specific summary algorithm. The specific summarization algorithm here includes: the MD5 Message Digest Algorithm (Message-Digest Algorithm, MD 5) and the Secure Hash Algorithm 1 (SHA-1) algorithms, and a number of variations thereof.
And 307b, detecting whether the data field attribute parameters are matched with the preset field attribute parameters and whether the field abstract information of the data field is matched with the preset field abstract information through the interface machine.
In the embodiment of the present invention, after receiving a first data file which is successfully verified, a first server with an interface machine acquires field attribute parameters of data included in the first data file based on the received first data file which is successfully verified, that is, a first server interface machine module acquires a field type of the data included in the received first data file, a field number of the data included in the first data file, a field format of the data included in the first data file, and a field delimiter of the data included in the first data file. The interface machine module of the first server is further configured to calculate field summary information of the received data included in the first data file based on the field of the acquired data included in the first data file.
And 307c, if the field attribute parameters of the data are matched with the preset field attribute parameters and the field abstract information of the data field is matched with the preset field abstract information, determining that the data field is successfully verified through the interface machine and generating a field verification success receipt.
In the embodiment of the invention, a first server with an interface machine acquires the field type of the data included in a first data file which is successfully verified, the field number of the data included in the first data file, the field format of the data included in the first data file, the field separator of the data included in the first data file, and the field summary information of the data included in the first data file. The first server interface machine compares the field type of the data included in the first data file, the field quantity of the data included in the first data file, the field format of the data included in the first data file, the field separator of the data included in the first data file, the field abstract information of the data included in the first data file, the preset field attribute parameters of the data included in the first data file, the field abstract information of the data included in the first data file and the preset field abstract information, and checks whether the comparison results are consistent. And the first server interface machine takes the first data file with the determined comparison result as the data file with the successfully verified field.
For example, after the first server determines that both the file check of the first data file and the field check of the data in the file are successful, a file check receipt beginning with the prefix s is generated, and the file check result is written back into the receipt. The verification result receipt code is shown below.
bj_a_20180911_OAGW_PC_LOGIN_00,s_bj_a_20180816_OAGW_PC_LOG IN_00_001.verf
bj_a_20180911_OAGW_PC_LOGIN_00.zip,s_bj_a_20180817_OAGW_PC_L OGIN_00_001.verf
f_bj_a_20180722_OAGW_PC_LOGIN_00.verf,s_bj_a_20180818_OAGW_PC _LOGIN_00_001.verf
Illustratively, the meaning represented by the successfully verified response piece code shown in the above figure may be further explained: and after the first server completes file verification and field verification on the first data file, the first server sends a verification receipt to the second server. Wherein the content of the response piece comprises: a compressed file (zip) packet, a file check success receipt beginning with f, and a field check success receipt beginning with s.
Based on the foregoing embodiment, in other embodiments of the present invention, the method for processing a data file may further include the following steps:
and 308, if the field verification of the data field fails, acquiring third target data corresponding to the field with the failed field verification.
In the embodiment of the invention, the interface machine of the first server performs field verification on the data in the first data file with successful data verification, and if the data field in the first data file fails to be verified, the interface machine acquires the third target data corresponding to the field with failed field verification. Therefore, the first server continues to check the data field after the first data file is successfully checked, and even if the field check fails, the interface machine can acquire the third target data corresponding to the failed field, so that the integrity of the data field check is ensured.
In the embodiment of the invention, the interface machine of the first server acquires the third target data corresponding to the failed field and sends the third target data to the second server, and the second server further processes the third target data.
And 310, receiving fourth target data sent by the second server, and verifying the fourth target data through the interface machine.
And the fourth target data is obtained by revising the third target data by the second server.
In the embodiment of the present invention, if the interface machine module of the first server determines that the field attribute parameter of the data included in the first data file does not match the preset field attribute parameter and/or the field summary information of the data included in the first data file does not match the preset field summary information, it is determined that the field verification of the data included in the first data file fails. And the interface machine acquires third target data which are included in the first data file and failed in data field verification, and sends the unmatched third target data to the second server. At this time, when the second server receives the third target data which fails to be verified, the second server revises the third target data to obtain revised fourth target data, the second server sends the fourth target data to the interface machine of the first server, and the interface machine of the first server verifies the fourth target data.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
The data file processing method provided by the embodiment of the invention comprises the steps of collecting a first data file to be uploaded in real time; distributing a first data file to an interface machine corresponding to the first data file; the first data file is verified through the interface machine; if the first data file is successfully verified, a verification success receipt is sent to the second server, and the first data file is stored in the database of the first server, so that the first data file is verified based on the first server with the interface machine, the verification success receipt can be returned to the second server after the verification is successful, the next period of data file is collected, and meanwhile, the first data file is uploaded to the database of the first server, and the next period of data file is collected without waiting for the completion of the uploading of the first data file in the relative technology.
Based on the foregoing embodiment, an embodiment of the present invention provides a data file processing method, which is applied to a first server, and as shown in fig. 4, the method includes the following steps:
and step A1, the second server sends data to an interface machine of the first server.
In the embodiment of the present invention, different servers send data files to a first server having an interface machine, and an interface and a module of the first server allocate the data files to corresponding interface machine units based on the types of the different data files, as shown in fig. 6. The first server is provided with a plurality of interface machine modules, creates data acquisition receiving catalogues for data files sent by different systems, creates different account numbers for the systems, and allocates different catalog authorities.
Illustratively, the first data file may be a data file that is collected locally by a different second server according to the interface specification. And the second server uploads the data file to the directory corresponding to the interface engine system. And finally, loading the data file successfully verified by the interface machine of the first server into the database Hive of the first server.
And step A2, the first server performs file verification on the data file through the interface machine to generate a file verification receipt, and sends the file verification receipt to the second server.
In the embodiment of the invention, the interface machine of the first server firstly performs file verification on the first data file and generates a file verification result based on the file verification result.
And step A3, the second server reads the file verification receipt.
In the embodiment of the invention, the second server refers to data of terminal servers of different users, each terminal server of the users is the second server, and the second server reads the return receipt content of the verification result of the first server with the interface machine on the uploaded data and judges whether data modification processing is needed or not.
And step A4, the second server revises the data file with the file verification failure and sends the revised target data to the first server with the interface machine.
In the embodiment of the invention, the second server receives the file verification receipt returned by the first server interface machine, and if the verification result in the receipt is coded as 00, the file verification is passed; if the codes are other, the file is checked to be wrong, the interface specification needs to be contrasted, and the data file is repackaged and uploaded after being corrected.
And step A5, the first server carries out field verification on the data included in the data file through the interface machine to generate a field verification receipt, and the field verification receipt is sent to the second server.
In the embodiment of the invention, after the interface machine of the first server completes the file verification of the first data file, the field verification of the data field in the successfully verified first data file is performed, and the field verification result receipt is generated based on the field verification result of the data in the first data file.
And step A6, the second server reads the field check receipt.
In the embodiment of the invention, the second server is provided with a plurality of second servers, and the second server reads the receipt content of the uploaded field check result of the first server with the interface machine and judges whether data modification processing is needed or not.
And step A7, the second server revises the data with the field verification failure and sends the revised data to the first server with the interface machine.
In the embodiment of the invention, the second server receives the field check receipt returned by the interface machine of the first server, and if the receipt is empty, the first server continues to carry out field check on the first data file; and if the receipt is not empty, the second server modifies the error data file in the receipt, and repacks and uploads the error data file.
Based on the foregoing embodiments, an embodiment of the present invention provides a first server for data file processing, where the first server may be applied to the data file processing method provided in the embodiments corresponding to fig. 1 to 3, and referring to fig. 5, the first server 4 includes: interface specification 41, data file 42, listening module 43, and verification module 44, wherein:
before receiving the file verification data, the interface server formulates a data file acquisition interface specification 41, the interface specification 41 formulates a specification which is known between the first server and the second server, and the two servers interact through the interface specification. The interface specification 41 includes the following:
the naming mode, file format and interface type of the data file 42 are established, and the file packaging mode is that the data file and the file verification instruction file are packaged into a zip file.
The monitoring module 43 will continuously monitor whether there is an uploaded data file directory on the server side of the interface machine. After the interface server monitors that the file zip with the data file directory is uploaded, the checking module 44 monitors the event, and then decompresses the monitored file zip package into the folder with the same name as the file zip package.
For example, taking a data file as the first data file, the snooping module of the first server having the interface machine is implemented based on the pyinotify library of pythond, and may continuously snoop the interface machine data file upload directory. After the compressed data file is uploaded, the checking tool immediately detects the event, and then decompresses the detected zip package into the folder with the same name as the zip package.
After the Zip decompression is completed, the file verification module is used for file verification, and the file verification process is as shown in fig. 7:
the verification module 44 includes: a file check 45 and a field check 46. The fields specifying the file check 45 specify the file include the file size, the number of records, the date, and the MD5 value.
Illustratively, the interface module of the first server reads the data file and the verification description file in the decompressed folder, and checks whether the file name, the file size, the file time, the record number, the MD5 value in the description file are consistent with the calculation result of the actual data file one by one.
And establishing a checking rule of file checking 45, wherein the checking rule comprises the file size, the file record number, the file date and an MD5 value.
Receipt module 47 includes: a file check validation receipt 48 and a field check validation receipt 49 are formulated.
Illustratively, receipt module 47 verifies that the receipt result is replaced with a code. For example, the interface and module of the first server generates a file check receipt with the prefix f as the beginning, writes the file check result into the receipt, correctly encodes the file check result into 00, writes different codes into different error reference interface specifications, and returns the different codes to the corresponding second server. And finally, adding a success prefix to the data file name successfully verified by the file.
The verification receipt 48 that establishes the file verification includes the contents: the receipt comprises a data file name, a verification time and a verification result coding field.
After the file is verified, the field is verified, and the field verification process is shown in fig. 8.
For example, the first server interface module first detects and reads a data file with a success prefix in the filename in the decompression folder (if there is no such file, no subsequent processing is performed), and performs field check according to the number of data fields, field format, field separators, and the like specified in the interface specification.
The rules for checking the fields of the interface machine, which are formulated based on the convention, comprise the field quantity of each data type, the field rules and the like.
Receipt 49, which formulates field checks, includes the contents: the field checks for all data records that are in error.
Illustratively, the first server provides the data file with correct field check to the ETL for data loading into Hive, and the record with error field check is written into the field check receipt with s as the prefix and returned to the corresponding second server. And finally adding done prefix to the data file name with successfully verified field.
The first server provided by the embodiment of the invention comprises a first data file to be uploaded, a second data file and a third data file, wherein the first data file is acquired in real time; distributing a first data file to an interface machine corresponding to the first data file; the first data file is verified through the interface machine; if the first data file is successfully verified, a verification success receipt is sent to the second server, and the first data file is stored in the database of the first server, so that the first data file is verified based on the first server with the interface machine, the verification success receipt can be returned to the second server after the verification is successful, the next period of data file is collected, and meanwhile, the first data file is uploaded to the database of the first server, and the next period of data file is collected without waiting for the completion of the uploading of the first data file in the relative technology.
An embodiment of the present invention provides a first server 5 having an interface, where the first server 5 may be applied to a data file processing method provided in the embodiments corresponding to fig. 1 to 3, and as shown in fig. 9, the server 5 may include: a processor 51, a memory 52, and a communication bus 53, wherein:
the communication bus 53 is used to realize a communication connection between the processor 51 and the memory 52.
The processor 51 is configured to execute a program of a data file processing method stored in the memory 52 to implement the steps of:
acquiring a first data file to be uploaded in real time;
distributing a first data file to an interface machine corresponding to the first data file;
checking the first data file through the interface machine;
and if the first data file is successfully verified, sending a verification success receipt to the second server, and storing the first data file in a database of the first server.
In other embodiments of the present invention, the processor 51 is configured to execute the real-time acquisition stored in the memory 52 of the first data file to be uploaded, so as to implement the following steps:
collecting a data compression file to be uploaded in real time;
and analyzing the data compression file to obtain a first data file.
In other embodiments of the present invention, the processor 51 is configured to execute the verification of the first data file in the memory 52 through the interface machine to implement the following steps:
and verifying the first data file through the interface machine based on the verification description file of the first data file in the data compression file.
In other embodiments of the present invention, the processor 51 is configured to execute a verification instruction file in the memory 52 based on a first data file in the data compression file, and verify the first data file through the interface machine to implement the following steps:
acquiring file attribute parameters of the first data file and file summary information of the first data file through an interface machine;
detecting whether the file attribute parameters of the first data file and the file abstract information of the first data file are matched with the preset file attribute parameters and the preset file abstract information through an interface machine;
wherein, the verification instruction file of the first data file comprises: presetting file attribute parameters and file abstract information;
and if the file attribute parameters of the first data file are matched with the preset file attribute parameters and the file abstract information of the first data file is matched with the preset file abstract information, determining that the first data file is successfully verified through the interface machine.
In other embodiments of the present invention, the processor 51 is configured to execute the following steps to allocate the first data file to the interface machine corresponding to the first data file in the memory 52:
determining a file type of a first data file;
and distributing the first data file to an interface machine corresponding to the file type of the first data file based on the file type of the first data file.
In another embodiment of the present invention, the processor 51 is configured to execute the following steps of, if the first data file is successfully verified in the memory 52, sending a verification success receipt to the second server, and storing the first data file in the database of the first server:
if the first data file fails to be verified, acquiring first target data which fails to be verified in the first data file and sending the first target data to a second server;
receiving second target data sent by a second server, and verifying the second target data through an interface machine;
and the second target data is obtained by revising the first target data by the second server.
In another embodiment of the present invention, the processor 51 is configured to execute the following steps of, if the first data file is successfully verified in the memory 52, sending a verification success receipt to the second server, and storing the first data file in the database of the first server:
if the first data file is successfully verified, verifying the data fields included in the first data file through the interface machine;
and if the data fields included in the first data file are verified successfully, sending a verification success receipt to the second server and storing the first data file to the first server database.
In other embodiments of the present invention, the processor 51 is configured to execute, if the first data file is successfully verified in the memory 52, verifying, by the interface machine, a data field included in the first data file, so as to implement the following steps:
acquiring field attribute parameters of data included in a first data file and field abstract information of the data included in the first data file through an interface machine;
detecting whether field attribute parameters of data included in the first data file and field abstract information of the data included in the first data file are matched with preset field attribute parameters and preset field abstract information through an interface machine;
and if the field attribute parameters of the data included in the first data file are matched with the preset field attribute parameters and the field abstract information of the data included in the first data file is matched with the preset field abstract information, determining that the field verification of the data included in the first data file is successful and generating a field verification success receipt.
In other embodiments of the present invention, the processor 51 is configured to execute the following steps of, if the data field included in the first data file in the memory 52 is successfully verified, sending a verification success receipt to the second server and storing the first data file in the first server database:
if the field verification of the data included in the first data file fails, acquiring third target data of the data field verification failure included in the first data file and sending the third target data to a second server;
receiving fourth target data sent by the second server, and verifying the fifth data through the interface machine; and the fourth target data is obtained by revising the third target data by the second server.
It should be noted that, in this embodiment, the same steps and the same contents as those in the other embodiments may refer to the implementation process in the data file processing method provided in the embodiments corresponding to fig. 1 to 3, and are not described herein again.
The first server provided by the embodiment of the invention can acquire a first data file to be uploaded in real time; distributing a first data file to an interface machine corresponding to the first data file; the first data file is verified through the interface machine; if the first data file is successfully verified, a verification success receipt is sent to the second server, and the first data file is stored in the database of the first server, so that the first data file is verified based on the first server with the interface machine, the verification success receipt can be returned to the second server after the verification is successful, the next period of data file is collected, and meanwhile, the first data file is uploaded to the database of the first server, and the next period of data file is collected without waiting for the completion of the uploading of the first data file in the relative technology.
Based on the foregoing embodiments, embodiments of the invention provide a computer storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the steps of:
acquiring a first data file to be uploaded in real time;
distributing a first data file to an interface machine corresponding to the first data file;
checking the first data file through the interface machine;
and if the first data file is successfully verified, sending a verification success receipt to the second server, and storing the first data file in a database of the first server.
In other embodiments of the present invention, the one or more programs are executable by the one or more processors to collect in real-time a first data file to be uploaded to implement the steps of:
collecting a data compression file to be uploaded in real time;
and analyzing the data compression file to obtain a first data file.
In other embodiments of the present invention, the one or more programs are executable by the one or more processors to verify the first data file via the interface machine to perform the steps of:
and verifying the first data file through the interface machine based on the verification description file of the first data file in the data compression file.
In other embodiments of the present invention, the one or more programs are executable by the one or more processors to perform the following steps of verifying a first data file in the data compression file via the interface engine based on a verification specification file of the first data file:
acquiring file attribute parameters of a first data file and file summary information of the first data file;
detecting whether the file attribute parameters of the first data file and the file abstract information of the first data file are matched with the preset file attribute parameters and the preset file abstract information; wherein, the verification instruction file of the first data file comprises: presetting file attribute parameters and file abstract information;
and if the file attribute parameters of the first data file are matched with the preset file attribute parameters and the file abstract information of the first data file is matched with the preset file abstract information, determining that the first data file is successfully verified.
In other embodiments of the present invention, the one or more programs are executable by the one or more processors to allocate the first data file to an interface machine corresponding to the first data file to implement the steps of:
determining a file type of a first data file;
and distributing the first data file to an interface machine corresponding to the file type of the first data file based on the file type of the first data file.
In other embodiments of the present invention, the one or more programs, which are executed by the one or more processors to send a verification success receipt to the second server if the first data file is verified successfully, and store the first data file in the database of the first server, may further implement the following steps:
if the first data file fails to be verified, acquiring first target data which fails to be verified in the first data file and sending the first target data to a second server;
receiving second target data sent by a second server, and verifying the second target data through an interface machine; and the second target data is obtained by revising the first target data by the second server.
In another embodiment of the present invention, the one or more programs are executable by the one or more processors to send a verification success receipt to the second server if the first data file is verified successfully, and store the first data file in the database of the first server, so as to implement the following steps:
if the first data file is successfully verified, verifying the data fields included in the first data file through the interface machine;
and if the data field included in the first data file is successfully verified, sending a verification success receipt to the second server and storing the first data file to the first server database.
In other embodiments of the present invention, the one or more programs are executable by the one or more processors to, if the first data file is successfully verified, verify data fields included in the first data file through the interface machine to implement the following steps:
acquiring field attribute parameters of data included in a first data file and field summary information of the data included in the first data file;
detecting whether field attribute parameters of data included by a first data file and field summary information of the data included by the first data file are matched with preset field attribute parameters and preset field summary information or not;
and if the field attribute parameters of the data included in the first data file are matched with the preset field attribute parameters and the field abstract information of the data included in the first data file is matched with the preset field abstract information, determining that the field verification of the data included in the first data file is successful and generating a field verification success receipt.
In other embodiments of the present invention, if the data field included in the first data file is successfully verified, the one or more programs may be executed by the one or more processors to send a verification success receipt to the second server and store the first data file in the first server database, and the following steps may be further implemented:
if the field verification of the data included in the first data file fails, acquiring third target data of the data field verification failure included in the first data file and sending the third target data to a second server;
receiving fourth target data sent by the second server, and verifying the fifth data through the interface machine; and the fourth target data is obtained by revising the third target data by the second server.
In other embodiments of the present invention, the one or more programs are executable by the one or more processors to allocate the first data file to the interface machine corresponding to the first data file, and may further implement the following steps:
acquiring a first data file meeting a specific specification in real time;
wherein the specific specification is a specification established by the second server and the first server with the interface machine based on the agreement, and the specification comprises: naming mode of data file, file format, interface type, file packing mode, verification description file, verification rule and verification receipt.
The computer-readable storage medium may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a magnetic Random Access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM), and the like; and may be various electronic devices such as mobile phones, computers, tablet devices, personal digital assistants, etc., including one or any combination of the above-mentioned memories.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method described in the embodiments of the present invention.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data file processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data file processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data file processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data file processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (11)
1. A data file processing method applied to a first server having an interface machine, the method comprising:
acquiring a first data file to be uploaded in real time;
distributing the first data file to an interface machine corresponding to the first data file;
verifying the first data file through the interface machine;
if the first data file is successfully verified, sending a verification success receipt to a second server, and storing the first data file to a database of the first server;
and loading the first data file which is verified while receiving the verification work of the next group of data files through the interface machine.
2. The method of claim 1, wherein the collecting in real-time a first data file to be uploaded comprises:
collecting a data compression file to be uploaded in real time;
and analyzing the data compression file to obtain the first data file.
3. The method of claim 2, wherein said verifying the first data file by the interface machine comprises:
and verifying the first data file through the interface machine based on the verification instruction file of the first data file in the data compression file.
4. The method according to claim 3, wherein the verifying the first data file through the interface machine based on the verification specification file of the first data file in the data compression file comprises:
acquiring file attribute parameters of the first data file and file abstract information of the first data file through the interface machine;
detecting whether the file attribute parameters of the first data file are matched with preset file attribute parameters and whether the file abstract information of the first data file is matched with preset file abstract information through the interface machine; wherein the verification specification file of the first data file comprises: the preset file attribute parameters and the preset file abstract information;
and if the file attribute parameters of the first data file are matched with the preset file attribute parameters and the file abstract information of the first data file is matched with the preset file abstract information, determining that the first data file is successfully verified through the interface machine.
5. The method of claim 1, wherein said assigning the first data file to an interface machine corresponding to the first data file comprises:
determining a file type of the first data file;
and distributing the first data file to an interface machine corresponding to the file type of the first data file based on the file type of the first data file.
6. The method of claim 1 or 4, further comprising:
if the first data file fails to be verified, acquiring first target data which fails to be verified in the first data file;
sending the first target data to the second server;
receiving second target data sent by the second server, and verifying the second target data through the interface machine; the second target data is obtained by revising the first target data by the second server.
7. The method of claim 1, wherein sending a verification success receipt to a second server and storing the first data file to the database of the first server if the first data file is verified successfully comprises:
if the first data file is successfully verified, verifying a data field included in the first data file through the interface machine;
and if the data field is verified successfully, sending a verification success response to the second server and storing the first data file to a database of the first server.
8. The method of claim 7, wherein if the first data file is verified successfully, verifying, by the interface machine, data fields included in the first data file comprises:
if the first data file is verified successfully, field attribute parameters of the data fields and field abstract information of the data fields are obtained through the interface machine;
detecting whether the field attribute parameters of the data fields are matched with preset field attribute parameters and whether the field abstract information of the data fields is matched with preset field abstract information through the interface machine;
and if the field attribute parameters of the data are matched with the preset field attribute parameters and the field abstract information of the data field is matched with the preset field abstract information, determining that the data field is successfully verified through the interface machine and generating a field verification success receipt.
9. The method of claim 7, further comprising:
if the field verification of the data field fails, acquiring third target data corresponding to the field with the failed field verification;
sending the third target data to a second server;
receiving fourth target data sent by a second server, and verifying the fourth target data through the interface machine; wherein the fourth target data is obtained by revising the third target data by the second server.
10. A first server, wherein the first server comprises: a processor, a memory, and a communication bus, comprising:
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is used for executing the program of the data file processing method in the memory so as to realize the following steps:
acquiring a first data file to be uploaded in real time;
distributing the first data file to an interface machine corresponding to the first data file;
verifying the first data file through the interface machine;
if the first data file is successfully verified, sending a verification success receipt to a second server, and storing the first data file to a database of the first server;
and loading the first data file which is verified while receiving the verification work of the next group of data files through the interface machine.
11. A computer storage medium, characterized in that the computer storage medium stores one or more programs executable by one or more processors to implement the steps of the data file processing method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910765633.4A CN112395266B (en) | 2019-08-19 | 2019-08-19 | Data file processing method, first server and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910765633.4A CN112395266B (en) | 2019-08-19 | 2019-08-19 | Data file processing method, first server and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112395266A CN112395266A (en) | 2021-02-23 |
| CN112395266B true CN112395266B (en) | 2022-12-13 |
Family
ID=74603515
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910765633.4A Active CN112395266B (en) | 2019-08-19 | 2019-08-19 | Data file processing method, first server and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112395266B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113132409B (en) * | 2021-04-29 | 2022-10-18 | 工银科技有限公司 | Data transmission method and device |
| CN113542422B (en) * | 2021-07-19 | 2023-10-17 | 北京星辰天合科技股份有限公司 | Data storage method and device, storage medium and electronic device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595597A (en) * | 2018-04-19 | 2018-09-28 | 平安科技（深圳）有限公司 | Data lead-in method, device, computer equipment and storage medium |
| CN109165017A (en) * | 2018-07-05 | 2019-01-08 | 平安科技（深圳）有限公司 | Data verification method, device and computer storage medium |
| CN110069449A (en) * | 2019-03-20 | 2019-07-30 | 平安科技（深圳）有限公司 | Document handling method, device, computer equipment and storage medium |
-
2019
- 2019-08-19 CN CN201910765633.4A patent/CN112395266B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595597A (en) * | 2018-04-19 | 2018-09-28 | 平安科技（深圳）有限公司 | Data lead-in method, device, computer equipment and storage medium |
| CN109165017A (en) * | 2018-07-05 | 2019-01-08 | 平安科技（深圳）有限公司 | Data verification method, device and computer storage medium |
| CN110069449A (en) * | 2019-03-20 | 2019-07-30 | 平安科技（深圳）有限公司 | Document handling method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112395266A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110309125B (en) | Data verification method, electronic device and storage medium | |
| CN108874409B (en) | Information updating method, device, server and system | |
| CN112395266B (en) | Data file processing method, first server and computer storage medium | |
| CN111382164B (en) | Service processing method based on block chain network | |
| CN111026767B (en) | Block chain data storage method and device and hardware equipment | |
| CN110554930A (en) | Data storage method and related equipment | |
| EP3739493A1 (en) | File verification method, file verification system and file verification server | |
| CN110063042B (en) | Database fault response method and terminal thereof | |
| CN110222028A (en) | A kind of data managing method, device, equipment and storage medium | |
| CN110088744A (en) | A kind of database maintenance method and its system | |
| CN111899019A (en) | Method and system for cross validation and sharing of blacklist and multiple parties | |
| CN111339551B (en) | Data verification method and related device and equipment | |
| CN114416638A (en) | Automatic electronic file filing method and system | |
| CN112328631A (en) | Production fault analysis method and device, electronic equipment and storage medium | |
| CN113672634A (en) | Policy data verification method, device, equipment and storage medium | |
| CN111338644A (en) | Task script deployment method and system | |
| CN109587202B (en) | Method, device and system for coprocessing between file verification and data processing | |
| CN111241082B (en) | Data correction method and device | |
| CN112597119A (en) | Method and device for generating processing log and storage medium | |
| CN110620800B (en) | Method and device for synchronously processing configuration data of power secondary system across safety zones | |
| CN112631991B (en) | File migration method and device | |
| CN115934670B (en) | Method and device for verifying copy placement strategy of HDFS (Hadoop distributed File System) multi-machine room | |
| CN116028444B (en) | File fingerprint generation method, device and system, electronic equipment and storage medium | |
| CN113157645B (en) | Cluster data migration method, device, equipment and storage medium | |
| CN114385595B (en) | Data migration method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |