CN112241408A - Temporary table-based invoice data batch storage method - Google Patents

Temporary table-based invoice data batch storage method Download PDF

Info

Publication number
CN112241408A
CN112241408A CN202011060185.7A CN202011060185A CN112241408A CN 112241408 A CN112241408 A CN 112241408A CN 202011060185 A CN202011060185 A CN 202011060185A CN 112241408 A CN112241408 A CN 112241408A
Authority
CN
China
Prior art keywords
data
batch
invoice data
formal
invoice
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011060185.7A
Other languages
Chinese (zh)
Inventor
唐凌云
孟繁杰
赵天鹏
冯帆
王轶晨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aisino Corp
Original Assignee
Aisino Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aisino Corp filed Critical Aisino Corp
Priority to CN202011060185.7A priority Critical patent/CN112241408A/en
Publication of CN112241408A publication Critical patent/CN112241408A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/12Accounting
    • G06Q40/123Tax preparation or submission

Abstract

The invention discloses a temporary table-based invoice data batch warehousing method, which comprises the following steps: inserting invoice data to be imported into a pre-created intermediate table; executing a batch comparison program of first database table data based on a first formal table for storing invoice data, and dividing the invoice data to be imported into first-class data and second-class data; executing a batch comparison program of data of a second database table based on a second formal table for storing the invoice data state, and dividing the invoice data to be imported into third-class data and fourth-class data; carrying out batch updating operation on the first formal table according to the first type of data; performing batch insertion operation on the first formal table according to the second type of data; performing batch updating operation on the second formal table according to the third type of data; and performing batch insertion operation on the second formal table according to the fourth type of data. According to the invention, the problems of long time consumption and low efficiency of the existing invoice data storage ring for tax entry management can be solved.

Description

Temporary table-based invoice data batch storage method
Technical Field
The invention belongs to the field of data processing, and particularly relates to a temporary table-based invoice data batch warehousing method.
Background
In the current software development process, JAVA language and structured databases are widely used. According to the past software development experience, the JAVA language is generally considered to be used for processing business logic, and the database is used for storing data. Therefore, during the writing process of the software code, the JAVA code uses a large amount of FOR loops to perform traversal processing of each piece of data. However, when the amount of processed data is large, traversal may cause the JAVA virtual machine to be overloaded, thereby seriously affecting software performance. Meanwhile, the powerful performance optimization space of the database on the business logic processing is generally ignored.
In terms of specific application fields, the JAVA-based data traversal processing manner is often applied to tax intake management. The tax entry management is an important link of group enterprise tax informatization, one important work of the tax entry management is to import invoice data downloaded from a tax bureau into a database of a tax entry management system, and the specific process is as follows: and dividing the invoice data to be imported into existing invoice data and non-existing invoice data by taking whether the invoice data to be imported already exists in a formal table of a database for storing the invoice data as a classification standard. And dividing the invoice data to be imported into invoice data with existing state and invoice data with non-existing state by taking whether the invoice data to be imported already exists in a formal table of a database for storing the state of the invoice data as a classification standard. And after the classification of the invoice data is finished, performing corresponding database operation on corresponding database formal tables according to each type of invoice data so as to realize the storage of the invoice data.
However, the classification of the invoice data is implemented by using a JAVA-based data traversal processing method. The invoice data classification mode can meet the business requirements when the invoice data volume is small. However, when the invoice data volume is large, the invoice data classification method takes too long time and is low in efficiency.
Disclosure of Invention
The invention aims to solve the problems of long time consumption and low efficiency of the existing invoice data warehousing ring for tax entry management.
In order to achieve the above object, the present invention provides a method for batch loading invoice data based on a temporary table, which is used for importing the obtained invoice data into a target database in batch, and comprises the following steps:
inserting invoice data to be imported into a pre-created intermediate table;
based on a first formal table for storing invoice data, executing a first database table data batch comparison program, and dividing the invoice data to be imported into first-class data and second-class data;
based on a second formal table used for storing invoice data states, executing a second database table data batch comparison program, and dividing the invoice data to be imported into third-class data and fourth-class data;
carrying out batch updating operation on the first formal table according to the first type of data;
performing batch insertion operation on the first formal table according to the second type of data;
performing batch updating operation on the second formal table according to the third type of data;
performing batch insertion operation on the second formal table according to the fourth type of data;
the intermediate table, the first formal table and the second formal table are data tables of the target database.
Preferably, the first database table data batch comparison program and the second database table data batch comparison program are both SQL scripts.
Preferably, the SQL scripts all use a count function.
Preferably, the SQL script is used to convert a comparison of two table data into a comparison of two numbers.
Preferably, the first type of data is invoice data that already exists in the first formal table, the second type of data is invoice data that does not already exist in the first formal table, the third type of data is invoice data that already exists in the second formal table, and the fourth type of data is invoice data that does not already exist in the second formal table.
Preferably, before the inserting the invoice data to be imported into the pre-created intermediate table, the method further includes:
and creating an intermediate table, and associating and setting the intermediate table with the first formal table and the second formal table at the same time.
Preferably, the specific manner of the association setting is as follows:
and setting the fields of the intermediate table and the fields of the corresponding formal table in an associated manner, and establishing indexes of the two tables based on the associated fields.
Preferably, after performing the corresponding operation on the corresponding formal table according to the predetermined data, the method further includes:
and judging whether the operation is successful, if not, returning to execute the operation until the operation is judged to be successful.
Preferably, after all the updating operations and the inserting operations are completed, the method further comprises:
and deleting the invoice data stored in the intermediate table.
Preferably, the invoice data stored in the intermediate table is deleted, and the adopted filtering condition is a tax number.
The invention has the beneficial effects that:
the invention discloses a temporary table-based invoice data batch warehousing method, which is characterized in that a pre-created intermediate table is adopted to store invoice data to be imported, a preset database table data batch comparison program is called to realize batch comparison of the data of the intermediate table and the data of a corresponding formal table so as to classify the invoice data, and corresponding database operation is carried out on the basis of the classified invoice data so as to realize batch warehousing of the invoice data.
Different from the existing invoice data warehousing method which adopts a JAVA-based data traversal processing mode to classify invoice data, the invoice data batch warehousing method based on the temporary table realizes batch judgment and rapid classification of invoice data on a database level, fully utilizes powerful performance optimization space of the database on business logic processing, and has the capacity of processing mass data. When the invoice data import task with the same volume is faced, the invoice data batch warehousing method based on the temporary table is relatively short in time consumption and high in efficiency.
Additional features and advantages of the invention will be set forth in the detailed description which follows.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.
FIG. 1 shows a flowchart of an implementation of a temporary table-based invoice data bulk-warehousing method according to an embodiment of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described in more detail below. While the following describes preferred embodiments of the present invention, it should be understood that the present invention may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Example 1: fig. 1 shows a flowchart of an implementation of the temporary table-based invoice data batch-warehousing method according to the present embodiment. Referring to fig. 1, the method for batch loading of invoice data based on a temporary table in this embodiment is used for batch importing the acquired invoice data into a target database, and includes the following steps:
s100, inserting invoice data to be imported into a pre-created intermediate table;
step S200, based on a first formal table for storing invoice data, executing a first database table data batch comparison program, and dividing the invoice data to be imported into first-class data and second-class data;
step S300, based on a second formal table for storing invoice data states, executing a second database table data batch comparison program, and dividing the invoice data to be imported into third-class data and fourth-class data;
s400, performing batch updating operation on the first formal table according to the first type of data;
s500, performing batch insertion operation on the first formal table according to the second type of data;
s600, performing batch updating operation on the second formal table according to the third type of data;
s700, performing batch insertion operation on the second formal table according to the fourth type of data;
the intermediate table, the first formal table and the second formal table are data tables of the target database.
In this embodiment, the first database table data batch comparison program is a first SQL script, and the second database table data batch comparison program is a second SQL script.
In this embodiment, both the first SQL script and the second SQL script employ a count function.
In this embodiment, the first SQL script and the second SQL script are both used to convert the comparison of the two table data into the comparison of two numbers.
In this embodiment, the first type of data is invoice data that already exists in the first formal table, the second type of data is invoice data that does not already exist in the first formal table, the third type of data is invoice data that already exists in the second formal table, and the fourth type of data is invoice data that does not already exist in the second formal table.
The invoice data batch storage method based on the temporary table further includes, before step S100, the steps of:
and creating an intermediate table, and associating and setting the intermediate table with the first formal table and the second formal table at the same time.
In this embodiment, the specific manner of the association setting is as follows:
and setting the fields of the intermediate table and the fields of the corresponding formal table in an associated manner, and establishing indexes of the two tables based on the associated fields.
The method for warehousing invoice data in batch based on the temporary table in this embodiment further includes, after performing corresponding operations on a corresponding formal table according to the established data, the steps of:
and judging whether the operation is successful, if not, returning to execute the operation until the operation is judged to be successful.
The invoice data batch warehousing method based on the temporary table further includes, after completing all the updating operations and the inserting operations:
and deleting the invoice data stored in the intermediate table.
In this embodiment, in the step of deleting the invoice data stored in the intermediate table, the filtering condition is a tax number.
The method for batch warehousing invoice data based on the temporary table in the embodiment is described in more detail based on the following five aspects:
1. establishing a temporary table:
the establishment of the temporary table is the key of the method. The fields of the temporary table correspond to the fields of the formal table one by one, and an index is established based on the associated fields of the two tables. In this embodiment, the temporary table is named as sp _ ptcj _ temp, the first formal table is named as sp _ ptcj, and the fields of the invoice code fpdm and the invoice number fphm are used as the associated fields to establish the index.
2. Insert full data into the temporary table:
after the invoice data set is obtained from the tax bureau, the non-screening full quantity is inserted into the temporary table sp _ ptcj _ temp, so that the advantage of the method is that the data can be put into a warehouse firstly without any traversal cyclic judgment operation in JAVA. Once the data enters the database, the subsequent business judgment can be converted into the operation of a pure database. This lays the foundation for the conversion of the following character comparisons into data comparisons.
3. Character comparison converts to numerical comparison:
the first SQL script of this embodiment is: select from sp _ ptcj _ temp a where (select count (1) as num from sp _ ptcj B where a. fpdm. AND a. fphm. B. fphm) 0;
the method for judging whether the invoice data exists in the first formal table is to compare the count () with 0, and the difference between the count () and the in () and the exsists () is that the in () and the exsists () both compare two characters or compare one character with one number, and the count () compares the characters after converting the characters into the numbers with 0, so that the three obtained structures are the same, but the performances are different greatly. This is done, or it may be done, for a great performance optimization, because: in the database, comparing two characters or comparing one character with one number requires converting the characters and the numbers into ascii code values first and then comparing. And two numbers can be compared directly without a code value conversion step. For example, comparing the sizes of A and a, it is necessary to convert A and a into ascii codes, the ascii code value of A is 65, the ascii code value of a is 97, and then 65 and 97 are compared. Comparing the sizes of A and 0 requires converting A and 0 into ascii codes, the ascii code value of A is 65, the ascii code value of 0 is 48, and comparing 65 and 48. The two numbers in the database can be directly compared without the step of converting ascii code values. Such as sizes of 1 and 0, for example. This saves a lot of conversion time for large data volumes.
4. Batch update and batch insert:
the invoice data classification aims at performing different database operations, and the invoice data is divided into four types in the embodiment, namely, the existing invoice data, the non-existing invoice data, the invoice data in the existing state and the invoice data in the non-existing state need to perform update operation on the corresponding database formal table, and the non-existing data and the invoice data in the non-existing state need to perform insert operation on the corresponding database formal table. Because we have taken four data sets, we do batch updates and batch insertions, which indirectly avoids the effect of a single update and insertion repeatedly occupying the number of database connections.
5. And (3) deleting the data of the temporary table:
after the series of operations are completed, the data in the temporary table become useless data, and the deletion of the data can reduce the load of the database and also can eliminate the interference on the next batch of data. When the filtering condition used by the user is the tax number, the tax number is the unique mark for distinguishing the batches, so that the temporary table can support the concurrent application of multiple tax numbers, and the operation efficiency of the whole system is greatly improved.
The invoice data batch warehousing method based on the temporary table has the following beneficial effects:
1) and through the application of the temporary table, the service operation originally carried out in JAVA is converted into the operation of a pure database, and a foundation is laid for utilizing database indexes and converting the database indexes into numerical comparison.
2) Through a smart SQL script writing method, character comparison is converted into numerical comparison, and single traversal cycle operation is converted into batch one-time operation.
3) And the mark is deleted by taking the tax number as the uniqueness, so that the concurrent operation of multiple tax numbers is well supported, and the operation efficiency of the whole system is greatly increased.
According to the invoice data batch storage method based on the temporary table, the temporary table is used for converting traversal cyclic judgment operation in JAVA into pure database operation, and character comparison is converted into digital comparison, so that the service processing efficiency in the data storage process is greatly improved, and the performance problem caused by a large amount of state judgment when an entry management system downloads invoices from a tax bureau is solved.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

Claims (10)

1. The invoice data batch storage method based on the temporary table is used for leading the obtained invoice data into a target database in batches, and is characterized by comprising the following steps:
inserting invoice data to be imported into a pre-created intermediate table;
based on a first formal table for storing invoice data, executing a first database table data batch comparison program, and dividing the invoice data to be imported into first-class data and second-class data;
based on a second formal table used for storing invoice data states, executing a second database table data batch comparison program, and dividing the invoice data to be imported into third-class data and fourth-class data;
carrying out batch updating operation on the first formal table according to the first type of data;
performing batch insertion operation on the first formal table according to the second type of data;
performing batch updating operation on the second formal table according to the third type of data;
performing batch insertion operation on the second formal table according to the fourth type of data;
the intermediate table, the first formal table and the second formal table are data tables of the target database.
2. The invoice data batch-warehousing method of claim 1, wherein the first database table data batch comparison program and the second database table data batch comparison program are both SQL scripts.
3. The invoice data batch-warehousing method according to claim 2, characterized in that the SQL scripts all adopt a count function.
4. The invoice data batch-warehousing method of claim 3, wherein the SQL script is used to convert a comparison of two table data into a comparison of two numbers.
5. The method according to claim 1, wherein the first type of data is invoice data already existing in the first formal table, the second type of data is invoice data not already existing in the first formal table, the third type of data is invoice data already existing in the second formal table, and the fourth type of data is invoice data not already existing in the second formal table.
6. The invoice data batch-warehousing method according to claim 1, before the inserting the invoice data to be imported into the pre-created intermediate table, further comprising:
and creating an intermediate table, and associating and setting the intermediate table with the first formal table and the second formal table at the same time.
7. The invoice data batch-warehousing method according to claim 6, characterized in that the specific way of the association setting is as follows:
and setting the fields of the intermediate table and the fields of the corresponding formal table in an associated manner, and establishing indexes of the two tables based on the associated fields.
8. The invoice data batch-warehousing method according to claim 1, characterized by further comprising, after performing corresponding operations on corresponding formal tables according to established data:
and judging whether the operation is successful, if not, returning to execute the operation until the operation is judged to be successful.
9. The invoice data batch-warehousing method of claim 1, further comprising, after completing all update operations and insert operations:
and deleting the invoice data stored in the intermediate table.
10. The invoice data batch-warehousing method according to claim 9, characterized in that the invoice data stored in the intermediate table is deleted, and the adopted filtering condition is tax number.
CN202011060185.7A 2020-09-30 2020-09-30 Temporary table-based invoice data batch storage method Pending CN112241408A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011060185.7A CN112241408A (en) 2020-09-30 2020-09-30 Temporary table-based invoice data batch storage method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011060185.7A CN112241408A (en) 2020-09-30 2020-09-30 Temporary table-based invoice data batch storage method

Publications (1)

Publication Number Publication Date
CN112241408A true CN112241408A (en) 2021-01-19

Family

ID=74171786

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011060185.7A Pending CN112241408A (en) 2020-09-30 2020-09-30 Temporary table-based invoice data batch storage method

Country Status (1)

Country Link
CN (1) CN112241408A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106325933A (en) * 2016-08-24 2017-01-11 明算科技(北京)股份有限公司 Method and device for synchronizing batch data
CN108304522A (en) * 2018-01-25 2018-07-20 深圳市买买提信息科技有限公司 Comparison method, device and the terminal device of difference between a kind of database
US20180260467A1 (en) * 2017-03-07 2018-09-13 Mcafee, Llc Aggregate, index based, synchronization of node contents
KR20190023682A (en) * 2017-08-30 2019-03-08 주식회사 영림원소프트랩 Bulk data convert processing method between heterogeneous system in enterprise software
CN110442578A (en) * 2019-07-30 2019-11-12 新华三大数据技术有限公司 Zipper table updating method, device, server and computer readable storage medium
CN110674152A (en) * 2019-09-24 2020-01-10 京东数字科技控股有限公司 Data synchronization method and device, storage medium and electronic equipment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106325933A (en) * 2016-08-24 2017-01-11 明算科技(北京)股份有限公司 Method and device for synchronizing batch data
US20180260467A1 (en) * 2017-03-07 2018-09-13 Mcafee, Llc Aggregate, index based, synchronization of node contents
KR20190023682A (en) * 2017-08-30 2019-03-08 주식회사 영림원소프트랩 Bulk data convert processing method between heterogeneous system in enterprise software
CN108304522A (en) * 2018-01-25 2018-07-20 深圳市买买提信息科技有限公司 Comparison method, device and the terminal device of difference between a kind of database
CN110442578A (en) * 2019-07-30 2019-11-12 新华三大数据技术有限公司 Zipper table updating method, device, server and computer readable storage medium
CN110674152A (en) * 2019-09-24 2020-01-10 京东数字科技控股有限公司 Data synchronization method and device, storage medium and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
USUALHEART: "快速比较mysql中两个表中的数据是否有差异并找出差异项", pages 2, Retrieved from the Internet <URL:https://blog.csdn.net/usualheart/article/details/107403759> *

Similar Documents

Publication Publication Date Title
EP3782044A1 (en) Learning etl rules by example
EP0912948A1 (en) Database apparatus
KR20030011220A (en) Data sort method, data sort apparatus, and data sort program
WO2001033436A1 (en) System for managing rdbm fragmentations
CN112148680B (en) File system metadata management method based on distributed graph database
CN109815238A (en) The dynamic adding method and device of database are realized with strict balanced binary tree
US7502791B2 (en) Database constraint enforcer
CN115757629A (en) Multi-source heterogeneous data increment synchronization method and system, storage medium and electronic equipment
US20160132535A1 (en) Acceleration method for database using index value operation and mixed-mode leveled cache
CN103020285B (en) Integration across database supports automatic coding and the system of multi-field combination
CN112241408A (en) Temporary table-based invoice data batch storage method
JP2001527240A (en) Management in data structures
CN112214521A (en) Rule query method, device, equipment and computer storage medium
CN112883048A (en) Data access method, device, server and readable storage medium
CN115905244A (en) Method for batch processing data based on memory
US5918231A (en) Object-oriented database management system with improved usage efficiency of main memory
CN111752954B (en) Large-scale feature data storage method and device
CN113076330A (en) Query processing method and device, database system, electronic equipment and storage medium
US11188541B2 (en) Join method, computer program and recording medium thereof
CN111796810A (en) Method and system for configurable rapid generation of Mysql database codes
KR102013839B1 (en) Method and System for Managing Database, and Tree Structure for Database
CN110674142A (en) Oracle database index optimization method
CN110807092A (en) Data processing method and device
CN112214792B (en) Authority management method and tool based on cockroachDB database
KR102267768B1 (en) Method for database programming of digital product management system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination