CN103810238A - Large-data-volume Excel file exporting method based on asynchronous processing - Google Patents
Large-data-volume Excel file exporting method based on asynchronous processing Download PDFInfo
- Publication number
- CN103810238A CN103810238A CN201310584833.2A CN201310584833A CN103810238A CN 103810238 A CN103810238 A CN 103810238A CN 201310584833 A CN201310584833 A CN 201310584833A CN 103810238 A CN103810238 A CN 103810238A
- Authority
- CN
- China
- Prior art keywords
- excel
- data
- file
- application module
- excel file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a data exporting method, particularly to a large-data-volume Excel file exporting method based on asynchronous processing. At present, an exporting method for processing massive data occupies overlarge memory resources. The method comprises the steps as follows: an Excel auxiliary application module divides Excel data into multiple data sets; when an application system sends an Excel data exporting request to the Excel auxiliary application module, the Excel auxiliary application module makes Excel data exporting requests to a plurality of processors; each processor builds a new Excel file in a designated storage path when processing the Excel data exporting request; and a plurality of Excel files are overlaid to form a complete Excel file, and a user checks and downloads the generated Excel file in the storage path. According to the technical scheme, the service processing time is short, occupied memory resources of single processors are small, the system performance cannot be affected, and the user use friendliness is improved.
Description
Technical field
The present invention relates to a kind of data export method.
Background technology
Excel export function is the function of the most basic and frequent application that provides of various information system, processes large-scale data and derives general the employing in batches of business and derive, and the too much traffic handling time of this method consumption, affect carrying out that follow-up business applies.Another kind of extreme mode is " once derive, continue to wait for ", and as shown in Figure 1, the method committed memory resource is excessive, causes system performance to decline, and user uses friendliness to decline.
Summary of the invention
The technical assignment of the technical problem to be solved in the present invention and proposition is that prior art scheme is improved and improved, and a kind of big data quantity Excel file deriving method based on asynchronous process is provided, to reach the object that improves processing speed and reduce memory source.For this reason, the present invention takes following technical scheme.
A big data quantity Excel file deriving method based on asynchronous process, is characterized in that comprising the following steps:
1) Excel data are divided into some data groups by Excel assistance application module;
2) in the time that application system sends Excel data derivation request to Excel assistance application module, Excel assistance application module proposes Excel data to multiple processors and derives requests;
3) each processor is processed Excel data derives while request, and all newly-built Excel files in designated store path, by accessing database, read record and write one by one in corresponding Excel file;
4) multiple processors form multiple Excel files under the store path of specifying; The complete Excel file that multiple Excel file stacks form, user checks and downloads the Excel file of generation under store path.
The asynchronous derivation of the technical program employing working mechanism, the data query and the file generated process that Excel data are derived to business are discrete, the file generated that takies larger time and internal memory is transferred to backstage Excel assistance application resume module, and in application, independently open up functional area file download service is provided.
As the further of technique scheme improved and supplemented, the present invention also comprises following additional technical feature.
Application system sends Excel data by Webservice interface to Excel assistance application module and derives request.
Excel data are divided into the data group that quantity is identical with processor by Excel assistance application module, and a processor is derived the data of a data group, all data of multiple processor asynchronous derivation Excel files.
Beneficial effect: the asynchronous derivation of the technical program employing working mechanism, the data query and the file generated process that Excel data are derived to business are discrete, the file generated that takies larger time and internal memory is transferred to backstage Excel supplementary module processing, and in application, independently open up functional area file download service is provided; Traffic handling time is short, is conducive to carrying out of follow-up business application, and it is little to take the memory source of single processor, and system performance is unaffected, improves user and uses friendliness.
Accompanying drawing explanation
Fig. 1 is existing Excel data export method.
Fig. 2 is process flow diagram of the present invention.
Fig. 3 is principle assumption diagram of the present invention.
Embodiment
Below in conjunction with Figure of description, technical scheme of the present invention is described in further detail.
As shown in Figure 2, the technical program adopts asynchronous call mode, and it returns to immediately master routine and carry out next step after response call request, and concrete business operation is put to running background.Replace synchronization call mode as shown in Figure 1, after step method of calling need wait allocating object to complete concrete business operation, just can return to master routine and carry out next step; The technical program guarantees the runnability of system better, and effectively shortens the processing time.
As shown in Figure 3, the present invention includes following steps:
1) Excel data are divided into some data groups by Excel assistance application module;
2) in the time that application system sends Excel data derivation request to Excel assistance application module, Excel assistance application module proposes Excel data to multiple processors and derives requests;
3) each processor is processed Excel data derives while request, and all newly-built Excel files in designated store path, by accessing database, read record and write one by one in corresponding Excel file;
4) multiple processors form multiple Excel files under the store path of specifying; The complete Excel file that multiple Excel file stacks form, user checks and downloads the Excel file of generation under store path.
Wherein, application system sends Excel data by Webservice interface to Excel assistance application module and derives request.For reducing the complicacy of structure, Excel data are divided into the data group that quantity is identical with processor by Excel assistance application module, and a processor is derived the data of a data group, all data of multiple processor asynchronous derivation Excel files.
A kind of big data quantity Excel file deriving method based on asynchronous process shown in above Fig. 2,3 is specific embodiments of the invention; substantive distinguishing features of the present invention and progress are embodied; can be according to actual use needs; under enlightenment of the present invention; it is carried out to the equivalent modifications of the aspects such as shape, structure, all at the row of the protection domain of this programme.
Claims (3)
1. the big data quantity Excel file deriving method based on asynchronous process, is characterized in that comprising the following steps:
Excel data are divided into some data groups by Excel assistance application module;
In the time that application system sends Excel data derivation request to Excel assistance application module, Excel assistance application module proposes Excel data to multiple processors and derives requests;
Each processor is processed Excel data and is derived while request, and all newly-built Excel files in designated store path, by accessing database, read record and write one by one in corresponding Excel file;
Multiple processors form multiple Excel files under the store path of specifying; The complete Excel file that multiple Excel file stacks form, user checks and downloads the Excel file of generation under store path.
2. a kind of big data quantity Excel file deriving method based on asynchronous process according to claim 2, is characterized in that: application system sends Excel data by Webservice interface to Excel assistance application module and derives request.
3. a kind of big data quantity Excel file deriving method based on asynchronous process according to claim 2, it is characterized in that: Excel data are divided into the data group that quantity is identical with processor by Excel assistance application module, a processor is derived the data of a data group, all data of multiple processor asynchronous derivation Excel files.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310584833.2A CN103810238A (en) | 2013-11-20 | 2013-11-20 | Large-data-volume Excel file exporting method based on asynchronous processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310584833.2A CN103810238A (en) | 2013-11-20 | 2013-11-20 | Large-data-volume Excel file exporting method based on asynchronous processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103810238A true CN103810238A (en) | 2014-05-21 |
Family
ID=50707008
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310584833.2A Pending CN103810238A (en) | 2013-11-20 | 2013-11-20 | Large-data-volume Excel file exporting method based on asynchronous processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103810238A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105119763A (en) * | 2015-09-24 | 2015-12-02 | 烽火通信科技股份有限公司 | RIA-based Web network management client big data rapid export method and system |
| CN105306596A (en) * | 2015-11-19 | 2016-02-03 | 中国建设银行股份有限公司 | File processing method, server and system applied in electronic commerce system |
| CN106341457A (en) * | 2016-08-25 | 2017-01-18 | 广州御银自动柜员机科技有限公司 | Device for asynchronously exporting data from cloud platform |
| CN107153717A (en) * | 2017-06-14 | 2017-09-12 | 成都三零凯天通信实业有限公司 | Million grades of method and system that recorded form are exported from database by browser |
| CN107526635A (en) * | 2017-09-05 | 2017-12-29 | 成都知道创宇信息技术有限公司 | A kind of asynchronous deriving method of data |
| CN107656968A (en) * | 2017-08-31 | 2018-02-02 | 武汉斗鱼网络科技有限公司 | High-volume business datum deriving method and system |
| CN107704442A (en) * | 2017-03-31 | 2018-02-16 | 广东网金控股股份有限公司 | A kind of excel deriving methods and device based on data safety |
| CN111104448A (en) * | 2019-12-17 | 2020-05-05 | 深圳前海环融联易信息科技服务有限公司 | Method and device for exporting large-data-volume Excel file, computer equipment and storage medium |
| CN112287006A (en) * | 2020-10-27 | 2021-01-29 | 上海淇馥信息技术有限公司 | Electronic form data exporting method and device and electronic equipment |
| CN112347179A (en) * | 2020-11-30 | 2021-02-09 | 中国农业银行股份有限公司 | Big data export method and device based on Excel customized template |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050177573A1 (en) * | 2000-07-21 | 2005-08-11 | Microsoft Corporation | Integrated method for creating a refreshable web query |
| CN101035025A (en) * | 2007-03-30 | 2007-09-12 | 华为技术有限公司 | Network device management method and system |
| CN101231665A (en) * | 2008-02-20 | 2008-07-30 | 中兴通讯股份有限公司 | Apparatus and method for educing data into Excel files from database |
| US20090300061A1 (en) * | 2007-12-03 | 2009-12-03 | Infosys Technologies Ltd. | System and method for universe generation |
| CN101609470A (en) * | 2009-07-02 | 2009-12-23 | 腾讯科技（深圳）有限公司 | A kind of preventing apparent death of multi-page browser method and device |
| US20100174571A1 (en) * | 2009-01-04 | 2010-07-08 | Sandra Lee Jerome | Open Dealership Management System developed in MySQL, PHP, HTML, and JavaScript |
| CN101996067A (en) * | 2009-08-19 | 2011-03-30 | 阿里巴巴集团控股有限公司 | Data export method and device |
| CN102867071A (en) * | 2012-10-19 | 2013-01-09 | 烽火通信科技股份有限公司 | Management method for massive network management historical data |
-
2013
- 2013-11-20 CN CN201310584833.2A patent/CN103810238A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050177573A1 (en) * | 2000-07-21 | 2005-08-11 | Microsoft Corporation | Integrated method for creating a refreshable web query |
| CN101035025A (en) * | 2007-03-30 | 2007-09-12 | 华为技术有限公司 | Network device management method and system |
| US20090300061A1 (en) * | 2007-12-03 | 2009-12-03 | Infosys Technologies Ltd. | System and method for universe generation |
| CN101231665A (en) * | 2008-02-20 | 2008-07-30 | 中兴通讯股份有限公司 | Apparatus and method for educing data into Excel files from database |
| US20100174571A1 (en) * | 2009-01-04 | 2010-07-08 | Sandra Lee Jerome | Open Dealership Management System developed in MySQL, PHP, HTML, and JavaScript |
| CN101609470A (en) * | 2009-07-02 | 2009-12-23 | 腾讯科技（深圳）有限公司 | A kind of preventing apparent death of multi-page browser method and device |
| CN101996067A (en) * | 2009-08-19 | 2011-03-30 | 阿里巴巴集团控股有限公司 | Data export method and device |
| CN102867071A (en) * | 2012-10-19 | 2013-01-09 | 烽火通信科技股份有限公司 | Management method for massive network management historical data |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105119763A (en) * | 2015-09-24 | 2015-12-02 | 烽火通信科技股份有限公司 | RIA-based Web network management client big data rapid export method and system |
| CN105119763B (en) * | 2015-09-24 | 2018-12-28 | 烽火通信科技股份有限公司 | The quick deriving method of the big data of Web network management client based on RIA and system |
| CN105306596A (en) * | 2015-11-19 | 2016-02-03 | 中国建设银行股份有限公司 | File processing method, server and system applied in electronic commerce system |
| CN106341457A (en) * | 2016-08-25 | 2017-01-18 | 广州御银自动柜员机科技有限公司 | Device for asynchronously exporting data from cloud platform |
| CN107704442A (en) * | 2017-03-31 | 2018-02-16 | 广东网金控股股份有限公司 | A kind of excel deriving methods and device based on data safety |
| CN107153717A (en) * | 2017-06-14 | 2017-09-12 | 成都三零凯天通信实业有限公司 | Million grades of method and system that recorded form are exported from database by browser |
| CN107656968A (en) * | 2017-08-31 | 2018-02-02 | 武汉斗鱼网络科技有限公司 | High-volume business datum deriving method and system |
| CN107656968B (en) * | 2017-08-31 | 2021-04-23 | 武汉斗鱼网络科技有限公司 | Method and system for exporting large-batch business data |
| CN107526635A (en) * | 2017-09-05 | 2017-12-29 | 成都知道创宇信息技术有限公司 | A kind of asynchronous deriving method of data |
| CN111104448A (en) * | 2019-12-17 | 2020-05-05 | 深圳前海环融联易信息科技服务有限公司 | Method and device for exporting large-data-volume Excel file, computer equipment and storage medium |
| CN112287006A (en) * | 2020-10-27 | 2021-01-29 | 上海淇馥信息技术有限公司 | Electronic form data exporting method and device and electronic equipment |
| CN112347179A (en) * | 2020-11-30 | 2021-02-09 | 中国农业银行股份有限公司 | Big data export method and device based on Excel customized template |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103810238A (en) | Large-data-volume Excel file exporting method based on asynchronous processing | |
| CN110166282B (en) | Resource allocation method, device, computer equipment and storage medium | |
| CN103856567A (en) | Small file storage method based on Hadoop distributed file system | |
| CN104407879B (en) | A kind of power network sequential big data loaded in parallel method | |
| CN101866359B (en) | Small file storage and visit method in avicade file system | |
| WO2019033605A1 (en) | Method and apparatus for asynchronous order processing, and storage medium and terminal | |
| CN101593202B (en) | Hash connecting method for database based on shared Cache multicore processor | |
| CN102663115B (en) | Main memory database access optimization method on basis of page coloring technology | |
| CN103677759A (en) | Objectification parallel computing method and system for information system performance improvement | |
| CN103701920A (en) | Method for configuring virtual application server under cloud environment | |
| CN103902593A (en) | Data transfer method and device | |
| CN104572505B (en) | System and method for ensuring eventual consistency of mass data caches | |
| CN110874271B (en) | Method and system for rapidly calculating mass building pattern spot characteristics | |
| CN101373445A (en) | Method and apparatus for scheduling memory | |
| CN103116475B (en) | Method of automatic simplifying allocation expansion | |
| CN106201918B (en) | A kind of method and system based on big data quantity and extensive caching quick release | |
| CN109271409B (en) | Database fragmentation execution method based on container resource allocation | |
| CN103218254B (en) | A kind of AFC (Automatic Fare Collection) data processing method of many queues | |
| CN112000703B (en) | Data warehousing processing method and device, computer equipment and storage medium | |
| CN114297002A (en) | Mass data backup method and system based on object storage | |
| CN102609364A (en) | Cache exchange method | |
| CN102768672A (en) | Disk space management method and device | |
| KR101830504B1 (en) | In-Memory DB Connection Support Type Scheduling Method and System for Real-Time Big Data Analysis in Distributed Computing Environment | |
| CN101609449A (en) | A kind of fast comparison system for data blocks based on Bloom Filter | |
| CN102195361B (en) | Method for acquiring and processing data of intelligent distribution terminal of multi-core single chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140521 |
|
| RJ01 | Rejection of invention patent application after publication |