Connect public, paid and private patent data with Google Patents Public Datasets

Large-data-volume Excel file exporting method based on asynchronous processing

Download PDF

Info

Publication number
CN103810238A
Authority
CN
Grant status
Application
Patent type
Prior art keywords
excel
data
exporting
method
processing
Prior art date
Application number
CN 201310584833
Other languages
Chinese (zh)
Inventor
陈顺德
王红凯
李熊
裘炜浩
王志强
洪建光
裴旭斌
沈志豪
蒋锦霞
肖政
陈清泰
Original Assignee
国家电网公司
国网浙江省电力公司信息通信分公司
浙江创维自动化工程有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/30286Information retrieval; Database structures therefor ; File system structures therefor in structured data stores
    • G06F17/30557Details of integrating or interfacing systems involving at least one database management system
    • G06F17/30569Details of data format conversion from or to a database
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/30286Information retrieval; Database structures therefor ; File system structures therefor in structured data stores
    • G06F17/30575Replication, distribution or synchronisation of data between databases or within a distributed database; Distributed database system architectures therefor
    • G06F17/30578Details of asynchronous replication and data reconciliation

Abstract

The invention relates to a data exporting method, particularly to a large-data-volume Excel file exporting method based on asynchronous processing. At present, an exporting method for processing massive data occupies overlarge memory resources. The method comprises the steps as follows: an Excel auxiliary application module divides Excel data into multiple data sets; when an application system sends an Excel data exporting request to the Excel auxiliary application module, the Excel auxiliary application module makes Excel data exporting requests to a plurality of processors; each processor builds a new Excel file in a designated storage path when processing the Excel data exporting request; and a plurality of Excel files are overlaid to form a complete Excel file, and a user checks and downloads the generated Excel file in the storage path. According to the technical scheme, the service processing time is short, occupied memory resources of single processors are small, the system performance cannot be affected, and the user use friendliness is improved.

Description

一种基于异步处理的大数据量Excel文件导出方法 Deduction Method based on asynchronous processing of large amounts of data Excel file

技术领域 FIELD

[0001] 本发明涉及一种数据导出方法。 [0001] The present invention relates to a method for deriving data.

背景技术 Background technique

[0002] Excel导出功能是各类信息系统提供的最为基本且频繁应用的功能,处理大规模数据导出业务一般采用分批次导出,这种方法消耗过多业务处理时间,影响后续业务应用的开展。 [0002] Excel export functions are functions provided by various information systems most basic and frequent applications, large-scale data processing export business is generally used to export batches, this method consumes too much time processing business, the impact of follow-up business applications . 另一种极端的方式是“一次导出,持续等待”,如图1所示,该方法占用内存资源过大,导致系统性能下降,且用户使用友好度下降。 At the other extreme approach is "one export, continued to wait", as shown in Figure 1, which take up too much memory resources, resulting in system performance and user friendliness decline.

发明内容 SUMMARY

[0003] 本发明要解决的技术问题和提出的技术任务是对现有技术方案进行完善与改进,提供一种基于异步处理的大数据量Excel文件导出方法,以达到提高处理速度及降低内存资源的目的。 [0003] The present invention to solve the technical problems and proposed technical task is to improve existing technical solutions and improvements to provide a method based on the large amount of data export Excel files asynchronous processing, to achieve improved processing speed and reduce memory resources the goal of. 为此,本发明采取以下技术方案。 To this end, the present invention takes the following technical solutions.

[0004] 一种基于异步处理的大数据量Excel文件导出方法,其特征在于包括以下步骤: DExcel辅助应用模块将Excel数据分成若干数据组; [0004] Based on the large data processing asynchronous Excel file export method, comprising the steps of: DExcel assistance application module Excel data into several data sets;

2)当应用系统向Excel辅助应用模块发送Excel数据导出请求时,Excel辅助应用模块向多个处理器提出Excel数据导出请求; 2) When the system is applied to the transmission request data export Excel Excel auxiliary application module, Excel Excel proposed secondary application module data export request to the plurality of processors;

3)每个处理器处理Excel数据导出请求时,都在指定存储路径新建一个Excel文件,通过访问数据库,读取记录逐条写入相应Excel文件中; 3) Each processor Excel data export request, a new Excel file are specified storage path, by accessing the database, reading the recorded one by writing the appropriate Excel file;

4)多个处理器在指定的存储路径下形成多个Excel文件;多个Excel文件叠加构成的完整的Excel文件,用户在存储路径下查看并下载生成的Excel文件。 4) a plurality of processors are formed at a plurality of specified storage path Excel files; Excel files superimposing a plurality of files constituting the complete Excel, Excel users to view and download files generated in storage path.

[0005] 本技术方案采用异步导出工作机制,将Excel数据导出业务的数据查询和文件生成过程分立,将占用较大时间和内存的文件生成交由后台Excel辅助应用模块处理,并在应用中独立开辟功能区域提供文件下载服务。 [0005] The present technical solution derived using asynchronous mechanism, the Excel file and data export data query generation process operations separate, will take up more time and memory auxiliary Excel file generating application referred to the background processing module, and in the application of independent open function area providing a file download service.

[0006] 作为对上述技术方案的进一步完善和补充,本发明还包括以下附加技术特征。 [0006] As a further supplement and improve the above technical solution, the present invention further comprises the following additional features.

[0007] 应用系统通过Webservice接口向Excel辅助应用模块发送Excel数据导出请求。 [0007] Excel application transmitting request data export via an interface to an Excel Webservice secondary application module.

[0008] Excel辅助应用模块将Excel数据分成数量与处理器相同的数据组,一个处理器导出一个数据组的数据,多个处理器异步导出Excel文件的所有数据。 [0008] Applications module Excel Excel auxiliary data into the same number of sets of data processor, a data processor deriving a data set, the plurality of data processors asynchronously export all Excel files.

[0009] 有益效果:本技术方案采用异步导出工作机制,将Excel数据导出业务的数据查询和文件生成过程分立,将占用较大时间和内存的文件生成交由后台Excel辅助模块处理,并在应用中独立开辟功能区域提供文件下载服务;业务处理时间短,有利于后续业务应用的开展,且占用单个处理器的内存资源小,系统性能不受影响,提高用户使用友好度。 [0009] beneficial effects: the technical solution using asynchronous export mechanism, data export to Excel data query and document generation process business division, will take up more time and memory are referred to the background document generation Excel auxiliary module processes and applications small business processing time is short, is conducive to follow-up business applications, and occupies a single processor, memory resources, system performance is not affected, increase user friendliness; file provides independent functional areas open up download service.

附图说明 BRIEF DESCRIPTION

[0010] 图1是现有的Excel数据导出方法。 [0010] FIG. 1 is a conventional method of deriving data Excel.

[0011] 图2是本发明的流程图。 [0011] FIG 2 is a flowchart of the present invention. [0012] 图3是本发明的原理结构图。 [0012] FIG. 3 is a schematic structural view of the present invention.

具体实施方式 detailed description

[0013] 以下结合说明书附图对本发明的技术方案做进一步的详细说明。 [0013] The following description in conjunction with the accompanying drawings described in detail further aspect of the present invention.

[0014] 如图2所示,本技术方案采用异步调用方式,其在响应调用请求后立即返回主程序并执行下一步骤,而将具体业务操作放至后台运行。 [0014] As shown in FIG 2, the present technical scheme adopts asynchronous mode call, which returns the main routine immediately after the call request and in response to the next step, and the discharge to the specific operational background. 取代如图1所示的同步调用方式,步调用方式需等调用对象完成具体的业务操作后才能返回主程序并执行下一步骤;本技术方案更好地保证系统的运行性能,且有效缩短处理时间。 Substituted synchronous call shown in FIG. 1, step returns to the main routine calling for an object with the other embodiment the specific operational and complete the next step; aspect of the present system to ensure better operating performance, and effectively shorten the treatment time.

[0015] 如图3所示,本发明包括以下步骤: [0015] As shown in FIG. 3, the present invention comprises the steps of:

DExcel辅助应用模块将Excel数据分成若干数据组; DExcel assistance application module Excel data into several data sets;

2)当应用系统向Excel辅助应用模块发送Excel数据导出请求时,Excel辅助应用模块向多个处理器提出Excel数据导出请求; 2) When the system is applied to the transmission request data export Excel Excel auxiliary application module, Excel Excel proposed secondary application module data export request to the plurality of processors;

3)每个处理器处理Excel数据导出请求时,都在指定存储路径新建一个Excel文件,通过访问数据库,读取记录逐条写入相应Excel文件中; 3) Each processor Excel data export request, a new Excel file are specified storage path, by accessing the database, reading the recorded one by writing the appropriate Excel file;

4)多个处理器在指定的存储路径下形成多个Excel文件;多个Excel文件叠加构成的完整的Excel文件,用户在存储路径下查看并下载生成的Excel文件。 4) a plurality of processors are formed at a plurality of specified storage path Excel files; Excel files superimposing a plurality of files constituting the complete Excel, Excel users to view and download files generated in storage path.

[0016] 其中,应用系统通过Webservice接口向Excel辅助应用模块发送Excel数据导出请求。 [0016] wherein the application system by sending a request Webservice Excel data export an interface to the secondary application module Excel. 为降低结构的复杂性,Excel辅助应用模块将Excel数据分成数量与处理器相同的数据组,一个处理器导出一个数据组的数据,多个处理器异步导出Excel文件的所有数据。 To reduce the complexity of the structure, the secondary application module Excel Excel data into the same number of sets of data processor, a data processor deriving a data set, the plurality of data processors asynchronously export all Excel files.

[0017] 以上图2、3所示的一种基于异步处理的大数据量Excel文件导出方法是本发明的具体实施例,已经体现出本发明实质性特点和进步,可根据实际的使用需要,在本发明的启示下,对其进行形状、结构等方面的等同修改,均在本方案的保护范围之列。 One kind of [0017] FIG. 2 and 3 based on the above large data Excel files asynchronous processing method of deriving specific embodiments of the present invention, it has demonstrated the substantive features and advances of the present invention, according to the actual needs of use, in light of the present invention, its equivalent modifications aspect shape, structure, etc., are the scope of protection of the present scheme in a column.

Claims (3)

1.一种基于异步处理的大数据量Excel文件导出方法,其特征在于包括以下步骤: Excel辅助应用模块将Excel数据分成若干数据组; 当应用系统向Excel辅助应用模块发送Excel数据导出请求时,Excel辅助应用模块向多个处理器提出Excel数据导出请求; 每个处理器处理Excel数据导出请求时,都在指定存储路径新建一个Excel文件,通过访问数据库,读取记录逐条写入相应Excel文件中; 多个处理器在指定的存储路径下形成多个Excel文件;多个Excel文件叠加构成的完整的Excel文件,用户在存储路径下查看并下载生成的Excel文件。 CLAIMS 1. A method of deriving mass data based on asynchronous processing Excel file, comprising the steps of: Excel Excel auxiliary module application data into a plurality of data groups; an Excel data export request when the auxiliary Excel application sends the application module, Excel Excel proposed application module auxiliary data export request to the plurality of processors; Excel data processor processes each export request, a new Excel file are specified storage path, by accessing the database, reading the recorded one by writing the appropriate Excel file ; a plurality of processors are formed at a plurality of specified storage path Excel files; Excel files superimposing a plurality of files constituting the complete Excel, Excel users to view and download files generated in storage path.
2.根据权利要求2所述的一种基于异步处理的大数据量Excel文件导出方法,其特征在于:应用系统通过Webservice接口向Excel辅助应用模块发送Excel数据导出请求。 According to one of the claim 2 derived based on a large amount of data Excel files asynchronous processing, wherein: send application data to Excel Excel application module via the secondary interface to Webservice export request.
3.根据权利要求2所述的一种基于异步处理的大数据量Excel文件导出方法,其特征在于=Excel辅助应用模块将Excel数据分成数量与处理器相同的数据组,一个处理器导出一个数据组的数据,多个处理器异步导出Excel文件的所有数据。 According to one of the claim 2 derived based on a large amount of data Excel files asynchronous processing, characterized in that the secondary application module = Excel Excel data into the same number of sets of data processor, a processor, a data exporting data sets, multiple asynchronous processors to export all data in Excel files.
CN 201310584833 2013-11-20 2013-11-20 Large-data-volume Excel file exporting method based on asynchronous processing CN103810238A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201310584833 CN103810238A (en) 2013-11-20 2013-11-20 Large-data-volume Excel file exporting method based on asynchronous processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201310584833 CN103810238A (en) 2013-11-20 2013-11-20 Large-data-volume Excel file exporting method based on asynchronous processing

Publications (1)

Publication Number Publication Date
CN103810238A true true CN103810238A (en) 2014-05-21

Family

ID=50707008

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201310584833 CN103810238A (en) 2013-11-20 2013-11-20 Large-data-volume Excel file exporting method based on asynchronous processing

Country Status (1)

Country Link
CN (1) CN103810238A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105119763A (en) * 2015-09-24 2015-12-02 烽火通信科技股份有限公司 RIA-based Web network management client big data rapid export method and system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050177573A1 (en) * 2000-07-21 2005-08-11 Microsoft Corporation Integrated method for creating a refreshable web query
CN101231665A (en) * 2008-02-20 2008-07-30 中兴通讯股份有限公司 Apparatus and method for educing data into Excel files from database
US20090300061A1 (en) * 2007-12-03 2009-12-03 Infosys Technologies Ltd. System and method for universe generation
US20100174571A1 (en) * 2009-01-04 2010-07-08 Sandra Lee Jerome Open Dealership Management System developed in MySQL, PHP, HTML, and JavaScript
CN101996067A (en) * 2009-08-19 2011-03-30 阿里巴巴集团控股有限公司 Data export method and device
CN102867071A (en) * 2012-10-19 2013-01-09 烽火通信科技股份有限公司 Management method for massive network management historical data

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050177573A1 (en) * 2000-07-21 2005-08-11 Microsoft Corporation Integrated method for creating a refreshable web query
US20090300061A1 (en) * 2007-12-03 2009-12-03 Infosys Technologies Ltd. System and method for universe generation
CN101231665A (en) * 2008-02-20 2008-07-30 中兴通讯股份有限公司 Apparatus and method for educing data into Excel files from database
US20100174571A1 (en) * 2009-01-04 2010-07-08 Sandra Lee Jerome Open Dealership Management System developed in MySQL, PHP, HTML, and JavaScript
CN101996067A (en) * 2009-08-19 2011-03-30 阿里巴巴集团控股有限公司 Data export method and device
CN102867071A (en) * 2012-10-19 2013-01-09 烽火通信科技股份有限公司 Management method for massive network management historical data

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105119763A (en) * 2015-09-24 2015-12-02 烽火通信科技股份有限公司 RIA-based Web network management client big data rapid export method and system

Similar Documents

Publication Publication Date Title
Seltzer et al. File system logging versus clustering: A performance comparison
Lidl et al. Drinking from the firehose: Multicast USENET news
Gu et al. SHadoop: Improving MapReduce performance by optimizing job execution mechanism in Hadoop clusters
Tiwari et al. Active flash: towards energy-efficient, in-situ data analytics on extreme-scale machines.
CN102708088A (en) CPU/GPU (Central Processing Unit/ Graphic Processing Unit) cooperative processing method oriented to mass data high-performance computation
Wu et al. High performance implementation of MPI derived datatype communication over InfiniBand
CN1642169A (en) Management system and method for large service system based on network storage and resource virtual process
CN102467637A (en) Anti-virus system under virtualization environment and anti-virus method thereof
CN101719149A (en) Data synchronization method and device
CN101916280A (en) Parallel computing system and method for carrying out load balance according to query contents
CN102906751A (en) Method and device for data storage and data query
CN102821111A (en) Real-time synchronizing method for file cloud storage
CN102214236A (en) Method and system for processing mass data
CN102142032A (en) Method and system for reading and writing data of distributed file system
CN101510203A (en) Big data quantity high performance processing implementing method based on parallel process of split mechanism
CN102857570A (en) Cloud synchronized method of files and cloud storage server
CN103166785A (en) Distributed type log analysis system based on Hadoop
US20120311589A1 (en) Systems and methods for processing hierarchical data in a map-reduce framework
CN102523204A (en) Quality of service (QoS)-oriented virtual machine network bandwidth allocation system
Alvaro et al. Blazes: Coordination analysis for distributed programs
Madhavapeddy et al. Multiscale not multicore: Efficient heterogeneous cloud computing
CN102202242A (en) Realization method of JavaScript interpreter based on set-top box browser
CN102800015A (en) Financial IC (Integrated Circuit) card long-distance individualized method and system
Madhavapeddy et al. Turning Down the LAMP: Software Specialisation for the Cloud.
CN101620634A (en) Data statistics device and data statistics method

Legal Events

Date Code Title Description
C06 Publication
EXSB Decision made by sipo to initiate substantive examination