WO2014183723A1 - 基于表格实现大数据量处理的方法及装置 - Google Patents

基于表格实现大数据量处理的方法及装置 Download PDF

Info

Publication number
WO2014183723A1
WO2014183723A1 PCT/CN2014/079793 CN2014079793W WO2014183723A1 WO 2014183723 A1 WO2014183723 A1 WO 2014183723A1 CN 2014079793 W CN2014079793 W CN 2014079793W WO 2014183723 A1 WO2014183723 A1 WO 2014183723A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
memory database
tables
large amounts
function
Prior art date
Application number
PCT/CN2014/079793
Other languages
English (en)
French (fr)
Chinese (zh)
Inventor
陈波
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to KR1020167010536A priority Critical patent/KR101777073B1/ko
Priority to JP2016528157A priority patent/JP6530390B2/ja
Publication of WO2014183723A1 publication Critical patent/WO2014183723A1/zh

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2219Large Object storage; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases

Definitions

  • the present invention relates to the field of computer data processing technologies, and in particular, to a method and apparatus for implementing large data volume processing based on a table. Background technique
  • a telecommunication network management system usually needs to implement processing of a large amount of data.
  • the so-called large amount of data that is, the amount of data displayed far exceeds the capacity that can be supported by the memory
  • the report management in the network management system is taken as an example to illustrate that the report management is
  • the system administrator provides a unified report output function for important information of the network management system.
  • the report management function the system administrator can customize the report task and output the report data, which is convenient for monitoring the operation of the network management system.
  • the report management system is generally used to output the corresponding form. Process such data.
  • the functions provided by the prior art are relatively simple. If complex typographic functions are to be implemented on the basis of such a form, complex secondary development is required, but the secondary development of the form can realize complex typographic functions.
  • the so-called complex typesetting refers to the typesetting functions such as global real-time sorting, line number display, column freezing, column heading automatic line-feeding display, blank column filtering, and the like.
  • it is difficult to meet the requirements of complex typographic functions such as real-time sorting.
  • the problem of the present invention is to provide a method and apparatus for implementing large data volume processing based on a table.
  • a method for implementing large data volume processing based on a table comprising:
  • At least one large data volume operation command to the table is identified, and a large amount of data is obtained from the in-memory database and presented through the table in accordance with the large data volume operation command.
  • the method further includes: indexing the data in the in-memory database according to the row number of the table, and when the data needs to be displayed, according to the row number of the table to the memory The corresponding data is read in the database.
  • the redeveloped table has a column header sorting function; the column header sorting function is configured to bind the redeveloped table to an in-memory database for sorting a result set .
  • the method further includes: first buffering the large data volume to an intermediate file, and then presenting the file according to the intermediate file. Large amount of data.
  • the method further comprises: identifying at least one function customization command, and performing corresponding adding or deleting operations on the complex typesetting function supported by the table according to the function customization command.
  • An apparatus for implementing large data volume processing based on a table comprising:
  • a deployment module configured to deploy a secondary developed form that implements complex typographic functions
  • an association module configured to establish an association relationship between the form and the in-memory database
  • An identification module configured to identify at least one large data volume operation command for the form
  • the obtaining module is configured to obtain a large amount of data from the in-memory database according to the large data volume operation command;
  • a rendering module configured to present a large amount of data retrieved from the in-memory database through the table.
  • the association module is configured to index data in the in-memory database according to the row number of the table, so that when the data needs to be displayed, the obtaining module reads the corresponding data into the in-memory database according to the row number of the table.
  • the redeveloped table has a column header sorting function; the column header sorting function is configured to bind the redeveloped table to an in-memory database for sorting the result set.
  • the obtaining module is configured to cache the large amount of data to an intermediate file after acquiring a large amount of data from the in-memory database according to a large data operation command, and the rendering module is configured on the form according to the intermediate file. Present a large amount of data.
  • the device for implementing large data volume processing based on a table further includes:
  • the customization module is configured to identify at least one function customization command, and according to the function customization command, perform corresponding addition or deletion operations on the complex layout function supported by the table.
  • the method and device for realizing large data volume processing based on a table provided by the present invention can realize secondary development of a complex typesetting function by using a table, and the secondary developed form It is bound to the in-memory database.
  • the table can support complex layout, global real-time sorting, and large data volume presentation.
  • FIG. 1 is a schematic flowchart of a method for implementing large data volume processing based on a table according to an embodiment of the present invention
  • FIG. 2 is a flow chart of a method for implementing large data volume processing based on a table according to another embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of an apparatus for implementing large data volume processing based on a table according to an embodiment of the present invention
  • FIG. 4 is a schematic structural diagram of an apparatus for implementing large data volume processing based on a table according to another embodiment of the present invention.
  • a method for implementing large data volume processing based on a table includes the following steps:
  • the complex typesetting function includes, but is not limited to, a global real-time sorting, a line number display, a column freeze, a column header automatic line feed display, and a blank column filter.
  • the pre-developed secondary development of the table may be performed according to the prior art that is known to those skilled in the art, and will not be described in detail herein.
  • the redeveloped table has a column head sorting function, and the column head sorting function is used to bind the redeveloped table and the in-memory database for sorting the result set.
  • the ordering operation instruction may be an Order by operation. This allows you to click on the header to perform global sorting.
  • the association relationship between the establishment table and the in-memory database may be an association relationship between the display logic of the table and the in-memory database.
  • the method may further include: indexing the data in the in-memory database according to the row number of the table, and reading the corresponding data according to the row number of the table to the in-memory database when the data needs to be displayed. .
  • the large data volume operation command is a variety of complex typesetting operation commands for the table input by the user.
  • the large The amount of data is cached to the intermediate file, and then the large amount of data is presented on the form according to the intermediate file, so that when the large amount of data is presented through the table and the complex typesetting function is performed on the large amount of data, the memory usage can be reduced, and the large size can be realized.
  • Data volume presentation and operation After obtaining a large amount of data from the in-memory database according to a large data operation command, the large The amount of data is cached to the intermediate file, and then the large amount of data is presented on the form according to the intermediate file, so that when the large amount of data is presented through the table and the complex typesetting function is performed on the large amount of data, the memory usage can be reduced, and the large size can be realized.
  • the method for implementing large data volume processing based on a table further includes:
  • the function customization command can add or delete column freeze, column width, header order, column hiding, etc. by the user clicking the right mouse button and selecting the corresponding custom menu input.
  • Typesetting function In other embodiments, the typesetting of the functions such as column freeze, column width, and header order can also be performed by the user directly dragging the header of the table, which is not described in detail herein.
  • the method for realizing large data volume processing based on the table provided by the embodiment of the present invention realizes column freezing by secondary development of the table; secondary development of the header, supporting column headings and sorting arrow display; and displaying the logic of the table
  • the in-memory database is bound, and when the specific data needs to be displayed, the records are read according to the line number to the in-memory database; the function of sorting the table column header is bound with the Order by operation of the in-memory database, so as to implement the click header globally. Sorting function; Cache in-memory database data into files, reduce memory usage, and realize large data volume rendering and operation. Therefore, the table provided by the embodiment of the present invention can simultaneously support functions of complex typesetting, global real-time sorting, and large data volume presentation.
  • an apparatus for implementing large data volume processing based on a table includes:
  • the deployment module 10 is configured to deploy a re-developed table that implements complex typographic functions; the association module 20 is configured to establish an association relationship between the display logic of the table and the in-memory database 40; and the identification module 30 is configured to identify the pair of tables At least one large data volume operation command 300;
  • the obtaining module 50 is configured to obtain a large amount of data from the in-memory database 40 according to the large data volume operation command 300;
  • a presentation module 60 configured to use the table to obtain a large amount of data obtained from the in-memory database 40 Presented.
  • the complex typesetting function includes, but is not limited to, a global real-time sorting, a line number display, a column freeze, a column header automatic line feed display, and a blank column filter.
  • the pre-developed secondary development of the form can be performed in accordance with the prior art already known to those skilled in the art.
  • the association module 20 is further configured to index the data in the in-memory database 40 according to the row number of the table, so that when the data needs to be displayed, the obtaining module 50 goes to the in-memory database according to the row number of the table. The corresponding data is read in 40.
  • the large data volume operation command 300 recognized by the identification module 30 is a user-entered various complex typesetting operation commands for the table.
  • the column header sorting function of the redeveloped table is configured to be bound to the Order by operation of the in-memory database 40 for sorting the result set, thereby implementing the click header For global sorting.
  • the obtaining module 50 caches the large data amount to an intermediate file (not shown),
  • the presentation module 60 presents a large amount of data on the form in accordance with the intermediate file. In this way, when a large amount of data is presented through a table and a complex typesetting function is performed on a large amount of data, the memory usage can be reduced, and a large amount of data can be presented and operated.
  • the apparatus for implementing large data volume processing based on a table further includes:
  • the customization module 70 is configured to identify at least one function customization command 700, and according to the function customization command 700, perform corresponding addition or deletion operations on the complex typesetting function supported by the table.
  • the customization module 70 can add or delete a column freeze, a column width, and a table by identifying at least one user-entered function customization command 700, for example, the function customization command 700 can be added or deleted by the user clicking the right mouse button and selecting a corresponding custom menu input.
  • Complex typographic features such as head order, column hiding, etc.
  • the typesetting of the column freeze, the column width, the header order, and the like may also be performed by the user directly dragging the header of the table, which is not described in detail herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/CN2014/079793 2013-11-11 2014-06-12 基于表格实现大数据量处理的方法及装置 WO2014183723A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020167010536A KR101777073B1 (ko) 2013-11-11 2014-06-12 테이블에 기반하여 대용량 데이터 처리를 구현하는 방법 및 장치
JP2016528157A JP6530390B2 (ja) 2013-11-11 2014-06-12 テーブルによるビッグデータ量処理方法および装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310557992.3 2013-11-11
CN201310557992.3A CN104636372B (zh) 2013-11-11 2013-11-11 基于表格实现大数据量处理的方法及装置

Publications (1)

Publication Number Publication Date
WO2014183723A1 true WO2014183723A1 (zh) 2014-11-20

Family

ID=51897782

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/079793 WO2014183723A1 (zh) 2013-11-11 2014-06-12 基于表格实现大数据量处理的方法及装置

Country Status (4)

Country Link
JP (1) JP6530390B2 (ko)
KR (1) KR101777073B1 (ko)
CN (1) CN104636372B (ko)
WO (1) WO2014183723A1 (ko)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113837593A (zh) * 2021-09-18 2021-12-24 北京百度网讯科技有限公司 虚拟数据分配方法、装置、设备以及存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107346312A (zh) * 2016-05-05 2017-11-14 中国移动通信集团内蒙古有限公司 一种大数据处理方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080168077A1 (en) * 2007-01-10 2008-07-10 Eric Lawrence Barsness Pre-loading of an in memory database
CN101526940B (zh) * 2008-03-19 2011-03-16 中兴通讯股份有限公司 一种应用系统物理数据库与内存数据库的数据同步方法
CN102411632A (zh) * 2011-12-27 2012-04-11 北京人大金仓信息技术股份有限公司 基于链表的内存数据库页式存储方法
CN102769535A (zh) * 2011-05-06 2012-11-07 中兴通讯股份有限公司 一种呈现告警数据统计结果的方法及系统

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09305616A (ja) * 1996-05-10 1997-11-28 Hitachi Ltd データ分析方法
JP2003016108A (ja) * 2001-06-29 2003-01-17 Casio Comput Co Ltd データ処理装置、および、プログラム
JP2003238207A (ja) * 2002-02-20 2003-08-27 Matsushita Electric Works Ltd 防曇物品とその製造方法
US7606792B2 (en) * 2004-03-19 2009-10-20 Microsoft Corporation System and method for efficient evaluation of a query that invokes a table valued function
US20060047696A1 (en) * 2004-08-24 2006-03-02 Microsoft Corporation Partially materialized views
JP4313845B2 (ja) * 2004-09-27 2009-08-12 修平 西山 マルチインスタンス・インメモリ・データベース
JP4939568B2 (ja) * 2009-04-28 2012-05-30 インターナショナル・ビジネス・マシーンズ・コーポレーション データベース間でデータを同期するための方法、並びにそのコンピュータ・システム及びコンピュータ・プログラム
CN101727319A (zh) * 2009-12-09 2010-06-09 上海宝信软件股份有限公司 可翻页的面向服务的数据表格控件及其实现方法
CN101876971A (zh) * 2010-06-22 2010-11-03 北京伟库电子商务科技有限公司 报表展现方法和装置
JP5598279B2 (ja) * 2010-11-16 2014-10-01 日本電気株式会社 分散メモリデータベースシステム、フロントデータベースサーバ、データ処理方法およびプログラム
JP5584917B2 (ja) * 2011-05-12 2014-09-10 株式会社日立製作所 データ分析システム及びデータ分析方法
US8839089B2 (en) * 2011-11-01 2014-09-16 Microsoft Corporation Multi-dimensional data manipulation and presentation
JP5527340B2 (ja) * 2012-02-27 2014-06-18 日本電気株式会社 ベクトル処理装置およびベクトル処理方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080168077A1 (en) * 2007-01-10 2008-07-10 Eric Lawrence Barsness Pre-loading of an in memory database
CN101526940B (zh) * 2008-03-19 2011-03-16 中兴通讯股份有限公司 一种应用系统物理数据库与内存数据库的数据同步方法
CN102769535A (zh) * 2011-05-06 2012-11-07 中兴通讯股份有限公司 一种呈现告警数据统计结果的方法及系统
CN102411632A (zh) * 2011-12-27 2012-04-11 北京人大金仓信息技术股份有限公司 基于链表的内存数据库页式存储方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113837593A (zh) * 2021-09-18 2021-12-24 北京百度网讯科技有限公司 虚拟数据分配方法、装置、设备以及存储介质
CN113837593B (zh) * 2021-09-18 2023-08-15 北京百度网讯科技有限公司 虚拟数据分配方法、装置、设备以及存储介质

Also Published As

Publication number Publication date
CN104636372A (zh) 2015-05-20
CN104636372B (zh) 2019-04-26
KR101777073B1 (ko) 2017-09-08
JP2016539415A (ja) 2016-12-15
JP6530390B2 (ja) 2019-06-12
KR20160058928A (ko) 2016-05-25

Similar Documents

Publication Publication Date Title
US9696894B2 (en) Analytic process design
US9953066B2 (en) Automatically generating and reporting merged performance and capacity statistics
WO2019237532A1 (zh) 一种业务数据的监控方法、存储介质、终端设备及装置
DE112016001737T5 (de) Systeme und Verfahren für die Benachrichtigung von Benutzern über Änderungen an Dateien in cloudbasierten Dateispeichersystemen
WO2014079304A1 (zh) 一种报表创建方法、装置和系统
CN108880857B (zh) 发现与呈现网络应用访问信息的方法和系统
JP2005198064A (ja) ファイル管理プログラム
US20150213066A1 (en) System and method for creating data models from complex raw log files
CN111078695B (zh) 计算企业内元数据关联关系的方法及装置
US11030254B2 (en) Storage estimation in a data indexing and query system
CN113688288B (zh) 数据关联分析方法、装置、计算机设备和存储介质
US11625526B2 (en) Systems and methods for displaying digital forensic evidence
CN103092997B (zh) 用于报表分析的联动查询系统和联动查询方法
CN106610774A (zh) 网页表格编辑方法及装置
CN113138973A (zh) 数据管理系统及工作方法
CN108073720B (zh) 应用于大数据系统的数据质量管理系统及方法
JP2008310450A5 (ko)
WO2014183723A1 (zh) 基于表格实现大数据量处理的方法及装置
CN114185750A (zh) 流程监控方法、装置、设备及存储介质
US20150356064A1 (en) Management system for displaying information relating to target
US20190340104A1 (en) Error finder tool
CN107894940A (zh) 一种日志分析装置及方法
CN112527747A (zh) 一种日志模板配置、展示方法及装置
JP2013025502A (ja) 情報処理装置および情報処理方法、プログラム並びに記憶媒体
CN109684159A (zh) 分布式消息系统的状态监控方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14798583

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20167010536

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2016528157

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14798583

Country of ref document: EP

Kind code of ref document: A1