WO2017085985A1 - Système, procédé et programme d'accélération de traitement de base de données - Google Patents

Système, procédé et programme d'accélération de traitement de base de données Download PDF

Info

Publication number
WO2017085985A1
WO2017085985A1 PCT/JP2016/076110 JP2016076110W WO2017085985A1 WO 2017085985 A1 WO2017085985 A1 WO 2017085985A1 JP 2016076110 W JP2016076110 W JP 2016076110W WO 2017085985 A1 WO2017085985 A1 WO 2017085985A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
database
processing
parallel
parallel processing
Prior art date
Application number
PCT/JP2016/076110
Other languages
English (en)
Japanese (ja)
Inventor
浩平 海外
Original Assignee
浩平 海外
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浩平 海外 filed Critical 浩平 海外
Priority to JP2017545609A priority Critical patent/JP6381823B2/ja
Publication of WO2017085985A1 publication Critical patent/WO2017085985A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures

Definitions

  • the present invention relates to a system, method, and program for speeding up database processing, and more particularly to a system, method, and program for realizing speedup using a graphic processor and peer-to-peer DMA.
  • a database management system in particular, a relational database management system (RDBMS) has become an indispensable component in today's information systems. Therefore, speeding up the processing of the RDBMS is extremely important for improving the efficiency of the information system, and many speeding up techniques have been proposed.
  • DBMS database management system
  • RDBMS relational database management system
  • Non-Patent Document 1 and Non-Patent Document 2 are known as techniques for speeding up database access by GPGPU.
  • the central processing unit In order to process data stored in a database on the storage, the central processing unit (CPU) first secures a buffer area on the main storage device, and then loads data from the secondary storage device to the buffer. Only when the loading process is completed can it be possible to access the data stored in the secondary storage device.
  • the bandwidth between the CPU and the main memory is 50 GB to 300 GB per second, whereas the bandwidth of the peripheral device bus connecting the storage and the CPU is about 4 GB to 15 GB per second. Inevitably become a bottleneck in performance. In the conventional database processing method, it is necessary to transfer a large amount of data through this bottleneck, and there is a problem that the performance improvement effect of parallel processing by GPGPU is offset.
  • GPUDirect RDMA http://docs.nvidia.com/cuda/gpudirect-rdma/index.html
  • GPGPU Accelerates PostgreSQL (http://www.slideshare.net/kaigai/gpgpu-accelerates-postgresql)
  • a database query acceleration system, method, and program that can be implemented at low cost are provided.
  • the present invention relates to a data processing method in a database processing system including a data storage means, a main storage device, a central processing unit, a parallel processing device, a peripheral device bus, and a database stored in the data storage means.
  • a device issuing an instruction to the data storage means to transfer the data in the database to the parallel processing device without going through the main storage device; and the data storage means, the data storage means in the database Transferring the data to the parallel processing device without going through the main storage device; the parallel processing device processing the data in the database in parallel; and the parallel processing device comprising the database And a result of processing the data therein is written to the main storage device.
  • this invention solves the said subject by providing the data processing method as described in the preceding paragraph whose said parallel processing apparatus is a graphics processing unit (GPU).
  • GPU graphics processing unit
  • the present invention is a data processing program executed by a database processing system including data storage means, a main storage device, a central processing unit, a parallel processing device, a peripheral device bus, and a database stored in the data storage means.
  • a device including a command to write a result of processing the data in the database to the main memory.
  • this invention solves the said subject by providing the data processing program as described in the preceding paragraph whose said parallel processing apparatus is a graphics processing unit (GPU).
  • GPU graphics processing unit
  • the present invention is a database processing system including data storage means, a main storage device, a central processing unit, a parallel processing device, a peripheral device bus, and a database stored in the data storage unit, wherein the central processing unit is Issuing an instruction to the data storage means to transfer the data in the database to the parallel processing device without going through the main storage device, and the data storage means sends the data in the database to the main storage device. Transfer to the parallel processing device without going through the storage device, the parallel processing device processed the data in the database in parallel, the parallel processing device processed the data in the database.
  • the present invention further solves the problem by providing the database processing system according to the preceding paragraph, wherein the parallel processing device is a graphics processing unit (GPU).
  • the parallel processing device is a graphics processing unit (GPU).
  • a database query acceleration system, method, and program that are not easily affected by the performance bottleneck of the peripheral device bus can be implemented at low cost.
  • FIG. 1 shows an example of a functional configuration of a computer according to an embodiment of the present invention.
  • the central processing unit (101) executes a program stored in the main storage device (102) and moves data between the main storage device (102) and the secondary storage device (103).
  • the central processing unit (101) may be a processor (CPU) used in a general computer.
  • the main storage device (102) may be a DRAM used in a general computer.
  • the parallel processing device (104) is a processor that has a large number of cores and can perform parallel processing, and may be a graphical processing unit (GPU). Further, the parallel processing device (104) may be a processor having a large number of cores called MIC (Many Integrated Core) or a dedicated design processor implemented by FGPA (Programmable Gate Array).
  • MIC Many Integrated Core
  • FGPA Programmable Gate Array
  • the peripheral device bus (106) is a means for connecting the main storage device (102), the secondary storage device (103), and the parallel processing device (104), and supports peer-to-peer DMA (direct memory access). .
  • the peripheral device bus (106) is desirably provided with an NVMe function for connecting an SSD (semiconductor disk) to the industry standard PCI-E.
  • the secondary storage device (103) is preferably an SSD based on flash memory technology.
  • the computer according to the present invention may be provided with a network connection function for transmitting and receiving data and user requests via the network.
  • FIG. 2 shows an example of the functional configuration of the relational DBMS software according to the present invention.
  • the query syntax analysis unit (201) receives a query sentence described in the SQL format, and creates a query syntax tree that is an internal format.
  • the query optimization unit (202) performs an optimization process on the query syntax tree and creates a query execution plan which is an internal format.
  • the query execution plan is executed by the central processing unit (101) and the parallel processing unit (104), and queries (queries), joins (joins), classifications (sorts), aggregations (aggregates), etc. for the data stored in the database. Is performed.
  • the present invention is not limited to a relational DBMS, and can be widely applied to any database management system that performs parallel processing on a large amount of data.
  • FIG. 3 shows an example of a relational DBMS query execution plan according to an embodiment of the present invention.
  • a query is represented by a scan process and a join process.
  • the scanning process is a process of reading only rows (records, tuples) that match a scanning condition (for example, a specified column (column, field) value is within a specified numerical range) from a table in the relational DBMS.
  • a scanning condition for example, a specified column (column, field) value is within a specified numerical range
  • Both are processes typically performed in a relational DBMS, and improving the efficiency of both leads to the efficiency of the entire information system. Since both handle large amounts of data, it is important to minimize the movement of data, especially the movement of data via the peripheral device bus (106), which is a bottleneck in performance.
  • FIG. 4 shows a first example of the relational DBMS processing method according to the embodiment of the present invention.
  • a thick arrow line represents a data flow
  • a broken arrow line represents a control flow or a logical relationship.
  • the central processing unit (101) dynamically generates an instruction sequence that can be executed by the parallel processing unit (104) according to the scanning condition of the query execution plan, and places it on the main storage unit (102) (S401). Since the table scanning process performs the same process (for example, numerical value range check) on a large number of rows, it is easy to generate an instruction for the parallel processing device (104).
  • the central processing unit (101) transfers the generated instruction sequence from the main storage unit (102) to the parallel processing unit (104) (S402).
  • the central processing unit (101) issues a command for transferring a set of rows in the database to be processed to the parallel processing unit (104) to the secondary storage unit (103) (S403).
  • This command is preferably to initiate data transfer by peer-to-peer DMA.
  • the secondary storage device (103) transfers a set of rows in the database to be processed to the parallel processing device (104) without going through the main storage device (102) in accordance with this command ( It is desirable that the central processing unit (101) is designed not to intervene in this transfer process) (S404).
  • the parallel processing device (104) performs a predetermined scanning process, and only the rows that meet the scanning conditions are transferred to the main memory (102) (S405). Further, only the relevant data in the columns (columns, fields) required for the subsequent processing (for example, data corresponding to the column ORDER_ID in the example of the above SQL statement) is transferred in each row to be transferred. It is desirable.
  • the data to be processed may be each block of the database, and the parallel processing device (104) selects only the row that is finally used (that is, the row that satisfies the scanning condition) as the parallel processing device (104). It is desirable that the data is transferred from the result buffer to the main storage device (102) by peer-to-peer DMA without using the central processing unit (101).
  • the main storage device (102) only the necessary data is finally placed in the main storage device (102), and a large amount of data is transferred to the main memory via the peripheral device bus (106) only for temporary work. Since the data is not transferred onto the storage device (102), the processing load of the central processing unit (101), the consumption of the main storage device (memory), and the bandwidth of the peripheral device bus (106) can be reduced. In addition, the time required for data transfer can be shortened, and the efficiency of the entire database processing system can be improved.
  • FIG. 5 shows a second example of the data processing method of the relational DBMS according to the embodiment of the present invention.
  • the efficiency of the combination of the combining process and the scanning process is shown.
  • a thick arrow line represents a data flow
  • a broken arrow line represents a control flow.
  • FIG. 5 it is assumed that the row (record, tuple) of the table A that matches the scanning condition A has already been loaded on the main storage device (102). It is desirable that this loading process be performed by a highly efficient method as shown in FIG.
  • the central processing unit (101) dynamically generates an instruction sequence that can be executed by the parallel processing unit (104) in accordance with the join condition of the query execution plan and the scanning condition B, and places it on the main storage unit (102). (S501). Since the table join process and the scan process perform the same process for many rows, it is easy to generate an instruction for the parallel processing device (104).
  • the central processing unit (101) transfers the generated instruction sequence and the processing target row of the table A from the main storage unit (102) to the parallel processing unit (104) (S502).
  • the central processing unit (101) issues an instruction for transferring a set of rows in the database to be processed to the parallel processing unit (104) to the secondary storage unit (103) (S503).
  • This command is preferably to initiate data transfer by peer-to-peer DMA.
  • the secondary storage device (103) transfers the set of rows of the table B in the database to the parallel processing device (104) without passing through the main storage device (102) according to this command (this It is desirable that the central processing unit (101) is designed not to intervene in the transfer process) (S504).
  • the parallel processing device (104) performs predetermined combining processing and scanning processing, and transfers only the rows that match the combining condition and scanning condition B to the main memory (102) (S505). Furthermore, it is desirable to transfer only relevant data of columns (columns, fields) required for the subsequent processing within each row to be transferred.
  • the data to be processed may be each block of the database, and the parallel processing device (104) parallels only the rows that are finally used (that is, the rows that satisfy the scanning condition B and the combining condition). It is desirable to design the transfer from the result buffer to the main memory (102) by peer-to-peer DMA without using the central processing unit (101) and placing it in the result buffer in the processing unit (104). Similar to the example shown in FIG. 4, according to the method of the present invention, only the necessary data is finally placed in the main storage device (102), and a large amount of data is stored only for temporary work.
  • the processing load of the central processing unit (101), the amount of memory consumption, and the bandwidth of the peripheral device bus (106) The width can be reduced.
  • the time required for data transfer can be shortened, and the efficiency of the entire database processing system can be improved.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Le problème décrit dans la présente invention concerne un procédé, un appareil et un programme pour l'accélération de traitement de base de données qui peuvent être mises en œuvre de manière économique. La solution selon la présente invention concerne un système comprenant un coprocesseur doté d'une fonction de traitement parallèle tel qu'un GPU, une séquence d'instructions pour le coprocesseur est générée de manière dynamique sur la base d'une interrogation à une base de données, une exécution parallèle est effectuée avec le coprocesseur, et des données sont transférées d'un dispositif de stockage secondaire au coprocesseur sans interposition d'une unité centrale, une mémoire principale et un bus d'équipement périphérique, ce qui permet d'améliorer l'efficacité de traitement du système entier.
PCT/JP2016/076110 2015-11-22 2016-09-06 Système, procédé et programme d'accélération de traitement de base de données WO2017085985A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2017545609A JP6381823B2 (ja) 2015-11-22 2016-09-06 データベース処理高速化のシステム、方法、及び、プログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2015228245 2015-11-22
JP2015-228245 2015-11-22

Publications (1)

Publication Number Publication Date
WO2017085985A1 true WO2017085985A1 (fr) 2017-05-26

Family

ID=58719175

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/076110 WO2017085985A1 (fr) 2015-11-22 2016-09-06 Système, procédé et programme d'accélération de traitement de base de données

Country Status (2)

Country Link
JP (1) JP6381823B2 (fr)
WO (1) WO2017085985A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020121359A1 (fr) * 2018-12-09 2020-06-18 浩平 海外 Système, procédé et programme d'amélioration de d'efficacité d'interrogations de base de données

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013528856A (ja) * 2010-04-29 2013-07-11 インターナショナル・ビジネス・マシーンズ・コーポレーション 階層型分散処理システムのためのソフトウェアのコンパイル
JP2014056425A (ja) * 2012-09-12 2014-03-27 Nippon Telegr & Teleph Corp <Ntt> データ管理装置、データ管理システム、処理割当方法および処理割当プログラム
JP2014170259A (ja) * 2013-03-01 2014-09-18 Hitachi Ltd データ処理方法、データ処理システム及びデータ処理プログラム

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6862027B2 (en) * 2003-06-30 2005-03-01 Microsoft Corp. System and method for parallel execution of data generation tasks
WO2015105043A1 (fr) * 2014-01-08 2015-07-16 日本電気株式会社 Système de calcul, dispositif de gestion de base de données, et procédé de calcul

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013528856A (ja) * 2010-04-29 2013-07-11 インターナショナル・ビジネス・マシーンズ・コーポレーション 階層型分散処理システムのためのソフトウェアのコンパイル
JP2014056425A (ja) * 2012-09-12 2014-03-27 Nippon Telegr & Teleph Corp <Ntt> データ管理装置、データ管理システム、処理割当方法および処理割当プログラム
JP2014170259A (ja) * 2013-03-01 2014-09-18 Hitachi Ltd データ処理方法、データ処理システム及びデータ処理プログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020121359A1 (fr) * 2018-12-09 2020-06-18 浩平 海外 Système, procédé et programme d'amélioration de d'efficacité d'interrogations de base de données

Also Published As

Publication number Publication date
JP6381823B2 (ja) 2018-08-29
JPWO2017085985A1 (ja) 2017-11-24

Similar Documents

Publication Publication Date Title
US10706052B2 (en) Method for performing in-memory hash join processing in relational database systems
Kim et al. Fast, energy efficient scan inside flash memory SSDs
US20190050325A1 (en) Intelligent high bandwidth memory appliance
Wang et al. A study of data partitioning on OpenCL-based FPGAs
US20170060811A1 (en) Matrix operands for linear algebra operations
US20180032477A1 (en) High performance inplace transpose operations
EP3642736B1 (fr) Délestage de calcul parallèle à un accélérateur de base de données
US11194522B2 (en) Networked shuffle storage
KR102442682B1 (ko) 머신 러닝 시스템에서 직렬화된 키 값 접근을 방지하기 위한 시스템 및 방법
Watanabe et al. Column-oriented database acceleration using fpgas
JP6381823B2 (ja) データベース処理高速化のシステム、方法、及び、プログラム
JP6316503B2 (ja) 計算機システム、アクセラレータ及びデータベースの処理方法
US20190354521A1 (en) Concurrent Data Processing in a Relational Database Management System Using On-Board and Off-Board Processors
JP6829427B2 (ja) データベース・クエリ効率化のためのシステム、方法、および、プログラム
US10997175B2 (en) Method for predicate evaluation in relational database systems
US20210278992A1 (en) Adjustable function-in-memory computation system
US20160170892A1 (en) Expression pattern matching in a storage subsystem
US11776594B2 (en) Scalable in situ DRAM-based accelerators and methods of operating the same
US9626397B2 (en) Discounted future value operations on a massively parallel processing system and methods thereof
US12001427B2 (en) Systems, methods, and devices for acceleration of merge join operations
JP7138313B1 (ja) コンピュテーショナル・ストレージの処理効率化の方法およびプログラム
Li et al. RPK-table based efficient algorithm for join-aggregate query on MapReduce
US20240143199A1 (en) Sparse Matrix Operations Using Processing-in-Memory
Lindemann et al. MAGPIE: a scalable data storage system for efficient high volume data queries
WO2018083751A1 (fr) Sous-système de mémorisation intelligent

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16865988

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017545609

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16865988

Country of ref document: EP

Kind code of ref document: A1