CN106066890B - Distributed high-performance database all-in-one machine system - Google Patents
Distributed high-performance database all-in-one machine system Download PDFInfo
- Publication number
- CN106066890B CN106066890B CN201610427650.3A CN201610427650A CN106066890B CN 106066890 B CN106066890 B CN 106066890B CN 201610427650 A CN201610427650 A CN 201610427650A CN 106066890 B CN106066890 B CN 106066890B
- Authority
- CN
- China
- Prior art keywords
- storage
- performance
- data
- database
- storage equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Abstract
The invention relates to a distributed high-performance database all-in-one machine system, which comprises a database all-in-one machine and an auxiliary module erected on the database all-in-one machine, wherein the database all-in-one machine comprises: the user host is used for carrying out service processing; the Oracle application host is used for returning a processing result to the user host; the storage equipment is used for storing data required by the user host; the data storage network is used for connecting the Oracle application host and the storage equipment; the auxiliary module comprises: the storage equipment management module is used for uniformly scheduling the storage equipment; a distributed high performance intelligent storage module; the system is used for communicating with a storage device management module to realize distributed high-performance intelligent storage of the storage device; and the Oracle host adaptation module is used for mapping the storage equipment into standard storage hardware resources. Compared with the prior art, the method has the advantages of large data processing amount, high processing speed, strong processing performance, high reliability and the like.
Description
Technical Field
The invention relates to the field of Oracle database storage management, in particular to a distributed high-performance database all-in-one machine system.
Background
The Oracle database is a very good DMBS introduced by Oracle corporation, and currently, the Oracle DBMS and related products are applied in almost all industrial fields all over the world. Whether the online transaction processing and data warehouse application of large enterprises or the medium and small online transaction processing business, there are many paradigm applications in which the Oracle database system is successfully used.
To deploy an Oracle database, the most common structure is implemented by a multi-node rac (real application cluster) plus shared storage. Fig. 1 is a typical construction mode, which includes 1 a user host 1, an Oracle RAC node 2 and a shared storage device 3, and such a structural mode has existed in a data center for more than ten years, and is mature, reliable and quite stable.
Small-sized or high-end PC servers are generally selected to be deployed as Oracle RAC nodes, that is, frequently called high-availability nodes of a database, and access a Shared Storage device (Shared Storage) through a high-speed FC Storage network. The User (User) is connected with the RAC node of the Oracle database through the TCP/IP network, the Oracle RAC node can process results and return the results to the User, and when one database node is abnormal, the use of the User is not influenced.
Today, with the rapid increase of data volume, the rapid development of new technology generates massive data to be processed, and the data scale is dozens of TB, hundreds of TB and even PB data. In this case, the conventional Oracle shared storage architecture has too much storage input/output waiting, which not only generates huge waste, but also causes performance bottleneck, so that the low-efficiency architecture mode cannot meet the business requirement of the user. Meanwhile, the system structure is increasingly becoming a bottleneck of informatization development in the aspects of storage performance, capacity expansion and management.
In the current shared storage device under the traditional architecture, the IOPS value is maintained at 5-10 ten thousand per second even under the support of a large number of disks. The current FC network can reach the interface rate of 16Gb/s at most, even if middle-high end storage is adopted, the multi-port parallel is adopted, the throughput rate can only reach the level of 4-5 GB/s, and in the case of TB-level data processing, the common service time is more than 20 hours, and the user service requirements can obviously not be met.
Therefore, a database deployment mode under the traditional Oracle RAC structure has a serious performance bottleneck problem in the existing large data volume processing.
Disclosure of Invention
The invention aims to provide a distributed high-performance database all-in-one machine system which is large in data processing amount, high in processing speed, strong in processing performance and high in reliability.
In order to achieve the purpose of the invention, the invention provides a distributed high-performance database all-in-one machine system, which comprises a database all-in-one machine and an auxiliary module erected on the database all-in-one machine, wherein the database all-in-one machine comprises:
the user host is used for carrying out service processing;
the Oracle application host is connected with the user host through a TCP/IP network and used for returning a processing result to the user host;
the storage equipment is used for storing data required by the user host;
the data storage network is respectively connected with the Oracle application host and the storage equipment and is used for enabling the Oracle application host to access the storage equipment;
the auxiliary module includes:
the storage equipment management module is erected on the storage equipment and used for uniformly scheduling the storage equipment to complete basic management work of the storage equipment;
the distributed high-performance intelligent storage module is erected on the storage equipment and is used for communicating with the storage equipment management module to realize distributed high-performance intelligent storage of the storage equipment;
and the Oracle host adaptation module is erected on the Oracle application host and used for mapping the storage equipment into standard storage hardware resources so that the Oracle application host accesses the storage equipment.
The Oracle application host is an open X86 architecture server.
The data storage network is an InfiniBand network.
The storage device is a distributed high-performance parallel storage system and comprises a plurality of storage nodes, and each storage node comprises a local disk of a PC (personal computer) server and a high-performance Flash SSD with corresponding capacity.
The storage device management module includes:
the storage scheduling unit is used for dividing the local disk of the PC server into different blocks, marking each block as an object data space and simultaneously marking the Flash SSD as a cache space;
the storage communication unit is used for communicating with the distributed high-performance intelligent storage module to realize high-performance intelligent storage of the storage equipment;
and the storage driving unit is used for realizing the read-write operation of the storage equipment according to the commands of the scheduling unit and the communication unit.
The distributed high-performance smart storage module includes:
the communication unit is used for communicating with the storage equipment management module to realize data access operation on the storage equipment;
the distributed storage unit is used for uniformly distributing the data in all the object data spaces, calling the data into the Flash SSD according to the access frequency, and simultaneously reserving 2 or 3 copies of the data in each storage node to prevent the data loss caused by node failure;
and the intelligent scheduling unit is used for marking the local disk of the failed PC server as invalid, redundantly distributing the data strategy in the failed disk to the rest disks, and uniformly distributing the data again after the failed disk is replaced.
The switching rate of each port of the switch in the InfiniBand network is 56 Gb/s.
The number of the local disks of the PC server contained in each storage node is between 12 and 24.
Compared with the prior art, the invention has the following beneficial effects:
(1) the Oracle application host adopts an open X86-structured server, and meets the large-scale OLTP/OLAP application requirements above TB level data volume.
(2) The data storage network selects an InfiniBand network switching technology which is novel in technology, high in network bandwidth, small in network delay and widely applied to a supercomputer, and adopts a switch with a switching rate of 56Gb/s per port, so that the bandwidth is increased by 3.5 times compared with that of a traditional FC network switch, and the IO throughput capacity of a database is greatly improved.
(3) The storage device is a distributed high-performance parallel storage system, and compared with a traditional special shared storage architecture, the IOPS and the throughput capacity of storage are greatly improved.
(4) The high-performance intelligent storage module is arranged, high-performance distribution of stored data is achieved, the data are automatically distributed according to a new redundancy strategy when problems occur in the storage equipment, and the high efficiency of the database all-in-one machine is guaranteed.
(5) The high-performance intelligent storage module is provided with backup for data, single-point faults do not exist, and the damage of any storage node does not influence service application, so that the reliability of the database all-in-one machine is ensured.
(6) In the storage device management module, a high-performance SSD disk and a high-efficiency hot spot data caching technology are introduced, so that the storage node has the characteristics of the high-performance SSD and the traditional disk storage characteristics.
Drawings
FIG. 1 is a diagram of a conventional Oracle RAC architecture;
FIG. 2 is a schematic diagram of a distributed high performance storage Oracle RAC architecture;
FIG. 3 is a schematic structural view of the present invention;
FIG. 4 is a schematic diagram of the disk management logic within a storage node;
FIG. 5 is a diagram of cache management.
In the figure: the system comprises a user host 1, an Oracle RAC node 2, a shared storage device 3, a distributed high-performance storage system 4, an Oracle host adaptation module 5, a distributed high-performance intelligent storage module 6, a storage device management module 7, disk management software 8, a memory DRAM 9, an NVDIMM 10, a Flash SDD 11 and a disk HDD 12.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
As shown in FIG. 1, the invention is based on Oracle database application, and realizes a series of new architecture software integration systems on standard hardware while ensuring the high availability of Oracle RAC, and builds a set of high-performance database technical architecture by opening a standard PC server and distributed high-performance software.
As shown in fig. 2 to fig. 3, in the embodiment, a distributed high-performance database all-in-one machine system is provided, a storage device of the distributed high-performance database all-in-one machine system is a distributed high-performance storage system 4, core components of the distributed high-performance database all-in-one machine system include an Oracle host adaptation module 5, a distributed high-performance intelligent storage module 6 and a storage device management module 7, and cooperative work of the components is completed to improve application performance of a database.
In the scheme, the Oracle RAC node adopts an open X86-structured server, meets the large-scale OLTP/OLAP application requirement, and deploys the service application based on the Oracle RAC. The data storage network selection technology is novel, the network bandwidth is high, the network delay is small, and the InfiniBand network switching technology is widely applied to the supercomputer, and compared with the traditional FC network switch, the network bandwidth of the switch with the switching rate of 56Gb/s per port is improved by 3.5 times by adopting the switch in the scheme, and the IO throughput capacity of the database is greatly improved.
The storage system application abandons the traditional special shared storage architecture, and establishes a distributed high-performance parallel storage system by using an open mode of a local disk of a standardized PC server to serve as a storage node. The novel PC server node for distributed high-performance storage has 12-24 physical disks and is configured with a high-performance Flash SSD with corresponding capacity. The IOPS and the throughput capacity of storage are improved by a hot spot data caching scheduling optimization technology in the storage node.
Fig. 3 shows the main logical structure of the technical architecture of the high-performance database all-in-one machine. The main functions performed by each module are as follows:
1. storage device management module
And the storage device management module (disk management module) uniformly schedules and manages the disk of the PC server and the important components of the high-speed FlashSSD, and completes the basic management work of the disk. The module divides a local disk into different blocks (units), each block becomes an object Data space (Data Unit, DU for short) managed by intelligent storage software, and simultaneously marks a Flash SSD as a cache space (Flash Unit), and the device driving module simultaneously undertakes actual Data writing work. And the storage equipment management module software is responsible for communicating with the distributed high-performance intelligent storage software.
As shown in fig. 4, for data migration of hybrid storage medium access used in this embodiment, configuring a cache at a storage node is an important method for improving performance of a storage system. By temporarily storing the shared data in the high-speed DRAM, the operation speed of mass shared data is obviously improved. Despite having read and write speeds on the order of nanoseconds (ns), the low storage capacity and high cost of DRAM limits its ability to be used in large-scale caching to promote HDD read and write access. The Flash SSD (Flash solid state disk) is used as a novel storage medium, has the advantages of low power consumption, high speed and large capacity, and has the cost and performance between those of a DRAM and an HDD, so that the Flash SSD can be used as a secondary cache between the DRAM and the HDD to effectively improve the I/O bandwidth and the IOPS. Although FlashSSD is very attractive as a cache, it has a large asymmetry in read and write operations (i.e., reads are much faster than writes) and flash memory has a limited number of erases. Existing cache content replacement algorithms are mainly directed to DRAM operations, and take little consideration of the characteristics of flash as a cache, and the architecture of multi-level caches. Therefore, the embodiment jointly designs a two-level cache management strategy based on the service characteristics.
2. Distributed high-performance intelligent storage software
The distributed high-performance intelligent storage software is the core of the whole data storage process. The intelligent storage software is responsible for the performance balanced distribution storage of the data of the multiple storage nodes, the data are uniformly distributed in all DUs, and the data are transferred into a Flash SSD according to the access frequency, so that the overall access performance of the system is improved.
The other major function of the distributed high-performance intelligent storage software is responsible for the reliability of data, and the intelligent storage software can keep 2-3 copies of the data among different storage nodes according to a service strategy, so that when one node fails, the data cannot be lost, and the application of upper database nodes cannot be influenced. Similarly, when a single disk fails, the intelligent storage software marks the failed disk as a failure state, and simultaneously, the data is further distributed in the residual disks in a strategic redundancy manner, so as to ensure the reliability of the data. And when the failed disk is replaced, automatically completing the balanced distribution of the newly added disk data and recovering the initial state. The distributed high-performance intelligent storage software is responsible for communicating with the storage equipment management module and the equipment driving module to coordinate and finish data access.
Therefore, the embodiment jointly designs the cache management strategy based on the service characteristics. As shown in fig. 5, aiming at the low capacity of DRAM and the deficiency of Flash SSD write operation, a super memory NVDIMM formed by DRAM and a nonvolatile memory chip is introduced to design a novel read-write separation technology. Designing a novel read-write separation method: aiming at the characteristic that financial data service has more and random small file write operation, a super memory NVDIMM is introduced. The volatile DRAM writes small dirty data into the nonvolatile NVDIMM firstly, when the dirty data of the NVDIMM are converged to a certain threshold value, the continuous blocks are written into the HDD in batch, and the random blocks are written into the Flash SSD. This embodiment significantly reduces the number of I/O operations compared to direct DRAM-to-HDD writing; compared with the writing from the DRAM to the Flash SSD, the erasing times of the Flash SSD are reduced, and the data consistency is improved. NVDIMM is mainly used for write cache of HDD, while Flash SDD is mainly used for read cache of HDD, thereby realizing separation of read-write operation to a greater extent.
3. Oracle host adaptation software
The Oracle database cannot directly identify the back-end distributed high-performance storage system, so Oracle host adaptation software needs to be deployed in the Oracle application host. The distributed high performance storage is mapped to standard storage hardware resources through Oracle host adaptation software, i.e. standard disks recognizable by the traditional Oracle ASM are mapped. The Oracle application host can directly access the storage hardware resource through the software module without any change. The Oracle host adaptation software module is responsible for communicating with the distributed high-performance intelligent storage software.
Through the organic combination of the storage equipment management module, the distributed high-performance intelligent storage software module and the Oracle host adaptive software module, an open type standardized PC server is used as a basis, the distributed high-performance intelligent storage software with high efficiency and reliability of data storage is used as a core, and a high-speed Flash SSD and a high-speed low-delay InfiniBand switch are combined. Through the integration and innovation of software and hardware, the performance far exceeding that of the traditional host and the storage system database structure is achieved, and the management burden is simplified.
Claims (6)
1. The utility model provides a distributed high performance database all-in-one machine system, includes database all-in-one machine and erects the auxiliary module on the database all-in-one machine, its characterized in that, the database all-in-one machine includes:
the user host is used for carrying out service processing;
the Oracle application host is connected with the user host through a TCP/IP network and used for returning a processing result to the user host;
the storage device is used for storing data required by a user host, is a distributed high-performance parallel storage system and comprises a plurality of storage nodes, and each storage node comprises a local disk of a PC (personal computer) server and a high-performance Flash SSD with corresponding capacity;
the data storage network is respectively connected with the Oracle application host and the storage equipment and is used for enabling the Oracle application host to access the storage equipment;
the auxiliary module includes:
the storage equipment management module is erected on the storage equipment and used for uniformly scheduling the storage equipment to complete basic management work of the storage equipment;
the distributed high-performance intelligent storage module is erected on the storage equipment and is used for communicating with the storage equipment management module to realize distributed high-performance intelligent storage of the storage equipment;
the Oracle host adaptation module is erected on the Oracle application host and is used for mapping the storage equipment into standard storage hardware resources so that the Oracle application host accesses the storage equipment;
the distributed high-performance smart storage module includes:
the communication unit is used for communicating with the storage equipment management module to realize data access operation on the storage equipment;
the distributed storage unit is used for uniformly distributing the data in all the object data spaces, calling the data into the Flash SSD according to the access frequency, and simultaneously reserving 2 or 3 copies of the data in each storage node to prevent the data loss caused by node failure;
and the intelligent scheduling unit is used for marking the local disk of the failed PC server as invalid, redundantly distributing the data strategy in the failed disk to the rest disks, and uniformly distributing the data again after the failed disk is replaced.
2. The distributed high-performance database all-in-one machine system according to claim 1, wherein the Oracle application host is an open X86 architecture server.
3. The distributed high-performance database kiosk system of claim 1 wherein the data storage network is an InfiniBand network.
4. The distributed high-performance database all-in-one machine system according to claim 1, wherein the storage device management module comprises:
the storage scheduling unit is used for dividing the local disk of the PC server into different blocks, marking each block as an object data space and simultaneously marking the Flash SSD as a cache space;
the storage communication unit is used for communicating with the distributed high-performance intelligent storage module to realize high-performance intelligent storage of the storage equipment;
and the storage driving unit is used for realizing the read-write operation of the storage equipment according to the commands of the storage scheduling unit and the storage communication unit.
5. The distributed high-performance database all-in-one system according to claim 3, wherein the switching rate per port of the switches in the InfiniBand network is 56 Gb/s.
6. The distributed high-performance database all-in-one machine system according to claim 1, wherein each storage node comprises between 12 and 24 local disks of the PC server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610427650.3A CN106066890B (en) | 2016-06-16 | 2016-06-16 | Distributed high-performance database all-in-one machine system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610427650.3A CN106066890B (en) | 2016-06-16 | 2016-06-16 | Distributed high-performance database all-in-one machine system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106066890A CN106066890A (en) | 2016-11-02 |
| CN106066890B true CN106066890B (en) | 2020-02-18 |
Family
ID=57421256
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610427650.3A Active CN106066890B (en) | 2016-06-16 | 2016-06-16 | Distributed high-performance database all-in-one machine system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106066890B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108153759B (en) * | 2016-12-05 | 2021-07-09 | 中国移动通信集团公司 | Data transmission method of distributed database, intermediate layer server and system |
| CN106843770A (en) * | 2017-01-23 | 2017-06-13 | 北京思特奇信息技术股份有限公司 | A kind of distributed file system small file data storage, read method and device |
| CN106991120A (en) * | 2017-02-22 | 2017-07-28 | 杭州沃趣科技股份有限公司 | A kind of method for realizing oracle database with computer room zero-data loss |
| CN106933514A (en) * | 2017-03-15 | 2017-07-07 | 郑州云海信息技术有限公司 | A kind of automatic location mode of cold and hot data of ASM disks group and system |
| CN109213827B (en) * | 2017-07-03 | 2022-07-08 | 阿里云计算有限公司 | Data processing system, method, router and slave database |
| CN107463336B (en) * | 2017-08-07 | 2020-09-22 | 苏州浪潮智能科技有限公司 | Method and system for processing failure of solid state disk in SSD (solid state disk) cache |
| CN107544757A (en) * | 2017-08-11 | 2018-01-05 | 郑州云海信息技术有限公司 | A kind of method and system based on MCS system quick search disk id and Firmware versions |
| CN109144412A (en) * | 2018-07-26 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of iSCSI adapter batch scanning method and system |
| CN110161930B (en) * | 2019-06-05 | 2021-11-09 | 安徽三实信息技术服务有限公司 | Data monitoring system and data monitoring method thereof |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102385628A (en) * | 2011-11-14 | 2012-03-21 | 北京锐安科技有限公司 | Java data base connectivity (JDBC)-based data distributed processing method |
| CN103516809A (en) * | 2013-10-22 | 2014-01-15 | 浪潮电子信息产业股份有限公司 | High-scalability and high-performance distributed storage system structure |
| CN103873559A (en) * | 2014-02-13 | 2014-06-18 | 南京斯坦德通信股份有限公司 | Database all-in-one machine capable of realizing high-speed storage |
| CN104035522A (en) * | 2014-06-16 | 2014-09-10 | 南京云创存储科技有限公司 | Large database appliance |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9185054B2 (en) * | 2010-09-15 | 2015-11-10 | Oracle International Corporation | System and method for providing zero buffer copying in a middleware machine environment |
-
2016
- 2016-06-16 CN CN201610427650.3A patent/CN106066890B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102385628A (en) * | 2011-11-14 | 2012-03-21 | 北京锐安科技有限公司 | Java data base connectivity (JDBC)-based data distributed processing method |
| CN103516809A (en) * | 2013-10-22 | 2014-01-15 | 浪潮电子信息产业股份有限公司 | High-scalability and high-performance distributed storage system structure |
| CN103873559A (en) * | 2014-02-13 | 2014-06-18 | 南京斯坦德通信股份有限公司 | Database all-in-one machine capable of realizing high-speed storage |
| CN104035522A (en) * | 2014-06-16 | 2014-09-10 | 南京云创存储科技有限公司 | Large database appliance |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106066890A (en) | 2016-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106066890B (en) | Distributed high-performance database all-in-one machine system | |
| CN107967124B (en) | Distributed persistent memory storage system and method | |
| US11029853B2 (en) | Dynamic segment allocation for write requests by a storage system | |
| CN102012791B (en) | Flash based PCIE (peripheral component interface express) board for data storage | |
| CN103873559A (en) | Database all-in-one machine capable of realizing high-speed storage | |
| US20180107601A1 (en) | Cache architecture and algorithms for hybrid object storage devices | |
| CN102024044A (en) | Distributed file system | |
| CN103455577A (en) | Multi-backup nearby storage and reading method and system of cloud host mirror image file | |
| CN102637147A (en) | Storage system using solid state disk as computer write cache and corresponding management scheduling method | |
| CN103516549B (en) | A kind of file system metadata log mechanism based on shared object storage | |
| CN103049220A (en) | Storage control method, storage control device and solid-state storage system | |
| CN105468297A (en) | Quick synchronization method for master and slave device data in cloud storage system | |
| CN109240603A (en) | full flash memory server | |
| CN111736764B (en) | Storage system of database all-in-one machine and data request processing method and device | |
| CN112379825B (en) | Distributed data storage method and device based on data feature sub-pools | |
| CN113760796B (en) | SSD solid state disk based on HBM buffering | |
| US11416403B2 (en) | Method and apparatus for performing pipeline-based accessing management in storage server with aid of caching metadata with hardware pipeline module during processing object write command | |
| WO2017107162A1 (en) | Heterogeneous hybrid internal storage component, system, and storage method | |
| CN102929813A (en) | Method for designing peripheral component interconnect express (PCI-E) interface solid hard disk controller | |
| CN100383721C (en) | Isomeric double-system bus objective storage controller | |
| CN208622084U (en) | Full flash memory server | |
| CN101859232A (en) | Variable length data memory interface | |
| Imazaki et al. | EFFICIENT SNAPSHOT METHOD FOR ALL-FLASH ARRAY. | |
| CN114415985A (en) | Stored data processing unit based on numerical control separation architecture | |
| KR20230163238A (en) | Computing system for managing distributed storage devices, and method of operating the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |