CN204102026U - Large database concept all-in-one - Google Patents
Large database concept all-in-one Download PDFInfo
- Publication number
- CN204102026U CN204102026U CN201420318958.0U CN201420318958U CN204102026U CN 204102026 U CN204102026 U CN 204102026U CN 201420318958 U CN201420318958 U CN 201420318958U CN 204102026 U CN204102026 U CN 204102026U
- Authority
- CN
- China
- Prior art keywords
- utility
- large database
- database concept
- switch
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 238000012545 processing Methods 0.000 claims abstract description 40
- 239000013307 optical fiber Substances 0.000 claims abstract description 8
- 238000003491 array Methods 0.000 claims abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000003860 storage Methods 0.000 abstract description 4
- 230000004044 response Effects 0.000 abstract description 2
- 238000000034 method Methods 0.000 description 6
- 238000007726 management method Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000009825 accumulation Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 238000009412 basement excavation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000008140 language development Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The utility model discloses large database concept all-in-one, belong to technical field of data processing.The large database concept all-in-one that the utility model provides, comprises several database processing units, several SSD disk arrays, optical fiber switch and SAN switch; Several SSD disk arrays are all connected with SAN switch, and several database processing units are connected with SAN switch and optical fiber switch respectively.The utility model adopts SSD disk array to improve read or write speed as shared storage device, realizes multinode load balancing, improves the response time by parallel database technology, can expanding system as required, ensures the stable of system performance.For ensureing that large database concept all-in-one runs continually and steadily, SAN switch adopts two, for double copies.The utility model adopts JobKeeper distributed scheduling engine, makes large database concept all-in-one reach load balancing, cluster performance is more excellent.
Description
Technical field
The utility model relates to a kind of large database concept all-in-one, is specifically the fast and large database concept all-in-one of stable performance of a kind of processing speed, belongs to technical field of data processing.
Background technology
Along with the develop rapidly of computer technology, increasing fast in geometry level of each FIELD Data of each row.These data are from each side, and from collecting sensor, digital pictures, the online video data of weather condition, the global positioning system signal etc. to the transaction record of shopping at network, mobile phone is complete.The sharply expansion of companion data scale, the data volume of every profession and trade accumulation is more and more huger, data type also more and more, data structure becomes increasingly complex, surmount the limit of power of traditional data management system, tupe, traditional serial data storehouse system has been difficult to the application demand adapting to this growth at full speed, in production practices, show obvious scarce capacity, the data processing needs of large data age cannot be met.
Summary of the invention
Technical problem to be solved in the utility model is to overcome prior art defect, provides the large database concept all-in-one that a kind of processing speed is fast, operating efficiency high energy adapts to large data processing.
In order to solve the problems of the technologies described above, the large database concept all-in-one that the utility model provides, comprises several database processing units, several SSD disk arrays, optical fiber switch and SAN switch; Several SSD disk arrays are all connected with SAN switch, and several database processing units are connected with SAN switch and optical fiber switch respectively.
The beneficial effects of the utility model are: the utility model adopts SSD disk array to improve read or write speed as shared storage device, realize multinode load balancing, the response time is improved by parallel database technology, can expanding system as required, ensure the stable of system performance; The utility model can enable the process of EB DBMS level second, greatly high user performs the service efficiency after query manipulation, it not only has very high performance advantage when retrieval and indexing data, the storage of all right supported data warehouse, data depth excavation and Intellectual analysis etc.
For ensureing that large database concept all-in-one runs continually and steadily, described SAN switch is two, for double copies.
This large database concept all-in-one adopts JobKeeper distributed scheduling engine, makes to reach large database concept all-in-one load balancing, cluster performance is more excellent.
Accompanying drawing explanation
Fig. 1 is the utility model large database concept all-in-one machine structure figure;
Fig. 2 is the inner connection diagram of the utility model large database concept all-in-one;
Fig. 3 is JobKeeper distributed scheduling automotive engine system Organization Chart.
Embodiment
Below in conjunction with accompanying drawing, the utility model is elaborated.
As shown in Figure 1, the utility model large database concept all-in-one comprises from bottom to top SSD redundant array of inexpensive disks 1, SSD disk array 2, SSD disk array 3, SAN switch 1, SAN switch 2, database processing unit 1, database processing unit 2, database processing unit 3 and optical fiber switch in installation code frame 1 successively, and all devices unifies powered operation by rack power supplies.
As shown in Figure 2, SSD redundant array of inexpensive disks 1, SSD disk array 2, SSD disk array 3 are interconnected communication with SAN switch 1, SAN switch 2 respectively, SAN switch 1, SAN switch 2 are connected with database processing unit 1, database processing unit 2, database processing unit 3 simultaneously respectively, realize data between SSD redundant array of inexpensive disks 1, SSD disk array 2, SSD disk array 3 and database processing unit 1, database processing unit 2, database processing unit 3 and transmit; Database processing unit 1, database processing unit 2, database processing unit 3 be connecting fiber switch all, and optical fiber switch provides external service network interface, connects external service network, realizes transmitting between data network.
In the utility model, each database processing unit arranges 2 pieces of HBA cards, for connecting SAN switch, and shared disk array.Each database processing unit installs unified relationship type database, and upper layer application is general database platform, and system automatically switches according to loading condition and accesses the database of isolated node.SSD disk array adopts flash memory as storage medium, and read or write speed relative mechanical hard disk is faster, and power dissipation ratio conventional hard is low, and the data realizing low-energy-consumption high-efficiency store.SAN switch is used for database processing unit shared disk array, realizes backup, ensure that the continual and steady operation of large database concept all-in-one by 2 SAN switches.
In this example, large database concept all-in-one adopts 3 SSD disk arrays and 3 database processing units, and in practical work process, the quantity of SSD disk array and database processing unit can increase and decrease as required, to realize the data processing needs of different stage.
In the utility model, each data processing unit independently loads system R, and shared array stores, and forms group schema.User submits database manipulation request to JobKeeper distributed scheduling engine.This operation requests, according to the actual motion loading condition of each data processing unit, is issued on the applicable data processing unit processed, feeds back to user's request results after the operation of this processing unit complete independently by JobKeeper distributed scheduling engine.Reach the advantage of the load balancing of efficient IOPS and the JobKeeper distributed scheduling engine utilizing SSD dish battle array in this manner, the efficiency of database manipulation is significantly improved.By introducing index module, executed in parallel framework and reading the mode of local disk, enable the process of EB DBMS level second.
As shown in Figure 3, the utility model large database concept all-in-one adopts JobKeeper distributed scheduling engine, JobKeeper is a kind of very-high performance distributed scheduling platform of Processing tasks, JobKeeper uninterruptedly can accept submitting to from each computing machine of task, again according to the pressure of machines all in current cluster, intelligence carries out allocating task, and then reaches cluster load balance, makes the performance of whole cluster remain optimum.Can also be monitored the situation of all machines of current distributed type assemblies by JobKeeper, tasks all in cluster is stopped, the function such as deletion.JobKeeper distributed scheduling engine comprises application layer, operation layer, data analysis layer, accumulation layer, virtual resources layer and dispatching platform, wherein:
Application layer: for managing the display module with result feedback is the basis carrying of whole system user oriented and developer.
Operation layer: for the business of the correlation function of application layer, digitized processing, for the demand task of application layer is carried out regularization division, forms unified place's physicochemical mode.
Data analysis layer: independently data processor, is the unified processing scheme to different demand data, carries out unified configuration management by JobKeeper dispatching platform.
Accumulation layer: store the result collection of data or the unit of other intermediate result collection.
Virtual resources layer: undertaken virtual by machine, forms wider service cluster.
JobKeeper distributed scheduling engine has following advantage:
1), high reliability: the management node Design Mode adopting " many main how standby, load balancing ", and the processing node Design Mode of " process real-time follow-up ", thus ensure that management node or processing node do not exist any Single Point of Faliure problem.
2), low dependence: adopt modular design method, provide service by the mode of unitized configuration and api interface to user.No matter which kind of language development is used in user's application, and which kind of processing mode, can transfer to platform to carry out unified management and running.
3), low intervention: adopt based on evented unified management pattern.No matter be dynamic expansion cluster disposal system ability, or the task transfers of Single Point of Faliure, or be real-time monitoring and the dynamic assignment of task, all automatically will complete in the unattended situation of system.
4), high real-time: under the support of high-level efficiency parallel distributed software, can realize the real-time distribution to task, Real-Time Scheduling, processes in real time.In the scope that machine performance allows, calculation task there will not be manifesting of accumulation, the control work of all tasks substantially all second level complete, there is unprecedented high efficiency.
The above is only preferred implementation of the present utility model; should be understood that; for those skilled in the art, can also make some improvement under the prerequisite not departing from the utility model principle, these improvement also should be considered as protection domain of the present utility model.
Claims (3)
1. a large database concept all-in-one, is characterized in that: described large database concept all-in-one comprises several database processing units, several SSD disk arrays, optical fiber switch and SAN switch; Several SSD disk arrays described are all connected with SAN switch, and several database processing units are connected with SAN switch and optical fiber switch respectively.
2. a kind of large database concept all-in-one according to claim 1, is characterized in that: described SAN switch is two, for double copies.
3. a kind of large database concept all-in-one according to claim 1, is characterized in that: described large database concept all-in-one adopts JobKeeper distributed scheduling engine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201420318958.0U CN204102026U (en) | 2014-06-16 | 2014-06-16 | Large database concept all-in-one |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201420318958.0U CN204102026U (en) | 2014-06-16 | 2014-06-16 | Large database concept all-in-one |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN204102026U true CN204102026U (en) | 2015-01-14 |
Family
ID=52270439
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201420318958.0U Expired - Lifetime CN204102026U (en) | 2014-06-16 | 2014-06-16 | Large database concept all-in-one |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN204102026U (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104035522A (en) * | 2014-06-16 | 2014-09-10 | 南京云创存储科技有限公司 | Large database appliance |
| CN107122264A (en) * | 2017-05-15 | 2017-09-01 | 成都优孚达信息技术有限公司 | mass data disaster-tolerant backup method |
-
2014
- 2014-06-16 CN CN201420318958.0U patent/CN204102026U/en not_active Expired - Lifetime
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104035522A (en) * | 2014-06-16 | 2014-09-10 | 南京云创存储科技有限公司 | Large database appliance |
| CN107122264A (en) * | 2017-05-15 | 2017-09-01 | 成都优孚达信息技术有限公司 | mass data disaster-tolerant backup method |
| CN107122264B (en) * | 2017-05-15 | 2020-06-09 | 成都优孚达信息技术有限公司 | Disaster-tolerant backup method for mass data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11422982B2 (en) | Scaling stateful clusters while maintaining access | |
| US9569457B2 (en) | Data processing method and apparatus for distributed systems | |
| US10394611B2 (en) | Scaling computing clusters in a distributed computing system | |
| CN105243155A (en) | Big data extracting and exchanging system | |
| CN102760081B (en) | The method and apparatus that resources of virtual machine distributes | |
| CN101866359B (en) | Small file storage and visit method in avicade file system | |
| US10061834B1 (en) | Incremental out-of-place updates for datasets in data stores | |
| CN107180113B (en) | Big data retrieval platform | |
| Tsai et al. | Scalable architectures for SaaS | |
| CN102012789A (en) | Centralized management type backup and disaster recovery system | |
| CN102164177A (en) | Method, device and system for sharing storage pool by cluster | |
| CN104035522A (en) | Large database appliance | |
| CN103312791A (en) | Internet of things heterogeneous data storage method and system | |
| CN103916459A (en) | Big data filing and storing system | |
| CN103037004A (en) | Implement method and device of cloud storage system operation | |
| CN104462185A (en) | Digital library cloud storage system based on mixed structure | |
| CN103491155A (en) | Cloud computing method and system for achieving mobile computing and obtaining mobile data | |
| CN103595799A (en) | Method for achieving distributed shared data bank | |
| WO2012101540A1 (en) | Application recovery in file system | |
| CN102508727A (en) | Method using software for power fail safeguard of caches in disk array | |
| CN103593420A (en) | Method for constructing heterogeneous database clusters on same platform by sharing online logs | |
| CN103294413A (en) | Mass data acquisition terminal supported distributed-memory real-time storage device and storage method | |
| CN102820998B (en) | Realize the dual computer fault-tolerant service system towards office application and date storage method thereof | |
| CN101788887A (en) | System and method of I/O cache stream based on database in disk array | |
| CN204102026U (en) | Large database concept all-in-one |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee | ||
| CP03 | Change of name, title or address |
Address after: Everwise road in Qinhuai District of Nanjing City, Jiangsu province 210000 No. 6 Baixia Nanjing high tech Industrial Park, building four, building 9 layer A Patentee after: NANJING YUNCHUANG BIG DATA TECHNOLOGY CO., LTD. Address before: Guanghua Road, Baixia District Nanjing city Jiangsu province 210014 No. 1 Baixia High-tech Industrial Park incubator building, 1 floor Patentee before: Nanjing Innovative Cloud Storage Technology Co., Ltd. |