CN107294771A - A kind of efficient deployment system and application method suitable for big data cluster - Google Patents
A kind of efficient deployment system and application method suitable for big data cluster Download PDFInfo
- Publication number
- CN107294771A CN107294771A CN201710348859.5A CN201710348859A CN107294771A CN 107294771 A CN107294771 A CN 107294771A CN 201710348859 A CN201710348859 A CN 201710348859A CN 107294771 A CN107294771 A CN 107294771A
- Authority
- CN
- China
- Prior art keywords
- module
- deployment
- operating system
- big data
- environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/04—Network management architectures or arrangements
- H04L41/042—Network management architectures or arrangements comprising distributed management centres cooperatively managing the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Stored Programmes (AREA)
Abstract
The present invention relates to big data application and management domain, and in particular to a kind of efficient deployment system and application method suitable for big data cluster.A kind of efficient deployment system suitable for big data cluster, it is characterised in that：Include allocating operating system module, framework deployment module and instrument start-up module；The environment that the allocating operating system module is used for operating system is disposed, comprising the starting module for start-up operation system environments deployment program, for whether testing the environment of the operating system in accordance with the test module of rule and for interrogating and examining operating system until what is meeted the requirements interrogates and examines module；The framework deployment module is used for the environment deployment program of Bootup infrastructure.It is an object of the invention to provide a kind of efficient deployment system and application method suitable for big data cluster, the system can coordinate Ambari automatically dispose, connection linux deployed environments, Ambari installation environments simultaneously carry out integrated deployment, the quick efficiency for improving Hadoop clustered deploy(ment)s.
Description
Technical field
The present invention relates to big data application and management domain, and in particular to a kind of efficient deployment suitable for big data cluster
System and application method.
Background technology
This century is the epoch of information technology high development, especially in recent years, with computer and information technology
Fast development and popularization and application, the scale of sector application system expand rapidly, the exponentially form of the data produced by sector application
Grown at top speed.The measurement unit of data shoots up as TB and PB from traditional KB, MB, GB.Even to this day, number
Reach that hundreds of TB even industry of tens of PB or hundreds of PB scales or enterprise have occurred according to scale, these industries or enterprise
Data processing needs be not that the disposal ability of traditional computer technology and information system can be handled, therefore, greatly
The management of data and the research of application mode are arisen at the historic moment, and are sought effective big data treatment technology, method, means and are had become
The active demand of real world.
In the application field of big data, HADOOP is a framework known to technical staff.Hadoop be one by
The distributed system architecture of Apache funds club exploitation, it realizes a distributed file system HDFS, i.e.,
Hadoop Distributed File System, the characteristics of HDFS has high fault tolerance can provide high-throughput and be answered to access
With the data of program, it is adapted to the application program for possessing super large data set.Wherein, the most crucial design of HADOOP framework is exactly
HDFS and MAPREDUCE.The former is used to store the file in HADOOP clusters on all memory nodes, is used as the core of storage.
And the latter then provides calculating for the data of magnanimity.User can rely on exploitation and fortune on this Distributed Computing Platform of HADOOP
The application program of row processing mass data.HADOOP is due to its high reliability, high scalability, high-efficiency, high fault tolerance and low
Cost and extensively liked by user, user can in the case where not knowing about distributed low-level details, develop distributed program.Fully
High-speed computation and storage are carried out using the power of cluster.
In the engineering that HDOOP is managed and monitored, people can select to use Ambari instruments.Ambari is used as mesh
Preceding main flow instrument, has used multiple advanced Katyuan technologies such as：Puppet, jetty, ember, jsden, ruby, spring etc..
As visual deployment big data component, the management difficulty of big data is greatly reduced, is had become using ambari deployment
The mainstream solution in big data field.
But, ambari is often deployed under linux environment, is faced with linux environment complexity, HADOOP ecosphere versions
Various, the cost of deployment is continuously increased.
The content of the invention
It is an object of the invention to provide a kind of efficient deployment system and application method suitable for big data cluster, this is
System can coordinate Ambari automatically dispose, and connection linux deployed environments, Ambari installation environments simultaneously carry out integrated deployment,
The quick efficiency for improving Hadoop clustered deploy(ment)s.
The present invention above-mentioned technical purpose technical scheme is that：One kind is applied to big data cluster
Efficient deployment system, it is characterised in that：Include allocating operating system module, framework deployment module and instrument start-up module；
The environment that the allocating operating system module is used for operating system is disposed, comprising for start-up operation system environments portion
The starting module of administration's program, for test the operating system environment whether in accordance with rule test module and for interrogating and examining behaviour
Make system until what is meeted the requirements interrogates and examines module；
The framework deployment module is used for the environment deployment program of Bootup infrastructure；
The instrument start-up module is used for the instrument for starting cluster supply, management and monitoring.
As a preference of the present invention, the operating system of the operating system environment deployment module deployment operates system for linux
System.
As a preference of the present invention, the framework of the framework deployment module deployment is hadoop distributed system architectures, institute
The cluster management instrument for stating the startup of instrument start-up module is ambari instruments.
As a preference of the present invention, the framework deployment module includes framework starting module, architecture testing module.
As a preference of the present invention, the installation and deployment module includes machine system configuration module, clustered node configuration mould
Block and installation kit download module；
The clustered node configuration module is used for the suitable operating system of stage+module based on cloud computing；
The machine system configuration module is used for the parameter improvement of operating system.
It is used for as a preference of the present invention, the installation kit download module is included by the locally downloading operation system of installation kit
System installation kit download module and framework installation kit download module；The operating system installation kit download module is used to download big data
Installation kit under Linux environment, the framework installation kit download module is used to download hadoop installation kits.
As a preference of the present invention, the installation kit download module also includes source configuration module, the source configuration module will
Source required for the operating system installation kit download module and the framework installation kit download module is configured in home environment.
A kind of application method of efficient deployment system suitable for big data cluster, is comprised the following steps：
Step 1：Installation and deployment step；
Stage+module suitable operating system of the clustered node configuration module based on cloud computing；
The parameter that the machine system configuration module is used for operating system is improved and set；
The installation kit that operating system and framework need is installed into locally by installation kit download module；
Step 2：Installation steps；
The installation kit downloaded in step 1 is installed in home environment；
Step 3：Operating system environment deploying step；
Linux environment deployment programs are opened, and deployment is tested, the operation in accordance with ambari is tested whether, interrogates and examines
Modify and set in the undesirable part of environment deployment program of the module to linux；
Step 4：Framework deploying step；
Bootup infrastructure deployment module, automatic deployment is carried out to architecture environment；
Step 5：Instrument start-up step；
When step 4 is detected, after test is normal, start instrument, complete deployment.
As a preference of the present invention, in step 1, comprising environment installation steps and principal and subordinate's setting steps, being installed in environment
In step, operating system is arranged in multiple host, in principal and subordinate's setting steps, main equipment and slave unit, institute are set respectively
Main equipment quantity is stated more than two.
As a preference of the present invention, in step 1, also comprising source configuration step, in this step, by all kinds of installation kit institutes
The source needed is all configured local.
Brief description of the drawings
Fig. 1 is the configuration diagram of the present invention；
Fig. 2 is the configuration diagram of installation and deployment module in the present invention；
Fig. 3 is the schematic flow sheet of the present invention.
Embodiment
Specific examples below is only explanation of the invention, and it is not limitation of the present invention, art technology
Personnel can make the modification without creative contribution to the present embodiment as needed after this specification is read, but as long as
All protected in scope of the presently claimed invention by Patent Law.
Embodiment 1, as described in this paper background technology, in today of information technology high development, many industries are many
The big data quantity of enterprise far beyond existing traditional computing technique and the disposal ability of information system, therefore, is sought
Effective big data treatment technology, ways and means is asked to have become the active demand of real world.
According to the investigation in industry and analysis report, the current total amount of data of Baidu alreadys exceed 1000PB, needs daily
The web data of processing up to 10PB --- 100PB；As the transaction platform that China E-Commerce Business amount of flow is maximum, Taobao,
The transaction data amount of accumulation is also up to more than 100PB；World-renowned Information Sharing platform, the twitter of social platform is every
Its quantity that gives out information also has been over 200,000,000 message, and the amount of posting reaches 80,000,000 to Chinese Sina weibo daily.
Not just internet environment, the data of communication environment are also in this way, the telephone logs of one province of China Mobile
Data are monthly up to 0.5PB --- 1PB；One public security bureau of provincial capital road vehicles monitoring data 3 years up to 20,000,000,000, it is total
Amount reaches 120TB.According to the IDC research reports prediction of world's authority IT information consultations analysis company, whole world data volume will in future
Explosive growth can be obtained, the situation of blowout occurs, the 0.8ZB said from 2009 rises to the 35ZB of the year two thousand twenty, ZB is similarly
Data metering unit, 1ZB=1000EB=1000000PB, this is between 10 years, and data total amount will increase by 44 times, annual average annual increasing
Length is more up to 40%.
Usually it is distributed file system by computer system configurations on the data structure field of big data.It is general next
Say, distributed file system can enable to access the file in being stored in these distributed file systems from multiple main frames.
Main frame can be positioned remotely, and can communicatedly be coupled with distributed file system via such as computer network.It is right
The access of file can enable a client to read or change the file being present in distributed file system and/or cause visitor
Family end can add new file to distributed file system.
Distributed file system can be by copying files to provide some advantages (such as reliability).By with file
Multiple copies, even if some parts of distributed file system are damaged, user can also access these files.
Computer cluster generally includes one group of connecting node for being configured to operate together.It is used as the knot for connecting into cluster
Really, node can be considered as to individual unit and is operated as individual unit.It can be connected for example, by computer network
Node.In the technical program, what is used is exactly distributed file system, Hadoop distributed file systems.
Hadoop distributed file systems can include single host node, and (the single host node can be referred to as title section
Point) and working node cluster.Host node can operate to coordinate the access to file.Working node can be with storage file
And/or perform the various actions related to the file accessed and/or modification is stored in Hadoop distributed file systems.
And Ambari, as current main flow instrument, use multiple advanced Katyuan technologies such as：puppet,jetty,
Ember, jsden, ruby, spring etc..As visual deployment big data component, the management for greatly reducing big data is difficult
Degree, the mainstream solution for having become big data field is disposed using ambari.
Ambari has supported hadoop most of components at present, for example including HDFS, MapReduce, Hive, Pig,
Spark etc..With the lifting of the maturity of the operational mode to ambari, ambari not only only supports the pipe of hadoop cluster
Reason, also inherits other definition services, such as elasticsearch, also apache drill etc. so that ambari couples
Hadoop management is more convenient.
Although management of the ambari to hadoop is more and more convenient, before management of the ambari to hadoop is realized,
Also need to configure hadoop running environment, the running environment to ambari is configured.And itself, Linux operations
System environments is more complicated, and hadoop ecosphere version also compares many, and the deployment to ambari is not easy to, it is necessary to spend work
Make personnel disposed manually, software installation, test, regulation, parameters revision etc., the cost of deployment is higher.
In the technical program, environment installation steps and principal and subordinate's setting steps are carried out first, many machines are chosen, and behaviour is installed
Make system.Linux operating systems may be selected.Linux is class Unix operating systems, is one multi-purpose based on POSIX and UNIX
Family, multitask, the operating system for supporting multithreading and multi -CPU.It can run main unix tool software, application program and net
Network agreement.It supports 32 and 64 hardware.Linux inherits design philosophys of the Unix using network as core, is a performance
Stable multiple-user network operating system.
In the present embodiment, in many machines, linux centos 7 version, CentOS (Community are installed
Enterprise Operating System, i.e.,：Community's Enterprise Operation System) it is one of Linux releases, it comes from
Red Hat Enterprise Linux provide that the source code disengaged is compiled according to open source code and formed.
, it is necessary to carry out the setting of principal and subordinate to multiple host after installation, multiple main equipments and multiple slave units are set.For example,
11 main frames altogether, set 4 main equipments, i.e. master, and remaining 7 are slave unit, i.e. slave, and ensure disk root
It is not less than 40G.
As a distributed structure/architecture, main equipment, i.e. master complete any distribution and scheduling, for example, total by one
Calculating task be divided into several parts, and require that slave unit, i.e. slave are performed.Slave unit is completed after calculating task,
Result of calculation is fed back on main equipment, main equipment carries out collecting calculating.
Under normal conditions, the quantity of main equipment 1 or 2 is just enough, but in the present embodiment, is set to 4,
Namenode HA needs 2, ResourceManager HA needs 2, each self installation, and do not interfere with each other, greatly improve
The High Availabitity of Hadoop clusters, it is to avoid because wherein one master machine of delaying, brings the out of service of group service.
After completing above-mentioned environment installation steps and principal and subordinate's setting steps, in addition it is also necessary to carry out other installation and deployment steps
Suddenly.Specifically, needing to carry out machine system configuration and installation kit download.
Machine system configuration module is operated, and starts to carry out big data need for improved to linux environment.Including in some parameters
Adjustment and setting.In addition, it is necessary to which the source of progress configuration step, that is, get out every installation kit before installation kit download is carried out
Yum sources, these sources are all configured local.Yum sources are quite just a directory entries, when we are soft using the installation of yum mechanism
During part, if desired depended software is installed, then yum mechanism will according to good path searching depended software defined in yum sources, and
Depended software is installed.
YUM is that " Yellow dog Updater, Modified " abbreviation, are a software package managers, and YUM is from specifying
Place (the rpm packet addresses of related web site or local rpm paths) download and RPM bags and install automatically, can be good at solving
Certainly dependence problem.In the technical program, i.e., yum sources are localized.
, it is necessary to carry out the installation of all kinds of installation kits after the completion of yum sources are configured.In the technical program, this kind of installation kit
Mainly there are two classes.One class is all kinds of installation kits of the big data under Linux environment, including jdk nts ntpd etc., this class
Operation is downloaded by operating system installation kit download module.JDK is Java development kits (Java Development
Kit abbreviation).It is a kind of development environment for being used to build application program, applet and the component issued in Java platform.
JDK is the basis of all java application programs, and all java application programs are built on this.It is one group of API,
It could also say that some java Class.And NFS is writing a Chinese character in simplified form for Network File System, i.e. NFS .NFS
Allow a system on network with other people share directories and file.By using NFS, user and program can be local as accessing
File equally accesses the file on far end system.
It is another kind of, it is hadoop program and associated component installation kit, this class is entered by framework installation kit download module
Row, which is downloaded, to be installed.
In these programs or component, comprising HDFS, i.e. Hadoop Distributed File System, i.e.,
Hadoop distributed file systems, are used as Apache Nutch architecture.Comprising MapReduce, this is, Map concept
It is mapping, Reduce concept is then reduction, and it is very easy to programming personnel and the program of oneself is operated in into distributed system
On system, it is highly suitable for the concurrent operation of large-scale dataset.In the technical program, MapReduce major function is several
Dispatched according to dividing with calculating task：System is automatic to be divided into many data blocks by a pending big data of operation, each
Data block corresponds to calculating task, and Automatic dispatching calculate node handles corresponding data block.Operation and task scheduling
Function is mainly responsible for distribution and scheduling calculate node (Map nodes or Reduce nodes), while being responsible for monitoring holding for these nodes
Row state, and it is responsible for the Synchronization Control that Map nodes are performed.Also need to that YARN, i.e. Yet Another Resource are installed
Negotiator, it is a kind of new Hadoop explorers, and it is a universal resource management system, can be upper layer application
There is provided unified resource management and scheduling, it is introduced as cluster in terms of utilization rate, resource unified management and data sharing
Bring big advantages.It has many advantages, such as, such as, substantially reduces JobTracker (namely present
ResourceManager resource consumption), and allow monitor each Job subtasks (tasks) state program distribution
, it is safer, more graceful.User can be written from oneself AppMst to different programming models, allow further types of programming model
It can run in Hadoop clusters, for saving as unit within the expression of resource, than more reasonable with remaining slot numbers before.
Except component described above, in addition it is also necessary to install such as TEZ, HIVE, Hbase, Pig, Sqoop, Oozie,
The components such as Zookeeper, Falcon, Storm, Flume, Ambari Metics.
After completing these and mapping out the work, i.e., linux operating system and master-slave equipment have been provided with, machine
System and clustered node are all configured to be completed, and the installation kit in yum sources, linux installation kit and hadoop is also all downloaded and pacified
Install complete.Into step 3, i.e. operating system environment deploying step.
As shown in Fig. 3 and Fig. 1, first, the starting module in allocating operating system module starts linux environment deployment
Program, program can carry out automatic detection after starting to linux environment, and the particular content detected is, current linux ring
Border, if the ambari service requirement in accordance with after.Test order can be default with User Defined, when test module works it
Afterwards, the test result of module meeting matching test module is interrogated and examined, disk is carried out to the situation that ambari is not met under current linux environment
Look into and change, until compliance with system preset requirement.
After step 3 is completed, automatically into step 4, the principle of step 4 is identical with step 3, but object is from linux ring
Border deployment becomes hadoop environment deployment, i.e. framework deploying step.In this step, hadoop environment deployment journey is started
Sequence, and equally being tested, test order can be carried out with system it is self-defined default, when hadoop environment deployment test gets off one
Cut normally, system can just enter step 5, instrument start-up step formally start ambari, complete ambari operation.
After Ambari operations, all environment can be tested again, test whether the feelings that there is poorly-implemented
Condition, by visual operation interface, the operation to hadoop is managed and monitored.
Such mode of operation, realizes the automation of linux operating system environments deployment, and support automatic detection and oneself
Moving plate is looked into, meanwhile, automatic deployment and the test of hadoop environment are also achieved, good environment is provided for Ambari startup
Factor, solve that user produces when being disposed in face of ambari in the past because Linux environment is complicated, hadoop ecosphere versions
Deployment efficiency that is many and producing is low, the high technical problem of cost.
The technical program possesses advantages below：
1st, comprising multiple main equipments and slave unit, each self installation is not interfere with each other, and greatly improves the High Availabitity of Hadoop clusters
Performance.
2nd, the yum sources of all installation kits are configured in home environment so that using just during the network environment deployment of system
It is prompt.
3rd, installation kit download module includes the operating system installation kit download module and the frame towards hadoop towards linux
Structure installation kit download module.
4th, linux environment deployment program can be detected and interrogated and examined automatically after starting, so as to carry out the operation in accordance with ambari
It is required that.
5th, hadoop environment deployment program can be tested automatically upon actuation, can just be started after being successfully tested
ambari。
Claims (10)
1. a kind of efficient deployment system suitable for big data cluster, it is characterised in that：Include allocating operating system module, framework
Deployment module and instrument start-up module；
The environment that the allocating operating system module is used for operating system is disposed, comprising for start-up operation system environments deployment journey
The starting module of sequence, for test the operating system environment whether in accordance with rule test module and for interrogate and examine operation system
System is until what is meeted the requirements interrogates and examines module；
The framework deployment module is used for the environment deployment program of Bootup infrastructure；
The instrument start-up module is used for the instrument for starting cluster supply, management and monitoring.
2. a kind of efficient deployment system suitable for big data cluster according to claim 1, it is characterised in that：The behaviour
The operating system for making the deployment of system environments deployment module is linux operating systems.
3. a kind of efficient deployment system suitable for big data cluster according to claim 1, it is characterised in that：The frame
The framework of structure deployment module deployment is hadoop distributed system architectures, the cluster management instrument that the instrument start-up module starts
For ambari instruments.
4. a kind of efficient deployment system suitable for big data cluster according to claim 1 or 2 or 3, it is characterised in that：
The framework deployment module includes framework starting module, architecture testing module.
5. a kind of efficient deployment system suitable for big data cluster according to claim 1 or 2 or 3, it is characterised in that：
The installation and deployment module includes machine system configuration module, clustered node configuration module and installation kit download module；
The clustered node configuration module is used for the suitable operating system of stage+module based on cloud computing；
The machine system configuration module is used for the parameter improvement of operating system.
6. a kind of efficient deployment system suitable for big data cluster according to claim 5, it is characterised in that：The peace
Filling bag download module and including is used under installation kit locally downloading operating system installation kit download module and framework installation kit
Carry module；The operating system installation kit download module is used to download installation kit of the big data under Linux environment, the framework
Installation kit download module is used to download hadoop installation kits.
7. a kind of efficient deployment system suitable for big data cluster according to claim 6, it is characterised in that：The peace
Fill bag download module and also include source configuration module, the source configuration module is by the operating system installation kit download module and described
Source required for framework installation kit download module is configured in home environment.
8. one kind is based on a kind of efficient deployment system suitable for big data cluster described in claim 1-7 any one
Application method, it is characterised in that comprise the following steps：
Step 1：Installation and deployment step；
Stage+module suitable operating system of the clustered node configuration module based on cloud computing；
The parameter that the machine system configuration module is used for operating system is improved and set；
The installation kit that operating system and framework need is installed into locally by installation kit download module；
Step 2：Installation steps；
The installation kit downloaded in step 1 is installed in home environment；
Step 3：Operating system environment deploying step；
Linux environment deployment programs are opened, and deployment is tested, the operation in accordance with ambari is tested whether, interrogates and examines module
Modify and set in the undesirable part of environment deployment program to linux；
Step 4：Framework deploying step；
Bootup infrastructure deployment module, automatic deployment is carried out to architecture environment；
Step 5：Instrument start-up step；
When step 4 is detected, after test is normal, start instrument, complete deployment.
9. a kind of application method of efficient deployment system suitable for big data cluster according to claim 8, its feature
It is：In step 1, comprising environment installation steps and principal and subordinate's setting steps, in environment installation steps, operating system is installed
In multiple host, in principal and subordinate's setting steps, main equipment and slave unit are set respectively, the main equipment quantity is more than two.
10. a kind of application method of efficient deployment system suitable for big data cluster according to claim 9, its feature
It is：In step 1, also in this step, the source required for all kinds of installation kits is all configured at this comprising source configuration step
Ground.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710348859.5A CN107294771A (en) | 2017-05-17 | 2017-05-17 | A kind of efficient deployment system and application method suitable for big data cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710348859.5A CN107294771A (en) | 2017-05-17 | 2017-05-17 | A kind of efficient deployment system and application method suitable for big data cluster |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107294771A true CN107294771A (en) | 2017-10-24 |
Family
ID=60095334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710348859.5A Pending CN107294771A (en) | 2017-05-17 | 2017-05-17 | A kind of efficient deployment system and application method suitable for big data cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107294771A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681777A (en) * | 2018-05-07 | 2018-10-19 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of the machine learning program operation based on distributed system |
| CN109165022A (en) * | 2018-07-18 | 2019-01-08 | 山东中创软件商用中间件股份有限公司 | A kind of big data cluster dispositions method, system, medium and equipment |
| CN111258589A (en) * | 2020-05-06 | 2020-06-09 | 成都四方伟业软件股份有限公司 | Multi-platform adaptation method for big data operation and maintenance monitoring |
| CN111309297A (en) * | 2020-03-27 | 2020-06-19 | 中国工商银行股份有限公司 | Script development system and method |
| CN111641680A (en) * | 2020-05-11 | 2020-09-08 | 紫光云技术有限公司 | Management method of Ambari high-availability cluster |
| CN112181445A (en) * | 2020-08-28 | 2021-01-05 | 深圳市华讯方舟光电技术有限公司 | Method, terminal and storage medium for improving operation efficiency of electromagnetic simulation software |
| CN114943091A (en) * | 2022-07-27 | 2022-08-26 | 成都中科合迅科技有限公司 | Elastic search encryption searching method based on linux kernel block device encryption function |
| CN116909584A (en) * | 2023-05-06 | 2023-10-20 | 广东国地规划科技股份有限公司 | Deployment method, device, equipment and storage medium of space-time big data engine |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105022779A (en) * | 2015-05-07 | 2015-11-04 | 云南电网有限责任公司电力科学研究院 | Method for realizing HDFS file access by utilizing Filesystem API |
| CN105468720A (en) * | 2015-11-20 | 2016-04-06 | 北京锐安科技有限公司 | Method for integrating distributed data processing systems, corresponding systems and data processing method |
| CN106301892A (en) * | 2016-08-02 | 2017-01-04 | 浪潮电子信息产业股份有限公司 | Hue service arrangement based on Apache Ambari and configuration and surveillance method |
-
2017
- 2017-05-17 CN CN201710348859.5A patent/CN107294771A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105022779A (en) * | 2015-05-07 | 2015-11-04 | 云南电网有限责任公司电力科学研究院 | Method for realizing HDFS file access by utilizing Filesystem API |
| CN105468720A (en) * | 2015-11-20 | 2016-04-06 | 北京锐安科技有限公司 | Method for integrating distributed data processing systems, corresponding systems and data processing method |
| CN106301892A (en) * | 2016-08-02 | 2017-01-04 | 浪潮电子信息产业股份有限公司 | Hue service arrangement based on Apache Ambari and configuration and surveillance method |
Non-Patent Citations (1)
| Title |
|---|
| 沈钊伟: "Ambari--大数据平台的搭建利器", 《HTTPS://WWW.IBM.COM/DEVELOPERWORKS/CN/OPENSOURCE/OS-CN-BIGDATA-AMBARI/INDEX.HTML》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681777A (en) * | 2018-05-07 | 2018-10-19 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of the machine learning program operation based on distributed system |
| CN108681777B (en) * | 2018-05-07 | 2021-07-20 | 北京京东尚科信息技术有限公司 | Method and device for running machine learning program based on distributed system |
| CN109165022A (en) * | 2018-07-18 | 2019-01-08 | 山东中创软件商用中间件股份有限公司 | A kind of big data cluster dispositions method, system, medium and equipment |
| CN111309297A (en) * | 2020-03-27 | 2020-06-19 | 中国工商银行股份有限公司 | Script development system and method |
| CN111309297B (en) * | 2020-03-27 | 2023-09-15 | 中国工商银行股份有限公司 | Script development system and method |
| CN111258589A (en) * | 2020-05-06 | 2020-06-09 | 成都四方伟业软件股份有限公司 | Multi-platform adaptation method for big data operation and maintenance monitoring |
| CN111641680A (en) * | 2020-05-11 | 2020-09-08 | 紫光云技术有限公司 | Management method of Ambari high-availability cluster |
| CN112181445A (en) * | 2020-08-28 | 2021-01-05 | 深圳市华讯方舟光电技术有限公司 | Method, terminal and storage medium for improving operation efficiency of electromagnetic simulation software |
| CN114943091A (en) * | 2022-07-27 | 2022-08-26 | 成都中科合迅科技有限公司 | Elastic search encryption searching method based on linux kernel block device encryption function |
| CN116909584A (en) * | 2023-05-06 | 2023-10-20 | 广东国地规划科技股份有限公司 | Deployment method, device, equipment and storage medium of space-time big data engine |
| CN116909584B (en) * | 2023-05-06 | 2024-05-24 | 广东国地规划科技股份有限公司 | Deployment method, device, equipment and storage medium of space-time big data engine |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107294771A (en) | A kind of efficient deployment system and application method suitable for big data cluster | |
| CN106020930B (en) | A kind of application management method and system based on application container | |
| CN104317610B (en) | Method and device for automatic installation and deployment of hadoop platform | |
| CN106534291B (en) | Voltage monitoring method based on big data processing | |
| US8495352B2 (en) | System and method for instantiation of distributed applications from disk snapshots | |
| US20110004564A1 (en) | Model Based Deployment Of Computer Based Business Process On Dedicated Hardware | |
| US20160110434A1 (en) | Method and system that determine whether or not two graph-like representations of two systems describe equivalent systems | |
| US20100280863A1 (en) | Automated Model Generation For Computer Based Business Process | |
| US20100262558A1 (en) | Incorporating Development Tools In System For Deploying Computer Based Process On Shared Infrastructure | |
| CN106293820A (en) | Exploitation test O&M integral system, deployment, full dose and increment updating method | |
| WO2009082384A1 (en) | Modelling computer based business process and simulating operation | |
| CN102075381A (en) | Automatic test platform server and system applied to cloud storage | |
| CN105959390A (en) | Unified management system and method of micro services | |
| Leite et al. | Automating resource selection and configuration in inter-clouds through a software product line method | |
| CN111324599A (en) | Block chain experiment system and management method | |
| CN109150964A (en) | A kind of transportable data managing method and services migrating method | |
| Štefanič et al. | Non-functional requirements optimisation for multi-tier cloud applications: An early warning system case study | |
| CN104794217A (en) | Tile map data and service updating method and system based on parallel computing mode | |
| Pham et al. | Autonomic fine-grained migration and replication of component-based applications across multi-clouds | |
| CN109240757A (en) | Configuration management system and method in a kind of big data component set | |
| Kikuchi et al. | Configuration procedure synthesis for complex systems using model finder | |
| Schmieders et al. | Architectural runtime models for privacy checks of cloud applications | |
| Duan et al. | A Case Study on Five Maturity Levels of A Kubernetes Operator | |
| Zhang et al. | Intelligent Grid Operation and Maintenance Management and Command Platform Based on Computer Distributed Network | |
| Shih | MASS HDFS: multi-agent spatial simulation hadoop distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171024 |
|
| RJ01 | Rejection of invention patent application after publication |