KR101851664B1 - Small platform server for distributed processing big data and operating method - Google Patents

Abstract

The present invention relates to a small platform server including a plurality of nodes to which different domains are allocated for distributed big data processing. The present invention includes a middleware embedded control board for supplying power to the small platform server and measuring the temperature and humidity of the small platform server, a master node controlled by the middleware embedded control board and building a cluster for distributed big data processing, and a plurality of slave nodes to which respective roles are assigned by the master node and in which different components are installed.

Description

TECHNICAL FIELD [0001] The present invention relates to a compact platform server for large data distribution processing, and a method of operating the same,

The present invention relates to a platform server for large data distribution processing, and more particularly to a platform server for physically implementing a master node and a plurality of slave nodes in a big data platform server, and a small middleware embedded control board And a small platform server for large data distribution processing.

Due to the development of information and communication technology, the spread of mobile terminals, and the proliferation of Internet services, large-scale data is being generated and various kinds are being diversified. Big Data is not just about collecting and accumulating data, but by finding hidden patterns in a large amount of data, it is possible to detect anomalies through the patterns and predict the near future. , And public administration such as administration are increasing.

In particular, in the road traffic field, the road traffic DB has been remarkably developed in terms of quantitative aspects due to the continuous expansion of the road traffic information system. However, the value of the utilization value due to the absence of the system for reproducing meaningful information through the accumulated amount of accumulated road traffic data .

Recently, a platform for processing big data generated from a vehicle sensor, weather information, and open road traffic information has been requested. However, a server configuration for processing large data requires a high cost and a large space And when it is configured with general PC class hardware, the processing speed is lowered and cloud configuration is difficult.

Patent Document 1 discloses a method for managing a big data processing apparatus and a management system for performing the method. In one PC server, a large data base-based large data Discloses a technology for collectively managing installation, setting, and control for a system.

In addition, Japanese Patent Application Laid-Open No. 10-2015-0030332 (hereinafter referred to as "Patent Document 2") is an invention related to a data distribution processing system and an operation method thereof. In the first execution of a mapping task executed on a central processing unit, Discloses a technique for calculating the processing capability of each of the servers.

However, in Patent Literature 1 and Patent Literature 2, as in the present invention, a master node and a plurality of slave nodes are physically configured in one server, and a master node and slave nodes each having a domain in one server rack It has a point and a distinction.

SUMMARY OF THE INVENTION The present invention is directed to provide a compact large data platform server capable of distributed processing in order to predict a traveling environment based on a big data.

The present invention provides a small data platform server that physically implements a master node and a plurality of slave nodes each having a respective domain in one server.

The present invention relates to a small-sized platform server including a plurality of nodes to which different domains are allocated for big data distribution processing, and a middleware embedded control board for supplying power to the small-sized platform server and measuring the temperature and humidity of the small- A master node controlled by the middleware embedded control board and constructing a cluster for large data distribution processing, and a plurality of slave nodes allocated with respective roles by the master node and having different components installed therein do.

According to the present invention, it is possible to construct a small-size data distribution server with a low cost and high efficiency and efficiently use it in analyzing a traveling environment based on a big data.

In addition, the present invention can be applied to a master node and a plurality of slave nodes in order to perform big data distribution processing, and is highly applicable to IoT, smart factory, and security fields.

1 is a configuration diagram of a small platform server for large data distribution processing of the present invention.
2 is a schematic view illustrating a master node and a plurality of slave nodes according to the present invention.
3 is a block diagram of a middleware embedded control board of the present invention.
4 is a diagram of a server rack of a small platform server for big data distribution processing of the present invention.
5 is an actual implementation example of a small platform server for large data distribution processing of the present invention.
6 is an actual implementation example of the middleware embedded control board of the present invention.
7 is a flowchart illustrating a method of operating a small platform server for large data distribution processing of the present invention.
8 is an exemplary diagram illustrating a method of assigning a role of a master node of the present invention.
9 is an exemplary diagram illustrating a method for assigning roles of a slave node of the present invention.

It is to be understood that the specific structural or functional description of embodiments of the present invention disclosed herein is for illustrative purposes only and is not intended to limit the scope of the inventive concept But may be embodied in many different forms and is not limited to the embodiments set forth herein.

The embodiments according to the concept of the present invention can make various changes and can take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. It is not intended to be exhaustive or to limit the invention to the particular forms disclosed, but on the contrary, is intended to cover all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like are used to specify that there are features, numbers, steps, operations, elements, parts or combinations thereof described herein, But do not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached hereto.

1 is a configuration diagram of a small platform server for large data distribution processing of the present invention. 1, a small platform server 10 for large data distribution processing includes a big data server 100 including a plurality of nodes in a server rack 700, a middleware embedded control board 200, a power supply device 300, a blowing fan 400, a temperature / humidity sensor 500, and a communication port 600.

The big data server 100 may be composed of one master node 110 and at least first to ninth slave nodes 120 to 195. The master node 110 may be referred to as a master server or a name node, and a slave node may be referred to as a slave server or a data node. According to the embodiment, the master node may be designated as another node in the big data server 100. [ The master node can set the domain to build the big data cluster with the slave nodes. Here, the master node 110 and the first to ninth slave nodes 120 to 195 may be homogeneous servers having different data processing capabilities or heterogeneous servers having different data processing capabilities. Although the embodiment is implemented with ten nodes, the present invention is not limited thereto, and the nodes may be physically expanded and operated.

The middleware embedded control board 200 can control the power supply to the master node and the slave nodes in the big data server 100 by using the relays individually. Also, the measured data can be received from the temperature / humidity sensor and the measured data can be transmitted to the web. At this time, the data can be checked by controlling the sensor module SHT25 to check the temperature and humidity inside the big data server 100, and the sensor data collected every 2 seconds can be transmitted when requested from the web. Also, it is possible to interlock with an external sensor using an expansion port outside the big data server 100, and to control the power supply to the fan to discharge heat in the big data server 100.

The middleware embedded control board 200 may include ten relays to control the master node and nine slave nodes, and a terminal block for power supply. It can also include four fan power supply ports and five sensor ports. The middleware embedded control board 200 includes five sensor ports including a Uart 1 channel, an I 2 C 1 channel, and a 1 wire 3 channel, and may include 10 GPIOs for controlling respective relays. The relay control number is received from the web, and the relay control GPIO is set to the blocking state, and after 5 seconds, the relay control is set to the normal state again.

The power supply device 300 may be disposed on one side of the server rack 700 to supply power to the configuration of the big data server 100, the middleware embedded control board 200, and the like. The blower fan 400 is disposed to dissipate heat generated from a plurality of nodes, and is disposed between the three nodes of the front side of the server rack and the seven nodes of the rear side of the server rack. The temperature / humidity sensor 500 can sense the temperature and humidity of the small data server 10 and provide the data to the web server. The communication port 600 may be RJ45 according to an embodiment, but is not limited thereto. The server rack 700 includes a frame 710 and a blower 720.

It is possible to construct a small and large data distribution server with low cost and high efficiency by arranging it in the small platform server as above, and it can be used effectively in big data analysis. In addition, it is possible to apply various domains by configuring a master node and a plurality of slave nodes in one big data server.

2 is a schematic view illustrating a master node and a plurality of slave nodes according to the present invention.

Referring to FIG. 2, the Big Data Server 100 includes a master node 110 and first to ninth slave nodes 120 to 195. For example, the master node 110 can operate by installing a NameNode, a ResourceManager, an Ambari Server, and a Zookeeper Server 1.

The first to ninth slave nodes 120 to 195 may have a DataNode, a NodeManager, and an Amabri Client installed in common, and may be individually installed according to respective functions.

3 is a block diagram of a middleware embedded control board of the present invention.

Referring to FIG. 3, the middleware embedded control board 200 includes a microprocessor 210, a PMIC 220, a relay 230, and DC-DC converters 240 and 250. The middleware embedded control board 200 may include ten relays to control the master node and nine slave nodes, and a terminal block for power supply.

 It can also include four fan power supply ports and five sensor ports. The middleware embedded control board 200 includes five sensor ports including a Uart 1 channel, an I 2 C 1 channel, and a 1 wire 3 channel, and may include 10 GPIOs for controlling respective relays. The microprocessor 210 can transmit and receive data to and from the web through the sensor information Ethernet received from the five sensor ports, and the PMIC 220 can control power through UART communication with the microprocessor. DC-DC converters 240 and 250 may supply power to the power supply and the blower fan, respectively.

4 is a diagram of a server rack of a small platform server for big data distribution processing of the present invention.

Referring to FIG. 4, the server rack 700 is provided with a frame 710 for interchangeably mounting various information processing apparatuses and servers, and the blowing openings 720 and 740 are disposed on the front and rear surfaces thereof. The server rack 700 forms a blower fan box 740 for installing a blower fan between the big data servers. The server rack 700 forms a power supply box 750 for installing a power supply on one side. An opening can be formed so that an RJ45 socket is installed on the rear surface of the server rack 700. [

5 is an actual implementation example of a small platform server for large data distribution processing of the present invention. 5, in the small platform server 10, three nodes are arranged on the front side of the server rack, and seven nodes are arranged on the rear side of the server rack. A middleware embedded control board is placed on the right front of the server rack and a power supply is placed on the right rear of the server rack. A blower fan is placed between the three nodes on the front of the server rack and the seven nodes on the rear of the server rack. By constructing such a small platform server, it is possible to construct a small, high-efficiency, high-efficiency distributed data processing server and efficiently use it in big data analysis. In addition, it is possible to apply various domains by configuring a master node and a plurality of slave nodes in one big data server.

6 is an actual implementation example of the middleware embedded control board of the present invention.

6, the middleware embedded control board 200 includes ten relays, a terminal block for power supply, four fan power supply ports, and five sensor ports for controlling the master node and nine slave nodes can do.

7 is a flowchart illustrating a method of operating a small platform server for large data distribution processing of the present invention.

Referring to FIG. 7, in operation method of small platform server for big data distribution processing, the master node first sets a domain for building a big data cluster (S710). It uses a fully qualified domain name (FQDN), which is a method of locating another host through a domain name instead of local IP in the local environment, and consists of a host name and a domain name.

In the embodiment, the master node may be set to 192.168.0.180 master.hdp.hkinet.com, the first slave node is 192.168.0.181 slave01.hdp.hkinet.com, the second slave node is 192.168.0.182 slave02 .hdp.hkinet.com, the third slave node is 192.168.0.183 slave03.hdp.hkinet.com, the fourth slave node is 192.168.0.184 slave04.hdp.hkinet.com, the fifth slave node is 192.168.0.185 slave05.hdp.hkinet.com, the sixth slave node is 192.168.0.186 slave06.hdp.hkinet.com, the seventh slave node is 192.168.0.187 slave07.hdp.hkinet.com, and the eighth slave node is 192.168. 0.188 slave08.hdp.hkinet.com, and the 9th slave node can be set as 192.168.0.189 slave09.hdp.hkinet.com. In other words, to distinguish the role of the master node from the slave node, the host name is designated as master or slave, followed by 'hdp.hkinet.com' as a unique domain name.

Subsequently, the master node sets up a password-less ssh (secure shell). To build a big data cluster, you can set slave access to slave nodes on the master node without a password. This is the setting required for building a big data cluster through Apache Ambari. The Ambary server is first installed on the master node, and the Ambary server accesses each slave node with ssh, It can be operated as a structure to install. The command to configure password-less ssh on each host is as follows.

# ssh-keygen -t dsa -P '' -f ~ / .ssh / id_dsa

# cat ~ / .ssh / id_dsa.pub >> ~ / .ssh / authorized_keys

# cat ~ / .ssh / id_dsa.pub | ssh root@master.hdp.hkinet.com 'cat >> ~ / .ssh / authorized_keys'

# cat ~ / .ssh / id_dsa.pub | ssh root@slave01.hdp.hkinet.com 'cat >> ~ / .ssh / authorized_keys'

.

. # cat ~ / .ssh / id_dsa.pub | ssh root@slave09.hdp.hkinet.com 'cat >> ~ / .ssh / authorized_keys'

Thereafter, the server is installed in the master node (S730). First, you need to install the Ambary server on the master node. In the embodiment, the Ambary server is an HDP cluster manager developed by Hortonworks, and can provide the functions necessary for building and operating the HDP. The command to install the Ambary server on the master node is as follows.

# wget -nv http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4.0.1/ambari.list -O /etc/apt/sources.list.d/ambari. list

# apt-key adv --recv-keys --keyserver keyserver.ubuntu.com B9733A7A07513CAD

# apt-get update

# apt-get install ambari-server

# ambari-server setup

After completing the installation of Amberly server, proceed with initial setup of Amberly server through setup command. The initial configuration options include setting the version of the Java framework and configuring the database. When the setting is completed, execute the command with the start command. When Ambara runs normally, you can access the following web page through http://192.168.0.180:8080 connection. Click the 'Launch Install Wizard' button on the screen below to start cluster configuration. We add the total of 10 nodes that we set up using FQDN method.

Thereafter, the role of the master node is allocated (S740).

Thereafter, the master node assigns each role of the slave nodes (S750).

Thereafter, the components allocated to the slave nodes are installed (S760). After confirming the connection of all the nodes and completing the role assignment process, the components corresponding to the given roles are installed to the respective nodes. After the installation is completed, it is checked whether each framework is properly installed, and then the cluster construction is completed.

FIG. 8 is a diagram illustrating a method of assigning a role of a master node according to the present invention, and FIG. 9 is an exemplary diagram illustrating a method of assigning roles of a slave node of the present invention.

Referring to FIGS. 8 and 9, it is possible to assign a role of a master node at a master node, select a component to be installed, assign a role of each slave node, and select a component to be installed. For example, the first slave node can select DataNode, NFSGGateway, NodeManager, Flume, Livy server, and the second slave node can select a DataNode, NodeManageer, Flumme, Spark Thrift Server, have.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

10; Small platform server
100; Big Data Server
200; Middleware Embedded Control Board
300; Power supply
400; Blowing fan
500; Temperature and Humidity Sensor
600; Communication port
700; Server rack

Claims (7)

To a small platform server including a plurality of nodes to which different domains are allocated for big data distribution processing,
A middleware embedded control board for supplying power to the small platform server and measuring the temperature and humidity of the small platform server;
A master node controlled by the middleware embedded control board and building a cluster for big data distribution processing; And
A plurality of slave nodes to which respective roles are allocated by the master node and different components are installed,
The middleware embedded control board includes:
Sensor ports including Uart 1 channel, I2C 1 channel, 1wire 3 channels;
A relay for controlling the master node and the plurality of slave nodes;
A microprocessor for transmitting and receiving sensor information received from the sensor port to the web via Ethernet;
A PMIC for controlling power through UART communication with the microprocessor; And
A compact platform server for big data distribution processing that includes DC-DC converters that power the power supply and blower fans. The method according to claim 1,
Wherein the master node establishes a domain for building a big data cluster with the slave nodes. The method according to claim 1,
Wherein the middleware embedded control board controls power supply to the master node and the slave nodes by using relays individually. The method according to claim 1,
Further comprising a server rack for mounting the master node, the slave nodes, and the middleware embedded control board,
A small platform server in which three nodes are arranged in a front part of the server rack, seven nodes are arranged in a rear part of the server rack, and a blowing fan is disposed between three nodes on the front side of the server rack and seven nodes on the rear side of the server rack . 3. The method of claim 2,
Wherein the master node and the slave nodes have different domain names. delete delete

Images