CN106790403B

CN106790403B - Method for realizing mobile cloud computing intermediate platform and method for realizing distribution

Info

Publication number: CN106790403B
Application number: CN201611074053.3A
Authority: CN
Inventors: 刘勇; 陆小慧; 张家明
Original assignee: ZTE Corp
Current assignee: ZTE Corp
Priority date: 2016-11-29
Filing date: 2016-11-29
Publication date: 2022-01-25
Anticipated expiration: 2036-11-29
Also published as: CN106790403A; WO2018099406A1

Abstract

The invention discloses a method for realizing a mobile cloud computing intermediate platform and a method for realizing distribution, wherein on one hand, the mobile cloud computing intermediate platform is established between a Hadoop and an Android system and is used for guaranteeing the adaptation and the operation between the Hadoop and the Android system, so that the Hadoop realizes the normal operation on the Android system, namely, the Android system supports the operation of the Hadoop. On the other hand, through the mobile cloud computing intermediate platform, the intelligent terminals can conveniently establish, join and move out of a wireless network/wired network established distributed system, and particularly, the intelligent terminals which are cheaper to use and convenient to replace can be used for establishing a distributed cluster.

Description

Method for realizing mobile cloud computing intermediate platform and method for realizing distribution

Technical Field

The invention relates to a big data technology, in particular to a method for realizing a mobile cloud computing intermediate platform and a method for realizing distribution.

Background

Hadoop is a distributed system infrastructure developed by the Apache Foundation. A user can develop a distributed program without knowing the distributed underlying details. The power of the cluster is fully utilized to carry out high-speed operation and storage. A Hadoop Distributed File System (HDFS) has the characteristic of high fault tolerance and is designed to be deployed on a (low-cost) PC server which is lower in cost than a small computer; moreover, the HDFS can provide high throughput (high throughput) to access data of the application program, and is suitable for the application program with a huge data set (large data set). HDFS relaxes the requirements of (relax) POSIX and can access (streaming access) data in a file system in the form of streams.

The most core design of the Hadoop framework is as follows: HDFS and MapReduce. HDFS provides storage for massive data, while MapReduce provides computation for massive data. At present, Hadoop open sources are free, hardware is cheap and development is convenient, so that the Hadoop open sources occupy a dominant position on a large data traffic market, and the Hadoop market share is increased year by year at a high speed.

In fact, beyond the cost of software, the economic cost of hardware is more significant. How to reduce hardware cost is a subject that every server manufacturer considers and seeks a solution.

Currently, Central Processing Units (CPUs) of Intel corporation occupy monopoly of the market, resulting in high prices of CPUs for PC servers. And the CPU processor used by the intelligent terminal is low in price. The price difference between the two is about 30 times, and the power consumption difference is about 1000 times. The performance difference between the two is averaged to a single core, and the strong processor is only about 5 times of the processor of the mobile terminal (such as a mobile phone). A cluster (such as a mobile phone cluster) formed by 100 intelligent terminals has the advantages that the cost can be controlled to be 1 to 2 ten thousand yuan, the physical volume is equivalent to that of a common PC server, the computing performance is 5 to 10 times that of the PC server, and the power consumption can be controlled to be 1/10 to 1/20 of the server.

At present, an Android system based on an intelligent terminal cannot support Hadoop operation. In addition, the existing intelligent terminal needs to establish a cluster, and besides being established through wireless signals (a mobile network and a WIFI network), hardware of the intelligent terminal can be modified, for example, a wired communication module is additionally arranged. The mode of the component cluster mainly depends on the realization of hardware, the realization is complicated, and the cost is difficult to control.

Disclosure of Invention

In order to solve the technical problems, the invention provides a method for realizing a mobile cloud computing intermediate platform and a method for realizing a distributed type, which can enable an Android system to support the operation of Hadoop.

In order to achieve the purpose of the invention, the invention provides a method for realizing a mobile cloud computing intermediate platform, which comprises the following steps:

carrying out network configuration; setting the operation configuration of a Hadoop platform according to an intelligent terminal added into the distributed system;

managing the resources for calculation and storage, and performing resource integration processing based on the resources for calculation and storage;

performing cloud computing management according to the information configured by the network and the integrated computing and storage resources;

the mobile cloud computing intermediate platform is established between a Hadoop platform and an Android system.

Optionally, the method further comprises: and providing a Java library which is lacked by the Android system for the Hadoop platform.

Optionally, the network configuration includes:

when the mobile cloud computing intermediate platform is started for the first time, a fixed address is allocated to an intelligent terminal provided with the mobile cloud computing intermediate platform;

and acquiring the IP address according to the connection mode of the distributed system to which the intelligent terminal belongs.

Optionally, the obtaining the IP address according to the connection mode of the distributed system to which the intelligent terminal belongs includes:

when the connection mode is wired connection, searching IP addresses of all intelligent terminals of the distributed system, and processing the maximum IP address to serve as the IP address;

when the connection mode is wireless connection, the IP address is automatically allocated by a wireless network dynamic host configuration protocol DHCP;

and when the connection mode is a rack customization mode, the IP address is acquired according to the difference of the slot positions.

Optionally, the managing the resources for computing and storing, and performing the resource integration processing based on the resources for computing and storing includes:

when a distributed system to which an intelligent terminal installed with the mobile cloud computing intermediate platform belongs receives writing and computing tasks, determining the resource use condition in the current distributed system;

calling a task allocation method of the Hadoop platform according to the confirmed resource use condition: when the storage size exceeds the upper limit of the set limit and does not accept the writing operation, when the computing capacity exceeds the upper limit of the set limit and does not accept the computing task; when the storage size is lower than the set limit lower limit, writing is preferentially carried out until the upper limit of the set limit is reached; when the calculation capacity is lower than the set limit lower limit, preferentially receiving the calculation task until the upper limit of the set limit is reached; and randomly distributing the tasks when the intelligent terminals with the storage size and the computing capacity between the upper limit and the lower limit in the distributed system acquire the tasks.

Optionally, the method further comprises: and providing hardware health monitoring and repairing operation for the intelligent terminal in the distributed system for installing the intelligent terminal of the mobile cloud computing intermediate platform.

Optionally, after the hardware has a problem, the method further includes: and performing reconnection/restarting operation, and prompting to replace the hardware after the reconnection/restarting still fails for a preset number of times.

Optionally, the method further comprises: and when the system log is larger than the set range value, performing first-in first-out FIFO storage.

The invention also provides a device for realizing the mobile cloud computing intermediate platform, which at least comprises a memory and a processor, wherein the memory stores the following executable instructions:

carrying out network configuration; setting the operation configuration of a Hadoop platform according to an intelligent terminal added into the distributed system; managing the resources for calculation and storage, and performing resource integration processing based on the resources for calculation and storage; performing cloud computing management according to the information configured by the network and the integrated computing and storage resources;

The invention also provides a method for realizing distribution, which comprises the following steps:

acquiring and installing a mobile cloud computing intermediate platform and a Hadoop platform to an intelligent terminal;

the mobile cloud computing intermediate platform performs data resource migration and computing resource integration in a distributed system to which the intelligent terminal belongs in a Hadoop platform;

and the mobile cloud computing intermediate platform receives the storage and computing resource conditions of the intelligent terminal in the distributed system fed back by the Hadoop platform, and performs cloud computing management.

Optionally, when a new intelligent terminal accesses the distributed system, the method further includes:

and the mobile cloud computing intermediate platform performs data resource migration and computing resource integration again according to the storage and computing resource conditions of the newly accessed intelligent terminal.

the new intelligent terminal is connected to a network where the distributed system is located, and the mobile cloud computing intermediate platform and the Hadoop platform are obtained and installed;

and the new intelligent terminal uses the installed mobile cloud computing intermediate platform to configure the Hadoop system network according to the connection mode of the network where the distributed system is located.

Optionally, the method further includes acquiring an IP address according to a connection mode of a distributed system to which the intelligent terminal belongs, and specifically includes:

Optionally, after the obtaining and installing the mobile cloud computing intermediate platform and the Hadoop platform, before the performing data resource migration and computing resource integration in the distributed system, the method further includes:

the mobile cloud computing intermediate platform provides a Java library which is lacked by the Android system of the intelligent terminal, and performs hardware selection and loading on the Android system of the intelligent terminal;

the mobile cloud computing intermediate platform allocates an IP address to the intelligent terminal according to a distributed system connection mode returned by the intelligent terminal; and writing the configuration file into the Hadoop platform.

Optionally, the performing data resource migration and computing resource integration includes:

the intelligent terminal with the storage size exceeding the set limit upper limit does not accept writing operation, and the functional terminal with the calculation capacity exceeding the set limit upper limit does not accept calculation tasks;

when the storage size of the intelligent terminal is lower than the set limit lower limit, writing is preferentially carried out until the upper limit of the set limit is reached; when the computing capacity of the intelligent terminal is lower than the set limit lower limit, preferentially receiving a computing task until the upper limit of the set limit is reached;

and randomly distributing the storage size and the computing capacity when the intelligent terminal between the upper limit and the lower limit acquires the task.

Optionally, the method further comprises: the mobile cloud computing intermediate platform stores the logs in an FIFO storage mode.

Optionally, the method further comprises: and the mobile cloud computing intermediate platform monitors the health condition of hardware according to the heartbeat condition of the intelligent terminal accessed by the distributed system fed back by the Hadoop platform.

Optionally, the mobile cloud computing intermediate platform provides an external interface for cloud computing management.

Optionally, when an intelligent terminal in the distributed system exits from the distributed system, the method further includes:

the mobile cloud computing intermediate platform receives a distributed system quitting application from the Hadoop platform, and data resource migration and computing resource integration in the distributed system are carried out again in the Hadoop platform;

and after the intelligent terminal successfully exits, the mobile cloud computing intermediate platform records logs.

The mobile cloud computing intermediate platform is established between the Hadoop and the Android system and used for guaranteeing the adaptation and operation between the Hadoop and the Android system, so that the Hadoop can normally operate on the Android system, namely the Android system supports the operation of the Hadoop. On the other hand, through the mobile cloud computing intermediate platform, the intelligent terminals can conveniently establish, join and move out of a wireless network/wired network established distributed system, and particularly, the intelligent terminals which are cheaper to use and convenient to replace can be used for establishing a distributed cluster.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a schematic diagram of a composition structure of a mobile cloud computing intermediate platform according to the present invention;

FIG. 2 is a schematic flow chart of the working principle of the mobile cloud computing intermediate platform according to the present invention;

FIG. 3 is a schematic flow diagram illustrating a process of implementing data storage of an HDFS cluster based on an Android system according to the present invention;

FIG. 4 is a schematic flow chart of calculation implemented by the HDFS cluster based on the Android system;

FIG. 5 is a schematic diagram of a structure of a computing module in the modified intelligent terminal according to the present invention;

FIG. 6 is a schematic diagram of a structure of a memory module in the modified intelligent terminal according to the present invention;

fig. 7 is a schematic diagram of networking of an intelligent terminal in a WIFI connection manner according to the present invention;

fig. 8 is a schematic flowchart of an embodiment of management of an intelligent terminal in a cluster according to the connection manner shown in fig. 7;

FIG. 9 is a schematic diagram of a network configuration of an intelligent terminal according to the present invention using a mobile network connection;

fig. 10 is a schematic flowchart of an embodiment of management of an intelligent terminal in a cluster according to the connection manner shown in fig. 9;

FIG. 11 is a schematic diagram of a networking of the intelligent terminal of the present invention using a wired connection;

fig. 12 is a schematic flowchart of an embodiment of managing the intelligent terminals in the cluster according to the connection manner shown in fig. 11;

FIG. 13 is a top view of a rack in the custom rack configuration of the present invention;

FIG. 14 is a side view of a frame in the custom frame construction of the present invention;

FIG. 15 is a schematic diagram of the implementation of hardware replacement in a custom rack architecture of the present invention;

fig. 16 is a flowchart illustrating an embodiment of a mobile cloud computing intermediate platform application according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.

Fig. 1 is a schematic diagram of a composition structure of the mobile cloud computing intermediate platform of the present invention, and as shown in fig. 1, the mobile cloud computing intermediate platform of the present invention is an intermediate layer established between a Hadoop platform and an Android system, and is used for ensuring the adaptation and operation between the Hadoop platform and the Android system. At least comprises the following steps: the system comprises a hardware driving module, a Java library supporting module, a configuration module, a computing storage resource management module and a cloud computing task scheduling module; wherein the content of the first and second substances,

and the hardware driving module is used for loading the hardware driving for the operating system. The driver serves mainly for the network controller, such as: when the communication is an RJ45 interface, the network controller is a USB-to-RJ 45 signal chip; when the communication is optical fiber communication, the network controller is a photoelectric conversion chip.

And the Java library support module is used for providing Java libraries lacking in the Android system for the Hadoop platform. The Android system supports a part of Java functions, and the Java library support module is used for complementing Java libraries which are lacked in the Android system.

The configuration module is used for carrying out network configuration; and automatically setting the operation configuration of the Hadoop platform according to the computing capacity and the storage size of the intelligent terminal such as the intelligent terminal added into the distributed system.

The network configuration specifically includes:

when a mobile cloud computing intermediate platform where a configuration module is located is started for the first time, a fixed address is allocated to an intelligent terminal provided with the mobile cloud computing intermediate platform (all machines are IP addresses when being started for the first time);

after the network is connected, determining the connection mode of the distributed system to which the intelligent terminal belongs, such as wired, wireless or customized frame types, and acquiring an IP address according to the connection mode: when the connection is wired, searching the IP addresses of all machines of the current network, and then processing the maximum IP address: at the fourth segment +1 of the largest IP address, when adding to 253, then add 1 to the third segment of the IP address, the fourth segment counting from 1; when the connection is wireless connection, the IP address is automatically allocated by a Dynamic Host Configuration Protocol (DHCP) of a wireless network, and the allocation of the IP address is not required to be considered, and only the allocated IP address needs to be read and written into a Hadoop Configuration file; when the rack is customized, the intelligent terminal acquires IP address configuration according to different slot positions, so that the configuration of the newly replaced electrified equipment is the same as that of the equipment with problems, namely the IP address of the same slot position is unchangeable.

And the computing and storage resource management module is used for integrating the computing and storage resource management module into a mobile cloud computing intermediate platform on which the computing and storage resource management module is positioned on the basis of the conventional Hadoop computing and storage control, managing computing and storage resources and integrating the computing and storage resources.

The specific implementation comprises the following steps: when the distributed system receives an external writing and computing task, firstly confirming the resource use condition of a machine in the current distributed system, and then calling a task allocation method of a Hadoop platform: storing write-in operation which is not to be accepted and exceeds the set limit upper limit, and calculating calculation tasks which are not to be accepted and exceed the set limit upper limit; when the storage is lower than the lower limit of the set limit, the data is written in preferentially until the upper limit of the set limit is reached; when the calculation is lower than the set limit lower limit, the calculation task is preferentially accepted until the upper limit of the set limit is reached; other machines between the upper and lower limits are randomly assigned when they acquire tasks.

And the cloud computing task scheduling module is used for providing the external interface, and performing cloud computing management including but not limited to starting, stopping, suspending, timing and the like according to the information configured by the network and the integrated computing and storage resources.

Further, the mobile cloud computing intermediate platform of the present invention further includes: a health monitoring module for monitoring the health of the patient,

the method is used for providing hardware health monitoring for the intelligent terminal and providing repair operation. It should be noted that the health monitoring module only monitors the hardware to be dead or alive, and performs reconnection/restart operations when the hardware has a problem, and prompts to replace the hardware after multiple reconnection/restart failures. The monitoring function may use a heartbeat detection mechanism to monitor hardware activity and death, use temperature monitoring to alarm the health of the hardware, and the like.

Further, the mobile cloud computing intermediate platform of the present invention further includes: a log processing module for processing the log of the user,

when the system log is larger than the set range value, the log is stored in a First-in First-out (FIFO) mode, namely, the long-term log is deleted firstly, and then the latest log is stored.

Fig. 1 only lists basic Hadoop components of the mobile cloud computing intermediate platform of the present invention, and other components may be installed by themselves, and whether or not to include other components, and how many other components are included, and are not used to limit the protection scope of the present invention, and are not described herein again.

Correspondingly, the invention also provides a method for realizing the mobile cloud computing intermediate platform, which comprises the following steps: establishing a mobile cloud computing intermediate platform between a Hadoop platform and an Android system, and further comprising:

and carrying out cloud computing management according to the information configured by the network and the integrated computing and storage resources.

The method of the invention also comprises the following steps: and providing a Java library lacking an Android system for the Hadoop platform.

Optionally, the network configuration includes:

when the established mobile cloud computing intermediate platform is started for the first time, a fixed address is allocated to an intelligent terminal for installing the mobile cloud computing intermediate platform; and acquiring the IP address according to the connection mode of the distributed system to which the intelligent terminal belongs. Wherein the content of the first and second substances,

the acquiring the IP address according to the connection mode of the distributed system to which the intelligent terminal belongs comprises the following steps:

when the connection mode is wireless connection, the IP address is automatically allocated by a wireless network DHCP;

and when the connection mode is a customized rack mode, acquiring the IP address according to the difference of the slot positions.

Optionally, the managing the resources of the computing and the storing, and the performing the resource integration processing based on the computing and the storing resources includes:

when a distributed system to which an intelligent terminal provided with a mobile cloud computing intermediate platform belongs receives writing and computing tasks, determining the resource use condition in the current distributed system;

Optionally, the method for implementing the mobile cloud computing intermediate platform further includes:

and providing hardware health monitoring and repairing operation for the intelligent terminal in the distributed system for installing the intelligent terminal of the mobile cloud computing intermediate platform.

When the hardware has problems, the method also comprises the following steps: and performing reconnection/restarting operation, and prompting to replace the hardware after the reconnection/restarting still fails for a preset number of times.

Optionally, the method for implementing the mobile cloud computing intermediate platform further includes: and when the system log is larger than the set range value, performing FIFO storage.

The invention also provides an apparatus for implementing a mobile cloud computing intermediary platform, comprising at least a memory and a processor, wherein,

the memory has stored therein the following executable instructions: establishing a mobile cloud computing intermediate platform between a Hadoop platform and an Android system; carrying out network configuration; setting the operation configuration of a Hadoop platform according to an intelligent terminal added into the distributed system; managing the resources for calculation and storage, and performing resource integration processing based on the resources for calculation and storage; and carrying out cloud computing management according to the information configured by the network and the integrated computing and storage resources.

The method for realizing the distribution by utilizing the mobile cloud computing intermediate platform established by the invention comprises the following steps:

The working principle of the mobile cloud computing intermediate platform of the invention is described in detail below by taking the working flow conditions of the modules of an intelligent terminal after the mobile cloud computing intermediate platform shown in fig. 1 of the invention is installed as an example. Fig. 2 is a schematic flow chart of the working principle of the mobile cloud computing intermediate platform of the present invention, as shown in fig. 2, including:

firstly, an intelligent terminal acquires and installs the mobile cloud computing intermediate platform and the Hadoop platform; the Java library support module of the mobile cloud computing intermediate platform provides a Java library which is lacked by an Android system of the intelligent terminal; the hardware driving module of the mobile cloud computing intermediate platform selects and loads the Android system of the intelligent terminal; the configuration module of the mobile cloud computing intermediate platform allocates an IP address to the intelligent terminal: the intelligent terminal returns the connection type of the distributed system to a configuration module of the mobile cloud computing intermediate platform; the configuration module of the mobile cloud computing intermediate platform performs corresponding IP address configuration according to different network connection modes of the intelligent terminal: when the connection is wired, searching the IP addresses of all machines of the current network, and then processing the maximum IP address: at the fourth segment +1 of the largest IP address, when adding to 253, then add 1 to the third segment of the IP address, the fourth segment counting from 1; when the connection is wireless, the IP address is automatically allocated by a wireless network DHCP; when the rack mode is customized, the IP address configuration is obtained according to the difference of the slot positions. The intelligent terminal reads the distributed IP address, writes the IP address into a configuration file of the Hadoop platform and takes effect;

then, a computing storage resource management module of the mobile cloud computing intermediate platform performs data resource migration and computing resource integration in a distributed system in a Hadoop platform, and the Hadoop platform feeds back the storage and computing resource conditions of the intelligent terminal to the computing storage resource management module of the mobile cloud computing intermediate platform;

then, the computing storage resource management module of the mobile cloud computing intermediate platform performs data resource migration and computing resource integration according to the storage and computing resource conditions of the newly accessed intelligent terminal: storing write-in operation which is not to be accepted and exceeds the set limit upper limit, and calculating calculation tasks which are not to be accepted and exceed the set limit upper limit; when the storage is lower than the lower limit of the set limit, the data is written in preferentially until the upper limit of the set limit is reached; when the calculation is lower than the set limit lower limit, the calculation task is preferentially accepted until the upper limit of the set limit is reached; other machines between the upper limit and the lower limit are randomly allocated when acquiring tasks;

finally, for the storage of the log, the log processing module of the mobile cloud computing intermediate platform performs FIFO storage; the health monitoring module of the mobile cloud computing intermediate platform monitors the health of hardware, the Hadoop platform feeds back the heartbeat condition of the accessed intelligent terminal to detect the death and the activity of the hardware, and feeds back the node temperature of the accessed intelligent terminal to monitor the running health condition; the cloud computing task scheduling module of the mobile cloud computing intermediate platform is provided for an external interface to carry out cloud computing management.

Subsequently, if the intelligent terminal applies for quitting the distributed system, the Hadoop platform applies for quitting the distributed system to the computing storage resource management module of the mobile cloud computing intermediate platform, the computing storage resource management module of the mobile cloud computing intermediate platform performs data resource migration and computing resource integration in the distributed system again in the Hadoop platform, and after quitting is successful, logs are recorded in the log processing module of the mobile cloud computing intermediate platform.

In summary, according to the Android system-based Hadoop platform, namely the mobile cloud computing intermediate platform shown in fig. 1, on one hand, the Hadoop platform realizes normal operation on the Android system, that is, the Android system supports the operation of the Hadoop platform. Furthermore, health monitoring of software and hardware during Hadoop platform operation is provided, so that the software is guaranteed to be repaired or the hardware is guaranteed to be replaced when the software is damaged. On the other hand, through the Hadoop platform based on the Android system, namely the mobile cloud computing intermediate platform shown in the figure 1, the intelligent terminal realizes a distributed system which is very convenient to build, join and move out of a wireless network/wired network. Therefore, the distributed system is established by using the cheaper and conveniently-replaced intelligent terminal.

The embodiment of the present invention is described by taking an intelligent terminal as a mobile phone terminal by way of example, but the intelligent terminal of the present invention includes but is not limited to: the system comprises a mobile phone terminal, a PAD and a set top box which operate an Android system, various wired and wireless handheld terminals which operate the Android system and the like.

For convenience of description, the distributed system is taken as an example for description in the embodiment of the present invention, but the present invention is not limited to the protection scope of the present invention.

The Hadoop platform calls various application interface (API) functions to the underlying Android system through the mobile cloud computing intermediate platform to perform tasks such as data reading and writing, computing and the like. The packaging of the Android API interface includes, but is not limited to:

for the reading and writing of the memory data of the intelligent terminal, the method can include:

getFileDir (): the method is used for obtaining the position information of the Hadoop platform in the data stored in the intelligent terminal memory, such as: data/data/Hadoop/files;

getCacheDir (): the method is used for obtaining the position information of the Hadoop platform in the data cached in the intelligent terminal memory, and comprises the following steps: data/data/Hadoop/cache;

openfileinput (string name): the method comprises the steps of directly obtaining input stream information of a/data/data/Hadoop/files/name file;

openFileOutput (String name, int mode): the method comprises the steps of directly obtaining an output stream of a/data/data/Hadoop/files/name file, wherein a mode is a right when the file is written;

content. mode _ prior: the method is a private mode (or called as a default mode) and can be accessed only by an application and an intelligent terminal in the same group; the written content covers the original file content;

content, mode _ APPEND: the mode is an additional mode (or called a private mode) and can only be accessed by the application and the intelligent terminals in the same group; if the file exists, the content is added, and if the file does not exist, the file is newly created and the content is written;

content, mode _ word _ read: the readable authority of all intelligent terminals in the group;

context, mode _ word _ write: writing permissions for all intelligent terminals in the group;

for writing and reading SDcard data, it may include:

getExternalStoragDirectory (): the method comprises the steps of obtaining SDcard position information of an intelligent terminal where a Hadoop platform is located, wherein the SDcard position information comprises/storage/SDcard;

getExternalStorageState (): the method is used for obtaining the current state of the SDcard of the intelligent terminal where the Hadoop platform is located, and the current state is usually MEDIA _ MOUNTED;

FileInputStream () for reading files;

BufferReader () used for reading files;

httpConnection () for saving the read stream as String data;

FileOutputStream () for writing to a file;

buffer writer () for writing a file;

in addition, the Hadoop calculation uses four arithmetic operations of addition, subtraction, multiplication and division of a Java library.

In the invention, the HDFS cluster based on the Android system has two types of nodes and runs in a mode of manager-worker: a Master (Master) node (NameNode) as manager and a plurality of Slave (Slave) nodes (DataNode) as workers. Each NameNode and DataNode corresponds to an intelligent terminal.

The NameNode is mainly responsible for managing an HDFS file system, and specifically comprises namespace management and block (block) management; the DataNode is mainly used for storing data files. Here, HDFS divides a file into blocks, which may be stored on one DataNode or on multiple datanodes. The DataNodes are responsible for reading and writing actual bottom files, if a Client (Client) program initiates a command for reading files on the HDFS, the files are divided into a plurality of blocks, and then the NameNode informs the Client of which block data are stored in which DataNodes, so that the Client directly interacts with the DataNodes.

The storage and calculation process of the HDFS cluster based on the Android system according to the present invention is described in detail below.

Fig. 3 is a schematic diagram of a process of implementing data storage by an HDFS cluster based on an Android system, taking file uploading to a distributed cluster for storage as an example, where a NameNode is responsible for managing metadata of all files stored on the HDFS, and will confirm a request of a client, and record names of the files and data nodes storing the files, that is, a set of datanodes. And storing the recorded information in a file allocation table in the memory. In this embodiment, it is assumed that the client sends a request to the NameNode, which indicates that the "shock.

Step 1: the client sends a message to the NameNode, which indicates that a 'shock.txt' file is to be written;

step 2: NameNode returns message to client end to inform client end to write the 'shock.txt' file to DataNode A, DataNode B and DataNode D, and directly contact with DataNode B;

and step 3: the client sends a message to the DataNode B, indicates the DataNode B to store a copy of the' swim.

And 4, step 4: the DataNode B sends a message to the DataNode A to indicate the DataNode A to store a copy of the shock file, and sends a copy to the DataNode D;

and 5: the DataNode A sends a message to the DataNode D to indicate the DataNode D to store a' swim.

Step 6: the DataNode D sends a confirmation message to the DataNode A;

and 7: the DataNode A sends a confirmation message to the DataNode B;

and 8: and the DataNode B sends a confirmation message to the client, which indicates that the writing of the' shock.

Thus, one copy of the "swim. txt" file is stored on three data nodes (i.e. intelligent terminals) of the data node A, the data node B and the data node D of the distributed cluster.

Fig. 4 is a schematic flowchart of computing implemented by the HDFS cluster based on the Android system, and as shown in fig. 4, in the HDFS based on the Android system, each MapReduce task is initialized to a Job, and each Job may be divided into two phases: a Map (Map) phase and a Reduce (Reduce) phase. These two phases are represented by two functions, namely a Map function and a Reduce function, respectively. Wherein the Map function receives an input in the form < key, value > and then also produces an intermediate output in the form < key, value >; the Reduce function receives an input in the form of < key (list of values) > and then processes this set of values, each Reduce producing 0 or 1 output, the output of Reduce also being in the form of < key, value >.

Generally, after the mobile cloud computing intermediate platform and the Hadoop platform of the invention are installed, a common intelligent terminal uses stable wireless connection, and after the Hadoop platform is configured by the mobile cloud computing intermediate platform of the invention, each component above the Hadoop can be used. For example, in an office, the intelligent terminals of everyone can be centralized in a cluster, so that the sharing of computing resources and storage resources is realized. Therefore, the calculated amount which cannot be achieved by one intelligent terminal can be formed into a cluster through more than ten intelligent terminals, and the calculated amount which cannot be achieved originally is achieved on the mobile cloud computing intermediate platform paper. The wireless connection further comprises: WIFI network, mobile network (such as 2G, 3G, 4G, etc.) two cluster group building methods.

In order to ensure the reliability of the cluster and enhance the communication speed, a wired connection mode can be used. The cluster hardware source using the wired connection mode can be a customized intelligent terminal, and the existing intelligent terminal can be used for carrying out wired communication transformation. The transformation of the existing intelligent terminal mainly comprises two parts: on one hand, a USB interface is reformed, and USB type communication is converted into twisted pair or optical fiber communication; on the other hand, the memory module includes at least a ROM, an SD card, and an interface. Memory module corruption is replaceable. In particular, the amount of the solvent to be used,

fig. 5 is a schematic diagram of a structure of a computing module in the modified intelligent terminal of the present invention, as shown in fig. 5, the computing module at least includes: a Central Processing Unit (CPU) chipset, a Random-Access Memory (RAM), and a network controller. The external interface comprises 3 interfaces of a power supply, a network port and a storage interface.

Among them, the USB interface may include but is not limited to: USB2.0 or USB3.0, respectively corresponding to twisted pair and optical fiber. Thus, different communication schemes are used for the two different types of USB. The network controller adopts a common USB (universal serial bus) adapter card scheme, and if the optical fiber is added with a photoelectric conversion chip, the network controller can be used. The network controller in fig. 5 is a signal conversion chip, such as: USB2.0 to RJ45 signal chip such as AX 88772B; USB 3.0-to-Gigabit Ethernet control chip such as RTL 8153.

In fig. 5, a Secure Digital (SD) card interface and a Read Only Memory (ROM) interface of an Android system customized system Image are merged into a storage interface, which is Only a change in hardware appearance, and actually, the two interfaces are also independent.

Fig. 6 is a schematic diagram of a structure of a memory module in an intelligent terminal after modification, as shown in fig. 6, the memory module at least includes two built-in components: ROM and SD card, externally display as a storage interface. The module may be replaced if damaged.

Hadoop is a large data platform with wide application, and all the currently used embodiments based on Hadoop can use the technical scheme provided by the invention to be transferred to an intelligent terminal provided with an Android system from a PC server. The embodiment of managing the intelligent terminals in the cluster is described in detail below by combining the mobile cloud computing intermediate platform according to different network connection modes of the intelligent terminals.

First embodiment

Fig. 7 is a schematic networking diagram of an intelligent terminal adopting a WIFI connection manner, fig. 8 is a schematic flow chart of an embodiment of management of the intelligent terminal in a cluster based on the connection manner shown in fig. 7, and with reference to fig. 7 and fig. 8, this embodiment takes a new node as an example of the intelligent terminal, and the step of adding the intelligent terminal to the cluster includes:

step 800: and successfully connecting the intelligent terminal to be added into the existing cluster WIFI network with the WIFI network.

Step 801: and the intelligent terminal to be added into the existing clustered WIFI network downloads and acquires and installs the mobile cloud computing intermediate platform and the Hadoop platform through the Android system of the intelligent terminal.

Step 802: and configuring the Hadoop platform system network by using the mobile cloud computing intermediate platform. The specific implementation process is shown in fig. 2, and is not described here again.

Assume that a Connection (Connection) parameter of a Connection mode is 1, which is denoted as a WIFI Connection mode, a Connection parameter is 2, which is denoted as a mobile network such as a 2G, 3G, 4G Connection mode, and a Connection parameter is 3, which is denoted as a wired Connection mode.

In this embodiment, assuming that the Connection parameter is 1, the mobile cloud computing intermediate platform allocates an IP address to a node (e.g., the intelligent terminal in step 800) newly added in this embodiment according to a DHCP mechanism in the WIFI network; and writing the allocated IP address into the cluster configuration file.

Step 803: and the main node in the existing cluster starts the Hadoop cluster component and starts the Hadoop cluster component successfully.

Step 804: the master node in the existing cluster performs data resource migration and computing resource integration (balance) in the cluster.

In this embodiment, the mobile cloud computing intermediate platform in the master node performs data resource migration and computing resource integration by itself according to the storage and computing resource conditions of the new node (e.g., the intelligent terminal). For example, suppose that the upper limit of the storage resource of each node is 80% of the total storage resource and the lower limit of the storage resource is 5% of the total storage resource; supposing that the upper limit of CPU computing resources of each node is that the CPU utilization rate reaches 80 percent and the lower limit of the CPU computing resources is that the CPU utilization rate reaches 5 percent; the setting of the upper limit and the lower limit can be adjusted according to actual requirements.

The mobile cloud computing intermediate platform performs the following operations according to the set upper limit value and the set lower limit value:

when the storage of the newly added node exceeds the upper limit of the storage resource, the write operation is not accepted;

when the calculation of the newly added node exceeds the upper limit of the CPU calculation resource, the calculation task is not accepted;

when the storage of the newly added node is lower than the lower limit of the storage resource, the node is written in preferentially until the upper limit of the storage resource is reached;

when the calculation of the newly added node is lower than the lower limit of the CPU calculation resource, the calculation task is preferentially received until the upper limit of the CPU calculation resource is reached;

other situations between the upper limit and the lower limit can be randomly allocated when the task is acquired.

Step 805 to step 806: and calling a mobile cloud computing intermediate platform interface by a main node in the existing cluster, and submitting a Hadoop task to operate.

Step 807: and each data node in the cluster completes task calculation and feeds back a task result to the master node.

In this embodiment, assuming that a data node applies for exiting a cluster as an intelligent terminal added to an existing cluster in step 800, the method includes the following steps:

step 808: and the intelligent terminal applies for quitting the cluster from the main node in the cluster.

Step 809: and the main node performs data resource migration and computing resource integration. The specific implementation is as described in step 804, and is not described herein again.

Step 810: and after the resource integration is completed, the intelligent terminal is allowed to quit the cluster.

Second embodiment

Fig. 9 is a schematic networking diagram of an intelligent terminal adopting a mobile network connection manner, fig. 10 is a schematic flow chart of an embodiment of management of the intelligent terminal in a cluster based on the connection manner shown in fig. 9, and with reference to fig. 9 and fig. 10, this embodiment takes a new node as an example of an intelligent terminal, and a step of adding the intelligent terminal to the cluster includes:

step 1000: and connecting the intelligent terminals of the mobile networks such as 2G, 3G, 4G and the like to be added into the existing cluster with the mobile network successfully.

Step 1001: and the intelligent terminal to be added into the mobile network of the existing cluster acquires and installs the mobile cloud computing intermediate platform and the Hadoop platform through the self Android system downloading.

Step 1002: and configuring the Hadoop system network by using the mobile cloud computing intermediate platform. The specific implementation process is shown in fig. 2, and is not described here again.

In this embodiment, assuming that the Connection parameter is 2, the mobile cloud computing intermediate platform allocates an IP address to a node (e.g., the intelligent terminal in step 1000) newly added in this embodiment according to a DHCP mechanism in the mobile network; and writing the allocated IP address into the cluster configuration file.

Step 1003: and the main node in the existing cluster starts the Hadoop cluster component and starts the Hadoop cluster component successfully.

Step 1004: the master node in the existing cluster performs data resource migration and computing resource integration (balance) in the cluster.

Further, the air conditioner is provided with a fan,

in consideration of the transmission rate and the traffic cost, in this embodiment, a data compression (compression) parameter is further set, and when the compression parameter is 1, it indicates that compression is required, and when the compression parameter is 0, it indicates that compression is not required.

When the Connection parameter is 2, that is, in this embodiment, a mobile network is adopted, the compression parameter is automatically set to 1, which indicates that compression is required during data transmission to obtain a high transmission rate, so as to reduce the traffic charge;

when the Connection parameter is 1 or 3, the compression parameter is automatically set to 0, and data transmission does not need to be compressed, so that the operation pressure of the CPU is reduced.

Step 1005 to step 1006: and calling a mobile cloud computing intermediate platform interface by a main node in the existing cluster, and submitting a Hadoop task to operate.

Step 1007: and each data node in the cluster completes task calculation and feeds back a task result to the master node.

In this embodiment, assuming that a data node applies for exiting a cluster as an intelligent terminal added to an existing cluster in step 1000, the method includes the following steps:

step 1008: and the intelligent terminal applies for quitting the cluster from the main node in the cluster.

Step 1009: and the main node performs data resource migration and computing resource integration. The specific implementation is as described in step 1004, and is not described herein again.

Step 1010: and after the resource integration is completed, the intelligent terminal is allowed to quit the cluster.

Third embodiment

Fig. 11 is a schematic networking diagram of an intelligent terminal adopting a wired connection manner, fig. 12 is a schematic flow chart of an embodiment of management of the intelligent terminal in a cluster based on the connection manner shown in fig. 11, and with reference to fig. 11 and fig. 12, this embodiment takes a new node as an example of the intelligent terminal, and the step of adding the intelligent terminal to the cluster includes:

step 1200: the intelligent terminals to be added to the existing cluster are successfully connected with the switches in the cluster by using twisted pairs or optical fibers. In this embodiment, the intelligent terminal to be added to the existing cluster uses the USB2.0 interface, and therefore, the switch of the cluster is connected by using the twisted pair.

Step 1201: and the intelligent terminal to be added into the mobile network of the existing cluster acquires and installs the mobile cloud computing intermediate platform and the Hadoop platform through the self Android system downloading.

Step 1202: and configuring the Hadoop system network by using the mobile cloud computing intermediate platform. The specific implementation process is shown in fig. 2, and is not described here again.

In this embodiment, assuming that the Connection parameter is 3, the mobile cloud computing intermediate platform searches for IP addresses of all nodes of the current cluster network, and then processes the maximum IP address: at the fourth segment +1 of the largest IP address, when adding to 253, then add 1 to the third segment of the IP address, the fourth segment counting from 1; and writing the allocated IP address into the cluster configuration file.

Step 1203: and the main node in the existing cluster starts the Hadoop cluster component and starts the Hadoop cluster component successfully.

Step 1204: the master node in the existing cluster performs data resource migration and computing resource integration (balance) in the cluster.

Step 1205 to step 1206: and calling a mobile cloud computing intermediate platform interface by a main node in the existing cluster, and submitting a Hadoop task to operate.

Step 1207: and each data node in the cluster completes task calculation and feeds back a task result to the master node.

In this embodiment, assuming that a data node applies for exiting a cluster as an intelligent terminal that joins the existing cluster in step 1200, the method includes the following steps:

step 1208: and the intelligent terminal applies for quitting the cluster from the main node in the cluster.

Step 1209: and the main node performs data resource migration and computing resource integration. The specific implementation is as described in step 1004, and is not described herein again.

Step 1210: and after the resource integration is completed, the intelligent terminal is allowed to quit the cluster.

Fourth embodiment

This embodiment is a case where the cluster model is determined to be a customized rack structure after the network is connected.

Firstly, after the rack is installed and electrified and uniformly converts alternating current into direct current to provide power for all equipment, version mirroring operation is carried out, namely the whole system shown in the figure 1 is packaged as a mirror image and is directly stored in a ROM.

And storing the hardware of the installed system into the shaft inlet of the rack in which the backup part is positioned. Initial installation requires manual hardware provisioning because of the large number of devices that need to be installed. The subsequent backup device is preferably an automatic storage device for 5-10 blocks.

Hardware is installed at a designated position through the same process as the backup replacement, as shown in fig. 13, fig. 13 is a top view of a rack in the customized rack structure of the present invention, and the computing memory modules are installed on the circular rings of the rack, as shown in fig. 13. In this embodiment, the ring rotates at a low speed.

On one hand, the low-speed rotation of the ring brings the air-cooling and heat-dissipating effects of the traditional equipment, and on the other hand, when hardware needs to be replaced, the low-speed rotation of the ring realizes the butt joint of the channels of the calculation storage backup module, so that the replaced module can be conveyed in place by the rack; simultaneously, centrifugal force can demolish the hardware of damage automatically.

Then, the hardware-connected rack switching equipment automatically acquires network parameters and joins the distributed cluster.

The device which is firstly integrated into the cluster is the main node, and the distributed computing task can be executed on a cloud computing task scheduling platform which is externally connected with the main node through a network.

And then, adding backup equipment, and detecting the number of the backup equipment of the rack at a subsequent timing so as to add the backup equipment in time. While requiring recovery of the damaged hardware devices.

Fig. 14 is a side view of a rack in a customized rack structure of the invention, and as shown in fig. 14, the circular slot level of the rack may be a multi-level design, and the lines with arrows indicate the operation paths of the backup hardware.

When hardware is replaced, as shown in fig. 15, the hardware slot is unlocked and locked, and the damaged hardware is separated from the rack due to centrifugal force; the backup hardware runs into the backup guide groove through the conveying device; the ring gradually reduces the speed to stop rotating, so that the backup guide groove is butted with the slot position of the hardware needing to be replaced, and the backup hardware is loaded into the hardware slot position through the operation of the conveying device; after the spare hardware is replaced, the ring gradually returns to the original operation rotation speed.

Fifth embodiment

In this embodiment, a specific working implementation of the mobile cloud computing intermediate platform and the mobile cloud computing system of the present invention is described in a typical application scenario. In this embodiment, it is assumed that 5 individuals respectively hold 1 smartphone of the Android system in one room to test and use the large game, because the game is large in calculation amount and generates a large amount of data, the game is very chunky on one mobile phone, and meanwhile, artificial intelligence analysis and mining are required to be performed based on the large data, so that scene discovery, friend playing recommendation and the like are provided for the players. Fig. 16 is a schematic flow chart of an embodiment of application of the mobile cloud computing intermediate platform of the present invention, and in this embodiment, for convenience of description, only a service flow between 2 devices, that is, a Master node and a Slave node is taken as an example, and the service flow between the Master node and other Slave nodes is the same as the service flow shown in fig. 16, and details are not repeated here. As shown in fig. 16, includes:

step 1600: and respectively installing a mobile cloud computing intermediate platform, a Hadoop and a game application on the Master node and the Slave node.

Step 1601: the method comprises the following steps of using a mobile cloud computing intermediate platform to carry out relevant configuration and effect on an Android system and a Hadoop platform, and roughly comprises the following steps:

the mobile cloud computing intermediate platform of the main node provides a Java library lacking an Android system for the Hadoop platform; hardware selection and loading are carried out on the Android system by a mobile cloud computing intermediate platform of the main node; the mobile cloud computing intermediate platform of the main node allocates an IP address, and the corresponding IP address is configured according to different feedback network connection modes; the Android system of the intelligent terminal reads the distributed IP address, writes the IP address into a Hadoop configuration file and takes effect; and the mobile cloud computing intermediate platform of the main node performs data resource migration and computing resource integration in the cluster in the Hadoop platform and informs other Slave nodes in the cluster to perform resource adjustment. The specific implementation is shown in fig. 2, and is not described herein again.

Step 1602: each node detects the resource occupation situation of the game application as a Master node and a Slave node in the graph 16, and feeds the resource occupation situation back to the respective Hadoop platform; and after the Hadoop platform of each Slave node receives the resource occupation condition of the game application, the resource occupation condition is fed back to the Master node.

Step 1603: and the Master node automatically performs data resource migration and computing resource integration according to the storage and computing resource conditions of each node.

Step 1604: the game application starts to be used, and after the Master node receives the use behaviors and the tasks, the Master node distributes calculation and storage tasks to each Slave node; and each Slave node feeds back a calculation result and a storage result to the Master node, the Master node collects the calculation result and the storage result and feeds back a game application calculation result and a storage result to the game application.

Step 1605: in the process of carrying out a computing task and a storage task of a game application, a mobile cloud computing intermediate platform of a main node stores a log in an FIFO (first in first out) mode; and/or the mobile cloud computing intermediate platform of the main node monitors the hardware health of each node.

Step 1606: when the game application is finished, the Master node informs each Slave node to release computing resources and storage resources; and each Slave node feeds back a calculation resource and storage resource release result to the Master node.

The above description is only a preferred example of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for realizing a mobile cloud computing intermediate platform is characterized by comprising the following steps:

carrying out network configuration on the intelligent terminal added into the distributed system; setting the operation configuration of a Hadoop platform installed in the intelligent terminal according to the intelligent terminal added into the distributed system;

the mobile cloud computing intermediate platform is established between a Hadoop platform and an Android system;

the network configuration of the intelligent terminal added to the distributed system includes:

and when the connection mode of the distributed system is wired connection, searching the IP addresses of all intelligent terminals added into the distributed system, and processing the maximum IP address to be used as the IP address of the intelligent terminal newly added into the distributed system.

2. The method of claim 1, further comprising: and providing a Java library which is lacked by the Android system for the Hadoop platform.

3. The method of claim 1, wherein the network configuration comprises:

4. The method according to claim 3, wherein the acquiring the IP address according to the connection mode of the distributed system to which the intelligent terminal belongs comprises:

5. The method of claim 1, wherein managing computing and storage resources, and wherein performing a resource consolidation process based on the computing and storage resources comprises:

6. The method according to any one of claims 1 to 5, further comprising: and providing hardware health monitoring and repairing operation for the intelligent terminal in the distributed system for installing the intelligent terminal of the mobile cloud computing intermediate platform.

7. The method of claim 6, wherein when the hardware has a problem, further comprising: and performing reconnection/restarting operation, and prompting to replace the hardware after the reconnection/restarting still fails for a preset number of times.

8. The method according to any one of claims 1 to 5, further comprising: and when the system log is larger than the set range value, performing first-in first-out FIFO storage.

9. An apparatus for implementing a mobile cloud computing intermediary platform, comprising at least a memory and a processor, wherein the memory has stored therein executable instructions to:

carrying out network configuration on the intelligent terminal added into the distributed system; setting the operation configuration of a Hadoop platform installed in the intelligent terminal according to the intelligent terminal added into the distributed system; managing the resources for calculation and storage, and performing resource integration processing based on the resources for calculation and storage; performing cloud computing management according to the information configured by the network and the integrated computing and storage resources;

10. A method for implementing distribution, comprising:

acquiring and installing a mobile cloud computing intermediate platform and a Hadoop platform to an intelligent terminal; the mobile cloud computing intermediate platform performs network configuration on the intelligent terminals added into the distributed system, and sets the operation configuration of a Hadoop platform installed in the intelligent terminals according to the intelligent terminals added into the distributed system;

the mobile cloud computing intermediate platform receives storage and computing resource conditions of the intelligent terminal in the distributed system fed back by the Hadoop platform, and carries out cloud computing management;

the method for network configuration of the intelligent terminal added into the distributed system by the mobile cloud computing intermediate platform comprises the following steps:

11. The method of claim 10, wherein when a new intelligent terminal accesses the distributed system, further comprising:

12. The method of claim 11, wherein when a new intelligent terminal accesses the distributed system, further comprising:

13. The method according to claim 12, further comprising obtaining the IP address according to a connection mode of a distributed system to which the intelligent terminal belongs, specifically comprising:

14. The method of claim 10, wherein after obtaining and installing the intermediate mobile cloud computing platform and the Hadoop platform, and before performing the migration of data resources and the integration of computing resources in the distributed system, further comprising:

15. The method according to claim 10 or 11, wherein the performing data resource migration and computing resource integration comprises:

16. The method of claim 10, 11 or 12, further comprising: the mobile cloud computing intermediate platform stores the logs in an FIFO storage mode.

17. The method of claim 10, 11 or 12, further comprising: and the mobile cloud computing intermediate platform monitors the health condition of hardware according to the heartbeat condition of the intelligent terminal accessed by the distributed system fed back by the Hadoop platform.

18. The method of claim 10, 11 or 12, wherein the mobile cloud computing intermediary platform provides an external interface for cloud computing management.

19. The method of claim 10, 11 or 12, wherein when an intelligent terminal in the distributed system exits the distributed system, the method further comprises: