CN112328681A - JS-based cloud storage platform, method, storage medium and equipment - Google Patents
JS-based cloud storage platform, method, storage medium and equipment Download PDFInfo
- Publication number
- CN112328681A CN112328681A CN202011210557.XA CN202011210557A CN112328681A CN 112328681 A CN112328681 A CN 112328681A CN 202011210557 A CN202011210557 A CN 202011210557A CN 112328681 A CN112328681 A CN 112328681A
- Authority
- CN
- China
- Prior art keywords
- service
- cloud
- algorithm
- layer
- cloud storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000011161 development Methods 0.000 claims abstract description 22
- 238000004806 packaging method and process Methods 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 20
- 238000012544 monitoring process Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 7
- 238000012356 Product development Methods 0.000 abstract description 5
- 238000000926 separation method Methods 0.000 abstract description 5
- 239000012141 concentrate Substances 0.000 abstract description 4
- 238000010276 construction Methods 0.000 abstract description 4
- 238000007726 management method Methods 0.000 description 13
- 238000013461 design Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000009440 infrastructure construction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4488—Object-oriented
- G06F9/449—Object-oriented method invocation or resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a JS-based cloud storage platform which comprises a cloud storage layer, a cloud engine layer, a cloud service SDK layer and an external interface layer; the cloud storage layer is used for providing a distributed storage system; the cloud engine layer is used for defining a uniform RESTful interface and providing an algorithm service function according to the parallel algorithm library; the cloud service SDK layer is used for packaging an REST service API interface for local calling, introducing an SDK development kit into a service system and instantiating a related service object when calling the SDK service; and the external interface layer is used for providing cloud service to the outside, and the cloud service is exposed to the outside through a RESTful interface. The invention realizes the foundation of a front-end and back-end separation framework and a universal back-end cloud storage interface, improves the front-end development efficiency of developers, does not need to pay attention to the construction of infrastructures such as back-end service development and the like, concentrates more on user experience and front-end realization, improves the product development and online progress, and quickly constructs personalized application. The invention also relates to a JS-based cloud storage method, a JS-based cloud storage medium and JS-based cloud storage equipment.
Description
Technical Field
The invention relates to the field of cloud computing, in particular to a JS-based cloud storage platform, a JS-based cloud storage method, a JS-based cloud storage medium and JS-based cloud storage equipment.
Background
Cloud storage is a new concept that has been extended and developed over the concept of cloud computing (cloud computing). The system is a system which integrates a large number of different types of storage devices in a network through application software to cooperatively work through functions of cluster application, a grid technology or a distributed file system and the like, and provides data storage and service access functions to the outside.
The cloud storage service can be divided into personal-level application and enterprise-level application, and specifically includes personal space service, data backup, centralized storage, remote sharing, video monitoring, centralized storage of systems such as an IPTV and the like, and large-capacity online storage of websites and the like.
The current cloud storage platform adopts a front-end and back-end separation technology which is an industry standard mode of Internet project development, and the core idea is that a front-end HTML page calls a RESTFUL API interface at the back end through AJAX and uses JSON data for interaction, but the front end of the mode seriously depends on the development of basic design interfaces such as the back end in the development process.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a JS-based cloud storage platform, a JS-based cloud storage method, a JS-based cloud storage medium and JS-based cloud storage equipment.
The technical scheme for solving the technical problems is as follows:
a JS-based cloud storage platform comprises a cloud storage layer, a cloud engine layer, a cloud service SDK layer and an external interface layer;
the cloud storage layer is used for providing a distributed storage system;
the cloud engine layer is used for defining a uniform RESTful interface, providing an algorithm service function according to a parallel algorithm library and realizing parallel association rule analysis, classification and clustering algorithms based on a Hadoop platform;
the cloud service SDK layer is used for packaging an REST service API interface for local calling, introducing an SDK development kit into a service system and instantiating a related service object when calling the SDK service;
the external interface layer is used for providing cloud service to the outside, and the cloud service is exposed to the outside through a RESTful interface.
The invention has the beneficial effects that: the cloud storage platform based on the JS comprises a cloud storage layer, a cloud engine layer, a cloud service SDK layer and an external interface layer; the cloud storage layer provides a distributed storage system; the cloud engine layer defines a unified RESTful interface, provides an algorithm service function according to a parallel algorithm library, and realizes parallel association rule analysis, classification and clustering algorithms based on a Hadoop platform, the cloud service SDK layer encapsulates an REST service API interface for local calling, an SDK development kit is introduced into a service system, and instantized related service objects are called when the SDK service is called; the external interface layer provides cloud service for the outside, and the cloud service is exposed to the outside through a RESTful interface. The invention realizes the foundation of a front-end and back-end separation framework and a universal back-end cloud storage interface, improves the front-end development efficiency of developers, does not need to pay attention to the construction of infrastructures such as back-end service development and the like, concentrates more on user experience and front-end realization, improves the product development and online progress, and quickly constructs personalized application.
On the basis of the technical scheme, the invention can be further improved as follows.
Further, the cloud engine layer comprises a parallel algorithm service, a job monitoring service and a log management service;
the parallel algorithm service is used for exposing a parallel algorithm interface outwards and acquiring the calling parameter information of the algorithm by inquiring the sub-resources under the algorithm resource;
the operation monitoring service is used for exposing an inquiry interface of the operation running state of the operation outwards, and the client inquires the execution condition of the specified operation according to the operation name;
and the log management service is used for providing management on the running logs of the cluster jobs.
The beneficial effect of adopting the further scheme is that: exposing a parallel algorithm interface outwards through a parallel algorithm service, and acquiring calling parameter information of the algorithm through sub-resources under query algorithm resources; the operation monitoring service exposes an inquiry interface of the operation running state outwards, and the client inquires the execution condition of the specified operation according to the operation name; the log management service provides management of the running logs of the cluster jobs. The packaging of the rear-end storage interface is realized, developers do not need to pay attention to infrastructure construction such as rear-end service development, and the product development efficiency is improved.
Further, the parallel algorithm service is specifically configured to deliver a parameter to the parallel algorithm service through a Post request, where a value of the parameter is encapsulated in an object;
when an algorithm calling request is received, a new thread is created, the new thread is used for executing corresponding algorithm calling processing, an asynchronous object is created at the same time, and the asynchronous object is sent to the new thread;
and after the new thread runs, returning a processing result through the asynchronous object, wherein the processing result is in a JSON format.
Further, the job monitoring service is specifically configured to query the running state of the job and obtain a query result, where the query result is in a JSON format.
Another technical solution of the present invention for solving the above technical problems is as follows:
a JS-based cloud storage method is based on the technical scheme of the JS-based cloud storage platform, and comprises the following steps:
the cloud storage layer provides a distributed storage system;
the cloud engine layer defines a unified RESTful interface, provides an algorithm service function according to a parallel algorithm library and realizes parallel association rule analysis, classification and clustering algorithms based on a Hadoop platform;
the cloud service SDK layer encapsulates an REST service API interface for local calling, introduces an SDK development kit into the service system, and instantiates related service objects when calling the SDK service;
the external interface layer provides cloud service for the outside, and the cloud service is exposed to the outside through a RESTful interface.
The invention has the beneficial effects that: the cloud storage method based on JS comprises a cloud storage layer, a cloud engine layer, a cloud service SDK layer and an external interface layer; the cloud storage layer provides a distributed storage system; the cloud engine layer defines a unified RESTful interface, provides an algorithm service function according to a parallel algorithm library, and realizes parallel association rule analysis, classification and clustering algorithms based on a Hadoop platform, the cloud service SDK layer encapsulates an REST service API interface for local calling, an SDK development kit is introduced into a service system, and instantized related service objects are called when the SDK service is called; the external interface layer provides cloud service for the outside, and the cloud service is exposed to the outside through a RESTful interface. The invention realizes the foundation of a front-end and back-end separation framework and a universal back-end cloud storage interface, improves the front-end development efficiency of developers, does not need to pay attention to the construction of infrastructures such as back-end service development and the like, concentrates more on user experience and front-end realization, improves the product development and online progress, and quickly constructs personalized application.
On the basis of the technical scheme, the invention can be further improved as follows.
Further, the cloud engine layer defines a uniform RESTful interface, and provides an algorithm service function according to a parallel algorithm library, and specifically includes:
the parallel algorithm service in the cloud engine layer exposes a parallel algorithm interface outwards, and obtains calling parameter information of the algorithm by inquiring sub-resources under algorithm resources;
the operation monitoring service in the cloud engine layer exposes an inquiry interface of the operation state of the operation outwards, and the client inquires the execution condition of the specified operation according to the operation name;
and the log management service in the cloud engine layer provides management for the running logs of the cluster jobs.
Further, the parallel algorithm service in the cloud engine layer exposes a parallel algorithm interface outwards, and obtains the calling parameter information of the algorithm by querying the sub-resources under the algorithm resource, and the method specifically includes the following steps:
the parallel algorithm service delivers parameters to the parallel algorithm service through Post requests, wherein values of the parameters are encapsulated in objects;
when an algorithm calling request is received, a new thread is created, the new thread is used for executing corresponding algorithm calling processing, an asynchronous object is created at the same time, and the asynchronous object is sent to the new thread;
and after the new thread runs, returning a processing result through the asynchronous object, wherein the processing result is in a JSON format.
Further, the method further comprises:
the job monitoring service queries the running state of the job and obtains a query result, wherein the query result is in a JSON format.
In addition, the present invention further provides a computer-readable storage medium, which includes instructions, and when the instructions are executed on a computer, the instructions cause the computer to execute the steps of the JS-based cloud storage method according to any one of the above technical solutions.
The invention further provides computer equipment which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor executes the program and realizes the steps of the JS-based cloud storage method in any one of the technical schemes.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments of the present invention or in the description of the prior art will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a JS-based cloud storage platform according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a JS-based cloud storage method according to another embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.
As shown in fig. 1, which is a schematic structural diagram of a JS-based cloud storage platform according to an embodiment of the present invention, the JS-based cloud storage platform includes a cloud storage layer, a cloud engine layer, a cloud service SDK layer, and an external interface layer;
the cloud storage layer is used for providing a distributed storage system;
the cloud engine layer is used for defining a uniform RESTful interface, providing an algorithm service function according to a parallel algorithm library and realizing parallel association rule analysis, classification and clustering algorithms based on a Hadoop platform;
the cloud service SDK layer is used for packaging an REST service API interface for local calling, introducing an SDK development kit into a service system and instantiating a related service object when calling the SDK service;
the external interface layer is used for providing cloud service to the outside, and the cloud service is exposed to the outside through a RESTful interface.
It should be appreciated that the cloud storage layer provides distributed storage, database storage, and parallel computing capabilities as a support platform for the cloud engine layer. Distributed computing platforms such as Hadoop or Spark can be used as operation supports. In the aspect of parallel computing, both Hadoop and Spark support a MapReduce computing framework, the MapReduce framework encapsulates some bottom-level details of distributed computing, such as task parallel, data distribution, load balance, task scheduling, task fault tolerance and the like, and a user does not need to consider the details as long as paying attention to parallel implementation of an algorithm. The Spark provides a computation engine of DAG besides a similar MapReduce framework, supports iterative computation operation, and can better adapt to the operation of an algorithm needing iterative convergence.
The cloud engine layer adopts a distributed and pluggable modular design idea, and can use an algorithm carried by a platform, and meanwhile, the service management of the algorithm is also included in the mining algorithm tools of a third party such as Mahout and MLLib access.
REST service (Representational State Transfer) is based on Network-based, with the emphasis on: architectural Styles (Architectural Styles), software architectures (software architectures), and networks (Network-based). REST is itself a design style rather than a standard. The three elements of REST generally refer to a unique resource identifier, an operation method for the resource, and a representation form of the resource. REST describes how Web standards, e.g., HTTP and URI, are used correctly. The best way to understand REST is to think about the Web and its way of working. REST is resource-centric, where resources represent resource entities throughout the network, and are identified by URIs.
The requestor of the resource obtains a concrete metaphor (Representational) of the resource through the URI. The Client obtains the concrete symbols to make the application program change its State, and the Client continuously changes its State with continuously obtaining the concrete symbols of the resource, so that the continuous and repeated (iterative) process is said to be a recurring State Transfer.
The JS-based cloud storage platform comprises a cloud storage layer, a cloud engine layer, a cloud service SDK layer and an external interface layer; the cloud storage layer provides a distributed storage system; the cloud engine layer defines a unified RESTful interface, provides an algorithm service function according to a parallel algorithm library, and realizes parallel association rule analysis, classification and clustering algorithms based on a Hadoop platform, the cloud service SDK layer encapsulates an REST service API interface for local calling, an SDK development kit is introduced into a service system, and instantized related service objects are called when the SDK service is called; the external interface layer provides cloud service for the outside, and the cloud service is exposed to the outside through a RESTful interface. The invention realizes the foundation of a front-end and back-end separation framework and a universal back-end cloud storage interface, improves the front-end development efficiency of developers, does not need to pay attention to the construction of infrastructures such as back-end service development and the like, concentrates more on user experience and front-end realization, improves the product development and online progress, and quickly constructs personalized application.
Further, the cloud engine layer comprises a parallel algorithm service, a job monitoring service and a log management service;
the parallel algorithm service is used for exposing a parallel algorithm interface outwards and acquiring the calling parameter information of the algorithm by inquiring the sub-resources under the algorithm resource;
the operation monitoring service is used for exposing an inquiry interface of the operation running state of the operation outwards, and the client inquires the execution condition of the specified operation according to the operation name;
and the log management service is used for providing management on the running logs of the cluster jobs.
Further, the parallel algorithm service is specifically configured to deliver a parameter to the parallel algorithm service through a Post request, where a value of the parameter is encapsulated in an object;
when an algorithm calling request is received, a new thread is created, the new thread is used for executing corresponding algorithm calling processing, an asynchronous object is created at the same time, and the asynchronous object is sent to the new thread;
and after the new thread runs, returning a processing result through the asynchronous object, wherein the processing result is in a JSON format.
Further, the job monitoring service is specifically configured to query the running state of the job and obtain a query result, where the query result is in a JSON format.
It should be understood that, in this embodiment, when it is found that the query is JSON data, the queried data is encapsulated into an object, and the object is modified and then saved, or when it is necessary to add an object, the JSON data is encapsulated and then saved, and deletion and query of the object are also included. As shown in fig. 2, a schematic flow diagram of a JS-based cloud storage method provided in an embodiment of the present invention is shown, where the method includes the following steps:
110. the cloud storage tier provides a distributed storage system.
120. And the cloud engine layer defines a uniform RESTful interface and provides an algorithm service function according to the parallel algorithm library.
130. And the cloud service SDK layer encapsulates the REST service API interface for local calling, introduces an SDK development kit into the service system, and instantiates related service objects when calling the SDK service.
140. The external interface layer provides cloud service for the outside, and the cloud service is exposed to the outside through a RESTful interface.
Further, the cloud engine layer defines a uniform RESTful interface and provides an algorithm service function according to a parallel algorithm library, and the method specifically includes:
the parallel algorithm service in the cloud engine layer exposes a parallel algorithm interface outwards, and obtains calling parameter information of the algorithm by inquiring sub-resources under algorithm resources;
the operation monitoring service in the cloud engine layer exposes an inquiry interface of the operation state of the operation outwards, and the client inquires the execution condition of the specified operation according to the operation name;
and the log management service in the cloud engine layer provides management for the running logs of the cluster jobs.
Further, the parallel algorithm service in the cloud engine layer exposes a parallel algorithm interface outwards, and obtains the calling parameter information of the algorithm by querying the sub-resources under the algorithm resource, and the method specifically includes the following steps:
the parallel algorithm service delivers parameters to the parallel algorithm service through Post requests, wherein values of the parameters are encapsulated in objects;
when an algorithm calling request is received, a new thread is created, the new thread is used for executing corresponding algorithm calling processing, an asynchronous object is created at the same time, and the asynchronous object is sent to the new thread;
and after the new thread runs, returning a processing result through the asynchronous object, wherein the processing result is in a JSON format.
Further, the method further comprises:
the job monitoring service queries the running state of the job and obtains a query result, wherein the query result is in a JSON format.
In addition, the present invention also provides a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to perform the steps of the JS-based cloud storage method according to any one of the above embodiments.
The invention also provides computer equipment which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the JS-based cloud storage method in any one of the above embodiments.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium.
Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media which may not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A JS-based cloud storage platform is characterized by comprising a cloud storage layer, a cloud engine layer, a cloud service SDK layer and an external interface layer;
the cloud storage layer is used for providing a distributed storage system;
the cloud engine layer is used for defining a uniform RESTful interface and providing an algorithm service function according to the parallel algorithm library;
the cloud service SDK layer is used for packaging an REST service API interface for local calling, introducing an SDK development kit into a service system and instantiating a related service object when calling the SDK service;
the external interface layer is used for providing cloud service to the outside, and the cloud service is exposed to the outside through a RESTful interface.
2. The JS-based cloud storage platform of claim 1,
the cloud engine layer comprises a parallel algorithm service, a job monitoring service and a log management service;
the parallel algorithm service is used for exposing a parallel algorithm interface outwards and acquiring the calling parameter information of the algorithm by inquiring the sub-resources under the algorithm resource;
the operation monitoring service is used for exposing an inquiry interface of the operation running state of the operation outwards, and the client inquires the execution condition of the specified operation according to the operation name;
and the log management service is used for providing management on the running logs of the cluster jobs.
3. The JS-based cloud storage platform of claim 2,
the parallel algorithm service is specifically configured to deliver a parameter to the parallel algorithm service through a Post request, where a value of the parameter is encapsulated in an object;
when an algorithm calling request is received, a new thread is created, the new thread is used for executing corresponding algorithm calling processing, an asynchronous object is created at the same time, and the asynchronous object is sent to the new thread;
and after the new thread runs, returning a processing result through the asynchronous object, wherein the processing result is in a JSON format.
4. The JS-based cloud storage platform of claim 2,
the job monitoring service is specifically used for querying the running state of the job and obtaining a query result, wherein the query result is in a JSON format.
5. A JS-based cloud storage method based on the JS according to any one of claims 1-4, wherein the JS-based cloud storage platform is characterized by comprising the following steps:
the cloud storage layer provides a distributed storage system;
the cloud engine layer defines a unified RESTful interface, provides an algorithm service function according to a parallel algorithm library and realizes parallel association rule analysis, classification and clustering algorithms based on a Hadoop platform;
the cloud service SDK layer encapsulates an REST service API interface for local calling, introduces an SDK development kit into the service system, and instantiates related service objects when calling the SDK service;
the external interface layer provides cloud service for the outside, and the cloud service is exposed to the outside through a RESTful interface.
6. The JS-based cloud storage method of claim 5, wherein the cloud engine layer defines a uniform RESTful interface and provides an algorithm service function according to a parallel algorithm library, and specifically comprises:
the parallel algorithm service in the cloud engine layer exposes a parallel algorithm interface outwards, and obtains calling parameter information of the algorithm by inquiring sub-resources under algorithm resources;
the operation monitoring service in the cloud engine layer exposes an inquiry interface of the operation state of the operation outwards, and the client inquires the execution condition of the specified operation according to the operation name;
and the log management service in the cloud engine layer provides management for the running logs of the cluster jobs.
7. The JS-based cloud storage method of claim 6, wherein the parallel algorithm service in the cloud engine layer exposes a parallel algorithm interface outwards, and obtains calling parameter information of the algorithm by querying sub-resources under algorithm resources, comprising the following steps:
the parallel algorithm service delivers parameters to the parallel algorithm service through Post requests, wherein values of the parameters are encapsulated in objects;
when an algorithm calling request is received, a new thread is created, the new thread is used for executing corresponding algorithm calling processing, an asynchronous object is created at the same time, and the asynchronous object is sent to the new thread;
and after the new thread runs, returning a processing result through the asynchronous object, wherein the processing result is in a JSON format.
8. The JS-based cloud storage method of claim 6, wherein the method further comprises:
the job monitoring service queries the running state of the job and obtains a query result, wherein the query result is in a JSON format.
9. A computer-readable storage medium comprising instructions, which when executed on a computer, cause the computer to perform the steps of the JS-based cloud storage method of any of claims 5-8.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps of the JS-based cloud storage method of any of claims 5-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011210557.XA CN112328681A (en) | 2020-11-03 | 2020-11-03 | JS-based cloud storage platform, method, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011210557.XA CN112328681A (en) | 2020-11-03 | 2020-11-03 | JS-based cloud storage platform, method, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112328681A true CN112328681A (en) | 2021-02-05 |
Family
ID=74323033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011210557.XA Pending CN112328681A (en) | 2020-11-03 | 2020-11-03 | JS-based cloud storage platform, method, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112328681A (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104954453A (en) * | 2015-06-02 | 2015-09-30 | 浙江工业大学 | Data mining REST service platform based on cloud computing |
-
2020
- 2020-11-03 CN CN202011210557.XA patent/CN112328681A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104954453A (en) * | 2015-06-02 | 2015-09-30 | 浙江工业大学 | Data mining REST service platform based on cloud computing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11836533B2 (en) | Automated reconfiguration of real time data stream processing | |
| US10447772B2 (en) | Managed function execution for processing data streams in real time | |
| US20190068690A1 (en) | Automated management of resource attributes across network-based services | |
| CN106998343B (en) | Remote procedure call processing method, device and system | |
| CN110557284B (en) | Data aggregation method and device based on client gateway | |
| JP2016514311A (en) | Database system providing single tenant and multi-tenant environments | |
| US20150032756A1 (en) | Normalized searchable cloud layer | |
| CN110781180B (en) | Data screening method and data screening device | |
| CN113282589A (en) | Data acquisition method and device | |
| CN108959294B (en) | Method and device for accessing search engine | |
| CN108696559B (en) | Stream processing method and device | |
| CN113157480A (en) | Error information processing method, device, storage medium and terminal | |
| CN115858905A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112328681A (en) | JS-based cloud storage platform, method, storage medium and equipment | |
| CN113271334B (en) | Service policy distribution method and device based on SaaS scene and electronic equipment | |
| CN111786801B (en) | Method and device for charging based on data flow | |
| CN112416865A (en) | File processing method and device based on big data | |
| CN113448652A (en) | Request processing method and device | |
| CN113760886B (en) | Method, apparatus, device and computer readable medium for providing data service | |
| CN113760836B (en) | Wide table calculation method and device | |
| CN112825107B (en) | Method and device for generating chart | |
| CN113765868B (en) | Service processing method and device | |
| CN118296032A (en) | Data query method and device | |
| CN112667627B (en) | Data processing method and device | |
| CN114827228A (en) | Link processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |