CN108984717A - A kind of digital content based on cloud computing technology is unified to be introduced and management method and system - Google Patents
A kind of digital content based on cloud computing technology is unified to be introduced and management method and system Download PDFInfo
- Publication number
- CN108984717A CN108984717A CN201810748906.XA CN201810748906A CN108984717A CN 108984717 A CN108984717 A CN 108984717A CN 201810748906 A CN201810748906 A CN 201810748906A CN 108984717 A CN108984717 A CN 108984717A
- Authority
- CN
- China
- Prior art keywords
- data
- management
- cloud module
- data processing
- digital content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The digital content based on cloud computing technology that the present invention provides a kind of is unified to be introduced and management method and system, when user submits data processing request, the necessary parameter of data processing and master data is submitted to give management cloud module by open interface, it manages cloud module and generates configuration file using existing information control, include the parameter of finished Parallel Algorithms for Data Mining and the essential information that storage cloud data access needs to use in configuration file；The data access function component control of storage cloud module extracts data from database and data file；After the completion of data preparation, data access function component informs management cloud module by scheduling information, starts the Parallel Algorithms for Data Mining chosen and carries out data processing.Compared with prior art, technical solution of the present invention is capable of providing bigger stronger better data processing performance, even if the response of business can be ensured.
Description
Technical field
The unified introducing of the digital content based on cloud computing technology that the present invention relates to a kind of and management method and system, are related to sea
Measure data content process field.
Background technique
Digital content is unified to introduce the content that operating service system is propagated with management system digital mobile, for the height of content
It is the basic of the business development market expansion that effect, which makes full use of, is based on such a theory, main to study the content based on cloud computing
The content of storage, use, distribution, retrieval etc. forms digital content introducing and administrative mechanism comprehensively, efficiently, safe.
Digital content works content is introduced into digital mobile propagation operating service system to promote, while analysis being provided
The correlation functions such as management, information publication, online examination.Unified introduce with management of digital content is to realize that digital content propagation pushes away
Wide and industrialization first step, therefore, content introducing play a significant role with building for management system.
Summary of the invention
The digital content based on cloud computing technology that the technical problem to be solved in the present invention is to provide a kind of is unified to be introduced and manages
Method and system are managed, have the characteristics that be capable of providing bigger stronger better data processing performance.
The technical solution adopted by the invention is as follows:
A kind of digital content based on cloud computing technology is unified to be introduced and management method, specific method include,
Request of data submits the necessary parameter of data processing and base by open interface when user submits data processing request
Notebook data gives management cloud module, and it includes processing in configuration file that management cloud module, which generates configuration file using existing information control,
The essential information that the parameter and storage cloud data access for the Parallel Algorithms for Data Mining crossed need to use；
The data access function component control of data preparation, storage cloud module extracts data from database and data file；
Data processing, after the completion of data preparation, data access function component passes through scheduling information and informs management cloud module, starting choosing
Good Parallel Algorithms for Data Mining carries out data processing.
The method also includes, database is separated, including,
Resources of production library, for storing the specific data content of data access；
Library is managed, for storing the specific rules content of data access；
Log library, for storing data access record log file.
The method also includes storage cloud module uses node type management method, and each back end manages oneself node
On all physical datas；Wherein, the file system name space of host node unified management storage cloud module, stores all excavations
Data to be processed；The data to be processed include the name space of physical data, access control, data block location information
With the backup information of data block.
A kind of digital content unification introducing and management system based on cloud computing technology, is easy to implement above-mentioned digital content system
One introducing and management method, it is characterised in that: including storage cloud module and management cloud module；Wherein,
Cloud module is managed, receives the data processing request that user submits, and control according to data processing request information and generate configuration
File；The data processing request information includes the necessary parameter of data processing and master data；Include in the configuration file
The essential information that the parameter and storage cloud data access of finished Parallel Algorithms for Data Mining need to use；
Cloud module, including data access function component are stored, data are extracted in control from database and data file, and pass through tune
It spends information and informs management cloud module, start the Parallel Algorithms for Data Mining chosen and carry out data processing.
The storage cloud module further includes database, and the database includes,
Resources of production library, for storing the specific data content of data access；
Library is managed, for storing the specific rules content of data access；
Log library, for storing data access record log file.
The storage cloud module management structure is node type management structure, including,
Host node, the file system name space of unified management storage cloud module store all excavations data to be processed；It is described
Data to be processed include the backup letter of the name space of physical data, access control, the location information of data block and data block
Breath；
Back end manages all physical datas on oneself node.
Compared with prior art, technical solution of the present invention is capable of providing bigger stronger better data processing performance, can
Even if the response of guarantee business.
Detailed description of the invention
Fig. 1 is that the storage cloud module of a wherein embodiment of the invention realizes structural schematic diagram.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that described herein, specific examples are only used to explain the present invention, not
For limiting the present invention.
Any feature disclosed in this specification (including abstract and attached drawing) unless specifically stated can be equivalent by other
Or the alternative features with similar purpose are replaced.That is, unless specifically stated, each feature is a series of equivalent or class
Like an example in feature.
A kind of digital content based on cloud computing technology is unified to be introduced and management method, specific method include,
Request of data submits the necessary parameter of data processing and base by open interface when user submits data processing request
Notebook data gives management cloud module, and it includes processing in configuration file that management cloud module, which generates configuration file using existing information control,
The essential information that the parameter and storage cloud data access for the Parallel Algorithms for Data Mining crossed need to use；
The data access function component control of data preparation, storage cloud module extracts data from database and data file；
Data processing, after the completion of data preparation, data access function component passes through scheduling information and informs management cloud module, starting choosing
Good Parallel Algorithms for Data Mining carries out data processing.
Different cloud data handling utilities can develop different data application services and connect according to actual data traffic type
Mouthful, different data application services is provided.Collaborative work in cloud data system between multiple data resources makes multiple data
Resource can externally provide same service, and provide bigger stronger better data processing performance.
The method also includes, database is separated, including,
Resources of production library, for storing the specific data content of data access；
Library is managed, for storing the specific rules content of data access；
Log library, for storing data access record log file.
Cloud computing data service system needs while meeting the needs of a large number of users, in parallel for magnanimity isomerous multi-source data
Service is provided.Therefore, the data storage of cloud data system, processing technique must have high-throughput and high transmission rates, highly reliable
The characteristics of property.
Unified introduce of data content is responsible for millions of copyrights, up to ten million tens sets of external systems of product with management system
With the rapid growth of business, the resource consumption request of same database loads production system and information management system is complete
The same resource database is used at each ability component of different business function, causes resource contention, for point of database
Library technology proposes requirement.Divide library technology using database, resources of production library, management library, log library are separated, ensure industry
Even if the response of business.
The method also includes storage cloud module uses node type management method, and each back end manages oneself node
On all physical datas；Wherein, the file system name space of host node unified management storage cloud module, stores all excavations
Data to be processed；The data to be processed include the name space of physical data, access control, data block location information
With the backup information of data block.
As shown in Figure 1, host node NameNode is deposited virtual cloud is managed collectively entirely in the realization structure of storage cloud module
Store up data module file system name space, store all excavation pending datas, contain physical data name space,
Access control, the location information of data block, backup information of data block etc., and a large amount of back end DataNode will be managed certainly
All physical datas on own node.
In addition, using the state prison of the logical Virtual management of storage equipment, multi-link redundancy management and hardware device
Control and Breakdown Maintenance.Meanwhile data encryption technology guarantees that the data of storage will not be accessed by the user of unauthorized, by various
Data backup, disaster tolerance technology and measure can guarantee that the data of storage will not lose, and guarantee the safety and stabilization of accumulation layer itself.
The present invention also uses mapping mechanism to realize data centralization and virtualization, passes through redundancy and synchronized update technology
Guarantee the Fault recovery and integrality of data.Prolong to reduce mass data transmission to the consumption and reduction data manipulation of bandwidth
When, using the Data Transference Technology of optimization, to effectively improve the execution efficiency of data mining scheduling algorithm.
A kind of digital content unification introducing and management system based on cloud computing technology, is easy to implement above-mentioned digital content system
One introducing and management method, it is characterised in that: including storage cloud module and management cloud module；Wherein,
Cloud module is managed, receives the data processing request that user submits, and control according to data processing request information and generate configuration
File；The data processing request information includes the necessary parameter of data processing and master data；Include in the configuration file
The essential information that the parameter and storage cloud data access of finished Parallel Algorithms for Data Mining need to use；
Cloud module, including data access function component are stored, data are extracted in control from database and data file, and pass through tune
It spends information and informs management cloud module, start the Parallel Algorithms for Data Mining chosen and carry out data processing.
The storage cloud module further includes database, and the database includes,
Resources of production library, for storing the specific data content of data access；
Library is managed, for storing the specific rules content of data access；
Log library, for storing data access record log file.
The storage cloud module management structure is node type management structure, including,
Host node, the file system name space of unified management storage cloud module store all excavations data to be processed；It is described
Data to be processed include the backup letter of the name space of physical data, access control, the location information of data block and data block
Breath；
Back end manages all physical datas on oneself node.
Claims (6)
1. a kind of digital content unification based on cloud computing technology introduces and management method, specific method include,
Request of data submits the necessary parameter of data processing and base by open interface when user submits data processing request
Notebook data gives management cloud module, and it includes processing in configuration file that management cloud module, which generates configuration file using existing information control,
The essential information that the parameter and storage cloud data access for the Parallel Algorithms for Data Mining crossed need to use；
The data access function component control of data preparation, storage cloud module extracts data from database and data file；
Data processing, after the completion of data preparation, data access function component passes through scheduling information and informs management cloud module, starting choosing
Good Parallel Algorithms for Data Mining carries out data processing.
2. digital content according to claim 1 is unified to be introduced and management method, the method also includes, by database into
Row separation, including,
Resources of production library, for storing the specific data content of data access；
Library is managed, for storing the specific rules content of data access；
Log library, for storing data access record log file.
3. the unified introducing of digital content according to claim 1 or 2 and management method, the method also includes storing cloud
Module uses node type management method, and each back end manages all physical datas on oneself node；Wherein, host node is united
The file system name space of one management storage cloud module, stores all excavations data to be processed；The data to be processed
Name space, access control, the location information of data block and the backup information of data block including physical data.
4. a kind of digital content unification based on cloud computing technology introduces and management system, it is easy to implement one of claims 1 to 3
The unified introducing of the digital content and management method, it is characterised in that: including storage cloud module and management cloud module；Wherein,
Cloud module is managed, receives the data processing request that user submits, and control according to data processing request information and generate configuration
File；The data processing request information includes the necessary parameter of data processing and master data；Include in the configuration file
The essential information that the parameter and storage cloud data access of finished Parallel Algorithms for Data Mining need to use；
Cloud module, including data access function component are stored, data are extracted in control from database and data file, and pass through tune
It spends information and informs management cloud module, start the Parallel Algorithms for Data Mining chosen and carry out data processing.
5. the unified introducing of digital content according to claim 4 and management system, which is characterized in that the storage cloud module
It further include database, the database includes,
Resources of production library, for storing the specific data content of data access；
Library is managed, for storing the specific rules content of data access；
Log library, for storing data access record log file.
6. the unified introducing of digital content according to claim 4 or 5 and management system, which is characterized in that the storage cloud
Module management structure is node type management structure, including,
Host node, the file system name space of unified management storage cloud module store all excavations data to be processed；It is described
Data to be processed include the backup letter of the name space of physical data, access control, the location information of data block and data block
Breath；
Back end manages all physical datas on oneself node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810748906.XA CN108984717A (en) | 2018-07-10 | 2018-07-10 | A kind of digital content based on cloud computing technology is unified to be introduced and management method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810748906.XA CN108984717A (en) | 2018-07-10 | 2018-07-10 | A kind of digital content based on cloud computing technology is unified to be introduced and management method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108984717A true CN108984717A (en) | 2018-12-11 |
Family
ID=64536671
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810748906.XA Pending CN108984717A (en) | 2018-07-10 | 2018-07-10 | A kind of digital content based on cloud computing technology is unified to be introduced and management method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108984717A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102882979A (en) * | 2012-10-18 | 2013-01-16 | 深圳讯腾软件技术有限公司 | System and method for collecting, gathering and distributing data on basis of cloud computing system |
| CN104954453A (en) * | 2015-06-02 | 2015-09-30 | 浙江工业大学 | Data mining REST service platform based on cloud computing |
| CN105005570A (en) * | 2014-04-23 | 2015-10-28 | 国家电网公司 | Method and apparatus for mining massive intelligent power consumption data based on cloud computing |
-
2018
- 2018-07-10 CN CN201810748906.XA patent/CN108984717A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102882979A (en) * | 2012-10-18 | 2013-01-16 | 深圳讯腾软件技术有限公司 | System and method for collecting, gathering and distributing data on basis of cloud computing system |
| CN105005570A (en) * | 2014-04-23 | 2015-10-28 | 国家电网公司 | Method and apparatus for mining massive intelligent power consumption data based on cloud computing |
| CN104954453A (en) * | 2015-06-02 | 2015-09-30 | 浙江工业大学 | Data mining REST service platform based on cloud computing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111723160B (en) | Multi-source heterogeneous incremental data synchronization method and system | |
| KR101956236B1 (en) | Data replication technique in database management system | |
| CN105940396B (en) | The level chunking of object in distributed memory system | |
| US9507807B1 (en) | Meta file system for big data | |
| Abourezq et al. | Database-as-a-service for big data: An overview | |
| CN111639082B (en) | Object storage management method and system of billion-level node scale knowledge graph based on Ceph | |
| CN104050276A (en) | Cache processing method and system of distributed database | |
| CN103399894A (en) | Distributed transaction processing method on basis of shared storage pool | |
| CN108268614B (en) | Distributed management method for forest resource spatial data | |
| US10241963B2 (en) | Hash-based synchronization of geospatial vector features | |
| US11226986B2 (en) | Data table partitioning management method and apparatus | |
| CN105554132A (en) | Hadoop online capacity expansion method | |
| US9767107B1 (en) | Parallel file system with metadata distributed across partitioned key-value store | |
| CN113760847A (en) | Log data processing method, device, equipment and storage medium | |
| Shangguan et al. | Big spatial data processing with Apache Spark | |
| US11210212B2 (en) | Conflict resolution and garbage collection in distributed databases | |
| Lawson et al. | Empress: extensible metadata provider for extreme-scale scientific simulations | |
| KR20190022600A (en) | Data replication technique in database management system | |
| US9275059B1 (en) | Genome big data indexing | |
| Steinbauer et al. | Dynamograph: a distributed system for large-scale, temporal graph processing, its implementation and first observations | |
| Azim et al. | Offsite 2-Way Data Replication toward Improving Data Refresh Performance | |
| CN108984717A (en) | A kind of digital content based on cloud computing technology is unified to be introduced and management method and system | |
| CN112115206A (en) | Method and device for processing object storage metadata | |
| Liu et al. | On storing and retrieving geospatial big-data in cloud | |
| Plagemann et al. | A data sharing facility for mobile ad-hoc emergency and rescue applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181211 |
|
| RJ01 | Rejection of invention patent application after publication |