CN113868221A - Mass big data storage, analysis and management system based on cloud computing - Google Patents
Mass big data storage, analysis and management system based on cloud computing Download PDFInfo
- Publication number
- CN113868221A CN113868221A CN202110973120.XA CN202110973120A CN113868221A CN 113868221 A CN113868221 A CN 113868221A CN 202110973120 A CN202110973120 A CN 202110973120A CN 113868221 A CN113868221 A CN 113868221A
- Authority
- CN
- China
- Prior art keywords
- data
- analysis
- unit
- user
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007405 data analysis Methods 0.000 title claims abstract description 64
- 238000013523 data management Methods 0.000 title claims abstract description 40
- 238000013500 data storage Methods 0.000 title claims abstract description 36
- 238000004458 analytical method Methods 0.000 claims abstract description 118
- 238000012545 processing Methods 0.000 claims abstract description 41
- 230000005540 biological transmission Effects 0.000 claims abstract description 18
- 238000007726 management method Methods 0.000 claims abstract description 10
- 230000032683 aging Effects 0.000 claims description 26
- 238000012217 deletion Methods 0.000 claims description 18
- 230000037430 deletion Effects 0.000 claims description 18
- 238000013144 data compression Methods 0.000 claims description 6
- 238000010835 comparative analysis Methods 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 description 6
- 238000000034 method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000036962 time dependent Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Data Mining & Analysis (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a mass big data storage, analysis and management system based on cloud computing, which comprises a server, a client and a cloud end, wherein the cloud end comprises a user private cloud end and a common cloud end, and the client comprises a user login module, an autonomous data retrieval module, a timing retrieval module, a data uploading module and a data analysis module; the server comprises a user management module, a data storage processing module, a data analysis processing module, a data management module and a data transmission module, wherein the data storage processing module is used for processing data according to user requirements and correspondingly transmitting the data to a private cloud end or a common cloud end of a user through the data transmission module, the data analysis processing module is used for analyzing the data according to the data analysis requirements of the user, and the data management module is used for sorting and optimizing the data. The invention provides a management system which is convenient for time-efficient storage and analysis of user data, and is convenient for autonomous open storage and data analysis and help seeking of the user data.
Description
Technical Field
The invention mainly relates to the technical field of data processing, in particular to a mass big data storage, analysis and management system based on cloud computing.
Background
With the development of the internet, data which needs to be analyzed and processed daily by people is increasing, the analysis capability of a common client to the data is limited, and the storage space is small, so that the increasing data processing requirement of people is difficult to meet.
According to the system for storing, analyzing and managing mass big data based on the cloud environment provided by the patent document with the application number of CN201711289856.5, the system is composed of a storage system, an analysis system and a management system. The system can automatically perform simple data analysis and automatically back up related data while inputting data, thereby reducing the burden of processing data by a user and avoiding permanent deletion of important data by the user due to carelessness. The cloud platform can provide services for the user in a safe, efficient and consistent mode, and the verification system arranged during login effectively prevents an external program from attacking the login account of the user, so that confidential data of the user are threatened in sequence, and the safety of the cloud platform is improved.
The system in the above patent can simply analyze data, automatically backup related data, and provide safety for user data, but is inconvenient for time-efficient storage and analysis of user data, and is inconvenient for autonomous open storage of user data and data analysis for help.
Disclosure of Invention
The invention mainly provides a mass big data storage, analysis and management system based on cloud computing, which is used for solving the technical problems in the background technology.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a mass big data storage, analysis and management system based on cloud computing comprises a server, a client and a cloud end, wherein the client and the cloud end are connected with the server through the Internet, the cloud end comprises a user private cloud end and a common cloud end, the client comprises a user login module, an autonomous data retrieval module, a timing retrieval module, a data uploading module and a data analysis module, the user login module is used for logging in a user, the autonomous data retrieval module is used for the user to autonomously select data to be uploaded, the timing retrieval module is used for automatically retrieving the data according to retrieval time and a retrieval mode set by the user and uploading the data through the data uploading module according to uploading time set by the user, the data uploading module is used for uploading the data to the server, and the data analysis module is used for the user to upload data analysis requirements;
the server comprises a user management module, a data storage processing module, a data analysis processing module, a data management module and a data transmission module, the user management module manages and logs in a user, the data storage processing module is used for processing data according to user requirements and correspondingly transmitting the data to a private cloud end or a common cloud end of the user through the data transmission module, the data analysis processing module is used for analyzing the data according to the data analysis requirements of the user and transmitting analysis results to a client end through the data transmission module, and the data management module is used for arranging and optimizing the data uploaded by the user and the cloud data.
Preferably, the client includes a data customization module, the data customization module includes an encryption selection unit, a backup selection unit and a timeliness selection unit, the encryption selection unit is used for a user to select data to be encrypted, the backup selection unit is used for the user to select data to be backed up, and the timeliness selection unit is used for the user to select a time period for storing the data, that is, the timeliness data. In the preferred embodiment, a user can autonomously select a processing mode of data to be uploaded through the data customization module so as to encrypt and backup the data and set the effective storage time.
Preferably, the client comprises a shared benefit module, the shared benefit module comprises a sharing unit and a benefit presentation unit, the server comprises a benefit calculation module, the sharing unit is used for displaying data uploaded to a common cloud end by a user, the benefit calculation module is used for calculating benefits according to browsing amount of the data uploaded by the common cloud end, and the benefit presentation unit is used for the user to carry out benefit presentation. In the preferred embodiment, the user is advocated to store the non-important data to the common cloud end in a public manner by sharing the data revenue, so that the data sharing is facilitated, and meanwhile, the space occupation of the non-essential data to the private cloud end of the user is reduced.
Preferably, the data uploading module comprises a duplicate checking unit and a data classification uploading unit, the server comprises a duplicate checking module, the duplicate checking unit is used for inputting data to be checked by a user, the duplicate checking module is used for comparing the data to be checked with data in a common cloud end to detect whether duplicate data exist, and the data classification uploading unit is used for distinguishing the data uploaded to a private cloud end and the data in the common cloud end of the user by the user. In the preferred embodiment, the repetition rate of the data to be stored of the user and the data in the shared cloud can be conveniently known through the duplication checking unit, so that the user can conveniently sort the data according to the requirement of the user.
Preferably, the autonomous data retrieval module includes a local retrieval unit and an online retrieval unit, the local retrieval unit is used for a user to retrieve local data to be uploaded, and the online retrieval unit is used for the user to retrieve the data to be uploaded on the network. In the preferred embodiment, the local retrieval unit and the online retrieval unit are used for facilitating the user to quickly search the data to be stored.
Preferably, the data analysis module includes an analysis requirement unit, an analysis result unit and a reward analysis unit, the analysis requirement unit is used for a user to edit specific requirements of data analysis, the analysis result unit is used for displaying an analysis result of the data analysis processing module, and the reward analysis unit is used for a user to edit analysis data and analysis requirements to be rewarded. In the preferred embodiment, the analysis requirement unit is convenient for the user to edit the data analysis requirements, such as data trend, data summarization, data chart, data comparison and the like, and the reward analysis unit can be used for the user to edit the analysis data and the analysis requirements to be rewarded.
Preferably, the data storage processing module comprises a data encryption unit, a data compression unit, a data backup unit and a classified storage unit, and the classified storage unit is used for storing data to a private cloud or a common cloud of a user. In the preferred embodiment, data encryption, compression and backup are realized through the data storage processing module, and the classified storage unit is convenient for storing the data to a private cloud or a common cloud of a user according to user settings.
Preferably, the data analysis processing module comprises a system analysis unit, an analysis release unit and an aging analysis unit, the system analysis unit is used for performing system analysis on data according to the analysis requirement of the user, the analysis release unit is used for uploading the reward analysis data and the analysis requirement to a common cloud end for reward analysis, and the aging analysis unit is used for comparing and analyzing the data to be analyzed with the data in the regional time period according to the analysis requirement of the user. In the preferred embodiment, the system analysis unit is convenient for performing system analysis on the data according to the analysis requirements of the user, the analysis publishing unit is convenient for uploading the rewarded analysis data and the analysis requirements to the common cloud, and the aging analysis unit is convenient for performing comparative analysis on the data to be analyzed and the data in the regional time period according to the analysis requirements of the user.
Preferably, the data management module comprises a cache deletion unit, a duplicate file deletion unit and a useless file prompting unit, and the useless file prompting unit is used for prompting a user to delete unused data in unit time. In the preferred embodiment, the data management module is used for deleting the cache data, the repeated data and the long-time unused data in the user uploading data and the user private cloud data on the premise of user permission, and the data management module can also be used for deleting the cache data and the repeated data in the common cloud.
Preferably, the data management module includes an age file prompting unit, and the age file prompting unit is configured to prompt the user of age data to be deleted due to expiration and prompt the user of whether to delay. In the preferred embodiment, the user can be prompted by the data management module about to expire the deleted aging data and whether the user is delayed.
Compared with the prior art, the invention has the beneficial effects that:
the system of the invention is convenient for the time-efficient storage and analysis of user data, and is convenient for the autonomous open storage and data analysis and help seeking of the user data;
the data to be stored can be conveniently and quickly searched by a user through the local retrieval unit and the online retrieval unit, the processing mode of the data to be uploaded can be independently selected by the user through the data self-defining module, the data is encrypted, backed up and set for effective storage time, the user is advocated to store non-important data to the common cloud end in a public storage mode through data profit sharing mode so as to facilitate data sharing, the space occupation of the non-essential data to the private cloud end of the user is reduced, the repetition rate of the data to be stored by the user and the data in the common cloud end is convenient to know through the duplication searching unit so as to facilitate the sorting of data storage by the user according to the self requirement, the requirement of the user for editing data analysis such as data trend, data summarization, data diagram, data comparison and the like can be edited by the user through the rewarding analysis unit, the data storage processing module is used for realizing encryption, compression and backup of data, the classified storage unit is used for storing the data to a private cloud end or a common cloud end of a user according to user setting, the system analysis unit is used for carrying out system analysis on the data according to the analysis requirement of the user, the analysis and release unit is used for uploading appreciated analysis data and analysis requirements to the common cloud end, the aging analysis unit is used for comparing and analyzing the data to be analyzed with the data in a regional time period according to the analysis requirement of the user, the data management module is used for deleting cache data, repeated data and long-time unused data in the user uploading data and the private cloud end data of the user on the premise of user permission, the data management module can also delete the cache data and the repeated data in the common cloud end, and the data management module can prompt the user of aging data to be deleted soon, and prompts the user whether to defer.
The present invention will be explained in detail below with reference to the drawings and specific embodiments.
Drawings
FIG. 1 is a block diagram of the overall structure of the present invention;
FIG. 2 is a block diagram of a client system architecture of the present invention;
FIG. 3 is a block diagram of a server system architecture according to the present invention.
Description of the drawings: 10. a server; 11. a user management module; 12. a data storage processing module; 121. a data encryption unit; 122. a data compression unit; 123. a data backup unit; 124. a classification storage unit; 13. a data analysis processing module; 131. a system analysis unit; 132. an analysis issuing unit; 133. an aging analysis unit; 14. a data management module; 141. a cache deletion unit; 142. a duplicate file deletion unit; 143. a useless file prompting unit; 144. an aging file prompting unit; 15. a data transmission module; 16. a benefit calculation module; 17. a duplicate checking module; 20. a client; 21. a user login module; 22. an autonomous data retrieval module; 221. a local retrieval unit; 222. an online retrieval unit; 23. a timing retrieval module; 24. a data uploading module; 241. a duplicate checking unit; 242. a data classification uploading unit; 25. a data analysis module; 251. an analysis requiring unit; 252. an analysis result unit; 253. a reward analysis unit; 26. a data self-defining module; 261. an encryption selection unit; 262. a backup selection unit; 263. a timeliness selection unit; 27. a shared revenue module; 271. a sharing unit; 272. a profit withdrawal unit; 30. a cloud end; 31. a user private cloud; 32. and sharing a cloud terminal.
Detailed Description
In order to facilitate an understanding of the invention, the invention will now be described more fully hereinafter with reference to the accompanying drawings, in which several embodiments of the invention are shown, but which may be embodied in different forms and not limited to the embodiments described herein, but which are provided so as to provide a more thorough and complete disclosure of the invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may be present, and when an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present, as the terms "vertical", "horizontal", "left", "right" and the like are used herein for descriptive purposes only.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, and the knowledge of the terms used herein in the specification of the present invention is for the purpose of describing particular embodiments and is not intended to limit the present invention, and the term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
Please refer to fig. 1-3, in a preferred embodiment of the present invention, a mass data storage, analysis and management system based on cloud computing includes a server 10, a client 20 and a cloud 30 connected to the server 10 via internet, the cloud 30 includes a user private cloud 31 and a common cloud 32, the client 20 includes a user login module 21, an autonomous data retrieval module 22, a timing retrieval module 23, a data upload module 24 and a data analysis module 25, the user login module 21 is used for logging in by a user, the autonomous data retrieval module 22 is used for the user to select data to be uploaded autonomously, the timing retrieval module 23 is used for automatically retrieving data according to a retrieval time and a retrieval mode set by the user and uploading the data according to an upload time set by the user via the data upload module 24, the data upload module 24 is used for uploading the data to the server 10, the server 10 includes a user management module 11, a data storage processing module 12, a data analysis processing module 13, a data management module 14 and a data transmission module 15, the user management module 11 manages a login user, the data storage processing module 12 is configured to process data according to a user requirement and correspondingly transmit the data to a private cloud 31 or a common cloud 32 of the user through the data transmission module 15, the data analysis processing module 13 is configured to analyze the data according to the data analysis requirement of the user and transmit an analysis result to the client 20 through the data transmission module 15, the data management module 14 is configured to perform sorting and optimization on the data uploaded by the user and the cloud 30 data, the client 20 includes a data customization module 26, the data customization module 26 includes an encryption selection unit 261, a backup selection unit 262 and a timeliness selection unit 263, the encryption selection unit 261 is used for a user to select data to be encrypted, the backup selection unit 262 is used for the user to select data to be backed up, the timeliness selection unit 263 is used for the user to select a time period for storing data, namely, timeliness data, the client 20 comprises a shared income module 27, the shared income module 27 comprises a sharing unit 271 and an income promoting unit 272, the server 10 comprises an profit calculation module 16, the sharing unit 271 is used for displaying data uploaded to the shared cloud end 32 by the user, the profit calculation module 16 is used for calculating income according to the browsing amount of the data uploaded by the shared cloud end 32, the income promoting unit 272 is used for the user to promote income, the data uploading module 24 comprises a data searching unit 241 and a data classification uploading unit 242, the server 10 comprises a data searching module 17, and the data searching unit 241 is used for the user to input data to be searched, the duplication checking module 17 is configured to compare data to be duplicated with data in the common cloud 32 to detect whether duplicated data exists, the data classification uploading unit 242 is configured to distinguish data uploaded to the user private cloud 31 and the common cloud 32 by the user, the autonomous data retrieval module 22 includes a local retrieval unit 221 and an online retrieval unit 222, the local retrieval unit 221 is configured to retrieve local data to be uploaded by the user, the online retrieval unit 222 is configured to retrieve data to be uploaded on the network by the user, the data storage processing module 12 includes a data encryption unit 121, a data compression unit 122, a data backup unit 123 and a classification storage unit 124, and the classification storage unit 124 is configured to store the data to the user private cloud 31 or the common cloud 32.
It should be noted that, in this embodiment, after the user logs in through the user login module 21, there are two ways to store data;
firstly, the user selects storage by himself, in this way, the user retrieves the data to be uploaded through the self-contained data retrieval module 22, selects the processing mode of the data to be uploaded through the data customization module 26, for example, selects the data to be encrypted through the encryption selection unit 261, selects the data to be backed up through the backup selection unit 262, performs time-dependent selection on the data through the time-dependent selection unit 263, deletes the data after a week, and after the completion, the data can be defined and transmitted to the private cloud 31 or the common cloud 32 of the user through the data uploading module 24, and the data is transmitted to the server 10 after the definition is completed;
secondly, information is automatically retrieved and transmitted, a user can set retrieval time, retrieval mode, uploading time and uploading path in the timing retrieval module 23, the timing retrieval module 23 automatically retrieves data and uploads the data to the server 10 after the setting is finished, and the automatic retrieval information can also be encrypted, backed up and selected in timeliness;
after the data storage processing module 12 receives the data, the data encryption unit 121, the data compression unit 122 and the data backup unit 123 process the data according to the user setting, and the classification storage unit 124 transmits the data to the user private cloud 31 or the common cloud 32 according to the user requirement;
further, when the user retrieves the data to be uploaded through the autonomous data retrieval module 22, the user may retrieve the local data to be uploaded through the local retrieval unit 221 or retrieve the data to be uploaded on the network through the online retrieval unit 222;
further, the user can compare the data to be checked with the data in the common cloud 32 through the duplication checking unit 241 to detect whether the duplicated data exists, so that the user can select the public data or the private data according to the quantity of the duplicated data;
further, the user can check the data uploaded to the shared cloud 32 in the sharing unit 271, the benefit calculating module 16 calculates the benefit according to the browsing amount of the data uploaded by the shared cloud 32, and the benefit cashing unit 272 is used for cashing the user.
Please refer to fig. 1-3, in another preferred embodiment of the present invention, the data analysis module 25 is used for the user to upload the requirement of data analysis, the data analysis module 25 includes an analysis requirement unit 251, an analysis result unit 252 and a reward analysis unit 253, the analysis requirement unit 251 is used for the user to edit the specific requirement of data analysis, the analysis result unit 252 is used for displaying the analysis result of the data analysis processing module 13, the reward analysis unit 253 is used for the user to edit the analysis data and the analysis requirement to be rewarded, the data analysis processing module 13 includes a system analysis unit 131, an analysis issue unit 132 and an aging analysis unit 133, the system analysis unit 131 is used for performing system analysis on the data according to the analysis requirement of the user, the analysis issue unit 132 is used for uploading the rewarded analysis data and the analysis requirement to the common cloud 32, for reward analysis, the aging analysis unit 133 is configured to compare and analyze the data to be analyzed with the data in the regional time period according to the analysis requirement of the user.
It should be noted that, in this embodiment, a user may analyze the uploaded data, and the user edits the data analysis requirements such as data trend, data summarization, data chart, data comparison, aging data analysis, and the like in the analysis requirement unit 251, the system analysis unit 131 performs data analysis according to the user requirements and transmits the analysis result to the analysis result unit 252 through the data transmission module 15, and the aging analysis unit 133 may compare the data to be analyzed with the data in the regional time set by the user and transmit the analysis result to the analysis result unit 252 through the data transmission module 15, for example, compare the sales of a certain company for a certain day with the sales of the company for a certain quarter one day one by one;
the user can edit the reward amount, the data to be analyzed and the analysis requirement through the reward analysis unit 253, and the analysis publishing unit 132 transmits the information to the common cloud 32 through the data transmission module 15, so that other users can analyze and reward the common cloud 32 conveniently.
Referring to fig. 1-3, in another preferred embodiment of the present invention, the data management module 14 includes a cache deletion unit 141, a duplicate file deletion unit 142, and a useless file prompting unit 143, the useless file prompting unit 143 is used to prompt the user to delete the unused data in the unit time, the data management module 14 includes an aging file prompting unit 144, and the aging file prompting unit 144 is used to prompt the user about aging data due to deletion and prompt the user whether to delay.
It should be noted that, in this embodiment, the cache deletion unit 141, the duplicate file deletion unit 142, and the useless file prompting unit 143 in the data management module 14 may delete the cache data, the duplicate data, and the long-time unused data in the user upload data and the user private cloud data 31 on the premise of user permission, the cache deletion unit 141 and the duplicate file deletion unit 142 may also delete the cache data and the duplicate data in the common cloud 32, and the aging file prompting unit 144 may prompt the user about to expire the deleted aging data and prompt the user whether to postpone.
The specific process of the invention is as follows:
after the user logs in through the user login module 21, data storage is carried out in two modes;
firstly, the user selects storage by himself, in this way, the user retrieves the data to be uploaded through the self-contained data retrieval module 22, selects the processing mode of the data to be uploaded through the data customization module 26, for example, selects the data to be encrypted through the encryption selection unit 261, selects the data to be backed up through the backup selection unit 262, performs time-dependent selection on the data through the time-dependent selection unit 263, deletes the data after a week, and after the completion, the data can be defined and transmitted to the private cloud 31 or the common cloud 32 of the user through the data uploading module 24, and the data is transmitted to the server 10 after the definition is completed;
secondly, information is automatically retrieved and transmitted, a user can set retrieval time, retrieval mode, uploading time and uploading path in the timing retrieval module 23, the timing retrieval module 23 automatically retrieves data and uploads the data to the server 10 after the setting is finished, and the automatic retrieval information can also be encrypted, backed up and selected in timeliness;
after the data storage processing module 12 receives the data, the data encryption unit 121, the data compression unit 122 and the data backup unit 123 process the data according to the user setting, and the classification storage unit 124 transmits the data to the user private cloud 31 or the common cloud 32 according to the user requirement;
when the user retrieves the data to be uploaded through the autonomous data retrieval module 22, the user can retrieve the local data to be uploaded through the local retrieval unit 221 or retrieve the data to be uploaded on the network through the online retrieval unit 222;
the user can compare the data to be checked with the data in the common cloud 32 through the duplication checking unit 241 to detect whether the repeated data exists, so that the user can select the public data or the private data according to the quantity of the repeated data;
the user can check the data uploaded to the shared cloud 32 in the sharing unit 271, the benefit calculation module 16 calculates the benefit according to the browsing amount of the data uploaded by the shared cloud 32, and the benefit cash-out unit 272 is used for the user to perform benefit cash-out;
the user can analyze the uploaded data, the user edits the data analysis requirements such as data trend, data summarization, data chart, data comparison, aging data analysis and the like in the analysis requirement unit 251, the system analysis unit 131 analyzes the data according to the user requirements and transmits the analysis result to the analysis result unit 252 through the data transmission module 15, the aging analysis unit 133 can compare the data to be analyzed with the data in the regional time set by the user and transmit the analysis result to the analysis result unit 252 through the data transmission module 15, for example, the sales of a certain company in a certain day and the sales of the company in a certain quarter are compared one by one;
the user can edit the reward amount, the data to be analyzed and the analysis requirement through the reward analysis unit 253, and the analysis publishing unit 132 transmits the information to the common cloud 32 through the data transmission module 15 so that other users can analyze and reward the common cloud 32 conveniently;
the cache deletion unit 141, the duplicate file deletion unit 142, and the useless file prompt unit 143 in the data management module 14 may delete cache data, duplicate data, and long-time unused data in the user upload data and the user private cloud data 31 on the premise of user permission, the cache deletion unit 141 and the duplicate file deletion unit 142 may also delete cache data and duplicate data in the common cloud 32, and the aging file prompt unit 144 may prompt the user about to expire the deleted aging data and prompt the user whether to delay.
The invention is described above with reference to the accompanying drawings, it is obvious that the invention is not limited to the above-described embodiments, and it is within the scope of the invention to adopt such insubstantial modifications of the inventive method concept and solution, or to apply the inventive concept and solution directly to other applications without modification.
Claims (10)
1. A mass big data storage, analysis and management system based on cloud computing comprises a server (10), a client (20) and a cloud (30) which are connected with the server (10) through the Internet, and is characterized in that the cloud (30) comprises a user private cloud (31) and a common cloud (32), the client (20) comprises a user login module (21), an autonomous data retrieval module (22), a timing retrieval module (23), a data uploading module (24) and a data analysis module (25), the user login module (21) is used for logging in by a user, the autonomous data retrieval module (22) is used for the user to autonomously select data to be uploaded, the timing retrieval module (23) is used for automatically retrieving data according to retrieval time and retrieval mode set by the user and uploading the data according to the uploading time set by the user through the data uploading module (24), the data uploading module (24) is used for uploading data to the server (10), and the data analysis module (25) is used for uploading data analysis requirements by a user;
the server (10) comprises a user management module (11), a data storage processing module (12), a data analysis processing module (13), a data management module (14) and a data transmission module (15), the user management module (11) manages a login user, the data storage processing module (12) is used for processing data according to user requirements and correspondingly transmitting the data to a private cloud end (31) or a common cloud end (32) of the user through the data transmission module (15), the data analysis processing module (13) is used for analyzing the data according to data analysis requirements of the user and transmitting an analysis result to a client (20) through the data transmission module (15), and the data management module (14) is used for sorting and optimizing data uploaded by the user and data of the cloud end (30).
2. The mass big data storage, analysis and management system based on cloud computing of claim 1, wherein the client (20) comprises a data customization module (26), the data customization module (26) comprises an encryption selection unit (261), a backup selection unit (262) and a timeliness selection unit (263), the encryption selection unit (261) is used for a user to select data to be encrypted, the backup selection unit (262) is used for the user to select data to be backed up, and the timeliness selection unit (263) is used for the user to select a time period for data storage, namely, timeliness data.
3. The mass big data storage, analysis and management system based on cloud computing of claim 1, wherein the client (20) comprises a shared profit module (27), the shared profit module (27) comprises a sharing unit (271) and a profit revenues unit (272), the server (10) comprises a profit computing module (16), the sharing unit (271) is used for displaying data uploaded to the common cloud (32) by users, the profit computing module (16) is used for computing profits according to browsing volumes of the data uploaded by the common cloud (32), and the profit revenues unit (272) is used for revenue revenues for users.
4. The mass big data storage, analysis and management system based on cloud computing according to claim 1, wherein the data uploading module (24) comprises a duplicate checking unit (241) and a data classification uploading unit (242), the server (10) comprises a duplicate checking module (17), the duplicate checking unit (241) is used for a user to input data to be checked, the duplicate checking module (17) is used for comparing the data to be checked with data in the common cloud end (32) to detect whether there is duplicate data, and the data classification uploading unit (242) is used for distinguishing the data uploaded to the user private cloud end (31) and the common cloud end (32).
5. The mass big data storage, analysis and management system based on cloud computing according to claim 1, wherein the autonomous data retrieval module (22) comprises a local retrieval unit (221) and an online retrieval unit (222), the local retrieval unit (221) is used for a user to retrieve local data to be uploaded, and the online retrieval unit (222) is used for a user to retrieve data to be uploaded on a network.
6. The mass big data storage, analysis and management system based on cloud computing according to claim 1, wherein the data analysis module (25) comprises an analysis requirement unit (251), an analysis result unit (252) and a reward analysis unit (253), the analysis requirement unit (251) is used for a user to edit specific requirements of data analysis, the analysis result unit (252) is used for displaying the analysis result of the data analysis processing module (13), and the reward analysis unit (253) is used for the user to edit analysis data and analysis requirements to be appreciated.
7. The mass big data storage, analysis and management system based on cloud computing of claim 1, characterized in that, the data storage processing module (12) comprises a data encryption unit (121), a data compression unit (122), a data backup unit (123) and a classification storage unit (124), the classification storage unit (124) is used for storing data to a user private cloud (31) or a common cloud (32).
8. The mass big data storage, analysis and management system based on cloud computing according to claim 6, wherein the data analysis processing module (13) comprises a system analysis unit (131), an analysis publishing unit (132) and an aging analysis unit (133), the system analysis unit (131) is used for performing system analysis on data according to analysis requirements of users, the analysis publishing unit (132) is used for uploading appreciated analysis data and analysis requirements to the common cloud end (32) for reward analysis, and the aging analysis unit (133) is used for performing comparative analysis on data to be analyzed and data in regional time periods according to analysis requirements of users.
9. The mass big data storage, analysis and management system based on cloud computing according to claim 1, wherein the data management module (14) comprises a cache deletion unit (141), a duplicate file deletion unit (142) and a useless file prompting unit (143), and the useless file prompting unit (143) is used for prompting a user to delete unused data in a unit time.
10. The mass big data storage, analysis and management system based on cloud computing according to claim 2, wherein the data management module (14) comprises an aging file prompting unit (144), and the aging file prompting unit (144) is used for prompting the user about to expire the deleted aging data and prompting the user whether to postpone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110973120.XA CN113868221B (en) | 2021-08-24 | 2021-08-24 | Mass big data storage, analysis and management system based on cloud computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110973120.XA CN113868221B (en) | 2021-08-24 | 2021-08-24 | Mass big data storage, analysis and management system based on cloud computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113868221A true CN113868221A (en) | 2021-12-31 |
| CN113868221B CN113868221B (en) | 2024-05-24 |
Family
ID=78988083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110973120.XA Active CN113868221B (en) | 2021-08-24 | 2021-08-24 | Mass big data storage, analysis and management system based on cloud computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113868221B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116074078A (en) * | 2023-01-10 | 2023-05-05 | 广东浩传管理服务有限公司 | Network communication safety monitoring system and method based on data encryption |
| CN116192847A (en) * | 2023-02-01 | 2023-05-30 | 南京艾若博特信息技术有限公司 | Block chain network dividing system adopting GPS positioning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150276208A1 (en) * | 2014-03-26 | 2015-10-01 | Rockwell Automation Technologies, Inc. | Cloud-level analytics for boiler networks |
| CN108170558A (en) * | 2017-11-29 | 2018-06-15 | 江苏摇铃网络科技有限公司 | A kind of magnanimity big data storage based under cloud environment, analysis and management system |
| CN110489577A (en) * | 2019-08-06 | 2019-11-22 | 腾讯医疗健康(深圳)有限公司 | Medical imaging management method and device, ophthalmoscopic image processing method, electronic equipment |
| CN112102905A (en) * | 2020-08-27 | 2020-12-18 | 杨凯 | Intelligent medical cloud service platform |
-
2021
- 2021-08-24 CN CN202110973120.XA patent/CN113868221B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150276208A1 (en) * | 2014-03-26 | 2015-10-01 | Rockwell Automation Technologies, Inc. | Cloud-level analytics for boiler networks |
| CN108170558A (en) * | 2017-11-29 | 2018-06-15 | 江苏摇铃网络科技有限公司 | A kind of magnanimity big data storage based under cloud environment, analysis and management system |
| CN110489577A (en) * | 2019-08-06 | 2019-11-22 | 腾讯医疗健康(深圳)有限公司 | Medical imaging management method and device, ophthalmoscopic image processing method, electronic equipment |
| CN112102905A (en) * | 2020-08-27 | 2020-12-18 | 杨凯 | Intelligent medical cloud service platform |
Non-Patent Citations (1)
| Title |
|---|
| 胡涛;周兵;郑明辉;纪忠原;: "基于Hadoop的移动云存储系统研究与实现", 华中科技大学学报(自然科学版), no. 2, 10 January 2014 (2014-01-10) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116074078A (en) * | 2023-01-10 | 2023-05-05 | 广东浩传管理服务有限公司 | Network communication safety monitoring system and method based on data encryption |
| CN116074078B (en) * | 2023-01-10 | 2023-07-14 | 广东浩传管理服务有限公司 | Network communication safety monitoring system and method based on data encryption |
| CN116192847A (en) * | 2023-02-01 | 2023-05-30 | 南京艾若博特信息技术有限公司 | Block chain network dividing system adopting GPS positioning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113868221B (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220398267A1 (en) | Content discovery systems and methods | |
| US10853744B2 (en) | Distributing a task to multiple workers over a network for completion while providing quality control | |
| JP5078674B2 (en) | Analysis system, information processing apparatus, activity analysis method, and program | |
| CN106407078B (en) | Client performance monitoring device and method based on information exchange | |
| CN113868221A (en) | Mass big data storage, analysis and management system based on cloud computing | |
| WO2016019632A1 (en) | System and method for implementing collection, management and authorization of personal big data | |
| US20110313820A1 (en) | Using virtual currency to compensate workers in a crowdsourced task | |
| KR101053968B1 (en) | Patent automatic evaluation method of patent automatic evaluation system | |
| JP4700462B2 (en) | Database usage system | |
| US10091323B2 (en) | Social discovery feed for facilitating social exploration in social networking environments | |
| CN111159561A (en) | Method for constructing recommendation engine according to user behaviors and user portrait | |
| JP4627539B2 (en) | Load test system, load test data creation method, and program thereof | |
| CN106815284A (en) | The recommendation method and recommendation apparatus of news video | |
| US20160188676A1 (en) | Collaboration system for network management | |
| FR2814569A1 (en) | METHOD AND SYSTEM FOR ANALYZING AND PROCESSING DATA ON THE INTERNET IN REAL TIME | |
| CN110008405A (en) | A kind of personalization message method for pushing and system based on timeliness | |
| CN101415028A (en) | Biology remote authentication and prewarning analysis system for medicine intermedium | |
| CN110457257B (en) | Archive knowledge management device for mobile terminal and mobile terminal | |
| JP5603447B2 (en) | Log management system, log management apparatus, and log management method | |
| Kapusta et al. | User Identification in the Process of Web Usage Data Preprocessing. | |
| CN114363103B (en) | Method, device and computer readable medium for processing conference information | |
| CN112800127B (en) | Data mining analysis method and device based on transaction bill | |
| KR101638986B1 (en) | Server, method and system for information service with big data classification user-based and recording medium thereof | |
| CN117453982B (en) | File management file classification system | |
| KR20240051519A (en) | Server, method and system for information service with big data classification user-based and recording medium t |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |