CN107770276A - It is a kind of to realize that user data manages the network system and method with renewal independently - Google Patents
It is a kind of to realize that user data manages the network system and method with renewal independently Download PDFInfo
- Publication number
- CN107770276A CN107770276A CN201711011779.7A CN201711011779A CN107770276A CN 107770276 A CN107770276 A CN 107770276A CN 201711011779 A CN201711011779 A CN 201711011779A CN 107770276 A CN107770276 A CN 107770276A
- Authority
- CN
- China
- Prior art keywords
- data
- data file
- user
- file
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/2107—File encryption
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention propose it is a kind of realize user data manage independently with update network system include：The terminal node of multiple users, for Various types of data caused by on-site collection business, and stored or updated and arrive data source nodes；One or more data source nodes, the Various types of data collected for storing user in terminal node；One or more connecting nodes, it is made up of some terminal node in subnet, or several terminal nodes are contributed a part of memory space and set up " shared node " respectively；The connecting node is connected with data source nodes, when user to user data file is operated, it is only necessary to the data file in the connecting node inside identical subnet is operated, just completes the operation of the data file to being stored in data source nodes；The stable operation of system is realized by file system background process, and by abnormality detecting program, various abnormal conditions are handled.
Description
Technical field
It is more particularly to a kind of to realize that user data manages the network with renewal independently the present invention relates to field of computer
System and method.
Background technology
Big data refers to the sea for needing new tupe to have stronger decision edge, insight and process optimization ability
Amount, high growth rate and diversified information assets.Write in Victor mayer-Schoenberg and Kenneth Cook
《The big data epoch》In book, big data processing refers to not have to random analysis method, statistical method, but is entered simultaneously using all data
Row analyzing and processing.So if big data analysis work will spend the time using distributed computing architecture than single computer
It is short.Its characteristic is using cloud storage technology, distributed data base, distributed treatment, is excavated in mass data valuable
Information." refinement " goes out valuable information from mass data, for this ability of data processing and the network architecture and huge
Big challenge.
With the fast development of network technology, the capacity and diversity of data quickly increase, and the algorithm of processing data is answered
Miscellaneous degree is but difficult to improve, and relies on personal experience and manual operations to describe data, labeled data, selection feature, extraction feature, place
The method for managing data, it has been difficult to meet the needs of big data rapid growth, how efficient process big data has become one
Urgent problem.In existing distributed big data treatment technology, the Hadoop distributed documents such as based on MapReduce
System and its data processing method, most resources can be all wasted in the data transfer between computer cluster how
Realize that user data is managed independently and updated, also becoming one must solve the problems, such as.
The research of deep learning method is broken through, and a direction for being worth exploring is specified to solve big data process problem.
In bibliography 1 " G.E.Hinton and R.R.Salakhutdinov, " Reducing the dimensionality
Ofdata with neural networks, " in Science, vol.313, no.5786, pp.504-507,2006 ",
Hinton et al. proposed the successively initialization training method for depth confidence net in 2006, and this is deep learning method
Study starting point, this method broken continue for decades deep learning systematic training is difficult and situation that effect is bad.Depth
Study is with different levels abstract by simulating human brain, and bottom data is successively mapped and obtains more abstract feature, and it can be from
Feature is automatically extracted in big data, and good treatment effect is obtained by the sample training of magnanimity.In fact, big data is fast
Speed increases and the research of deep learning is complementary, and the rapid growth of one side big data needs a kind of efficient process magnanimity
The method of data, the training of another aspect deep learning system need the sample data of magnanimity.In brief, big data can make
The performance of deep learning reaches ultimate attainment.
The set of one or more data sets be it is so big or complicated so that traditional database management tools and/or
Data handling utility (for example, statistics bag of relational database and desktop computer) can not manage data in tolerable time quantum
Collection.Generally, the use of the application of big data is things and is directed to or is absorbed in terminal user.For example, web page search engine, society
Media application, market application and retail application is handed over to use and manipulate big data.Can be by allowing modern more processes, multinuclear to take
The distributed data base that is fully utilized of parallel processing capability of business device supports big data.
The rows such as existing finance, telecommunications, game, web page search engine, social media application, market application and retail application
Industry, user are producing substantial amounts of data all the time, in order to ensure the safety of user data and efficiency, it is necessary to propose a kind of new
User data manage independently with renewal network system and method.
The content of the invention
In order to meet the needs of set forth above, the present invention proposes following technical scheme.
The present invention proposes a kind of network system for realizing user data and managing independently and updating, suitable for the number to user
Be managed and update according to file, it is described realize user data manage independently with update network system include：Multiple users'
Terminal node, for Various types of data caused by on-site collection business, and stored or updated and arrive data source nodes；The user
Terminal node be physically under the jurisdiction of different subnets, the user inside the subnet is physically connected using higher bandwidth
Connect, realize the high-speed transfer of data；One or more data source nodes, all kinds of numbers collected for storing user in terminal node
According to；Also include：One or more connecting nodes, it is made up of some terminal node in subnet, or several terminal nodes point
A part of memory space is not contributed and is set up " shared node "；The connecting node is connected with data source nodes, works as user
When being operated to subscriber data file, it is only necessary to which the data file in the connecting node inside identical subnet is grasped
Make, just complete the operation of the data file to being stored in data source nodes；System is realized by file system background process
Stable operation, and by abnormality detecting program, various abnormal conditions are handled.
In certain embodiments, data file includes data file head, and data file head is subdivided into following components：
Data file ID, creation time and the finger print information part with data File owner's private key encryption.
In certain embodiments, the business is the related business of real-time process, business, the financial circles of factory's correlation
Business, game service and other types of business.
In certain embodiments, stored using improved distributed data set pair data file, improved distribution
Data file in object data set is stored in a manner of subregion (Partition), the data of different subregions
It can be distributed on different machines, be handled in bottom by parallel computation.
In certain embodiments, data File owner is that all share of data file person members share key.
The present invention also proposes a kind of to be suitable to realizing that user data manages independently and the side that runs in the network system of renewal
Method, the encryption for realizing data update and shared, comprise the following steps：
The data file of encryption is stored in data source section by S1, data File owner by the way of data encryption upload
Point；
S2, data File owner specify the users to share data file；
S3, when data File owner needs to update the data file of itself, data file is Resealed, using data
The data file of encryption is stored in data source nodes by the mode that encryption uploads.
In certain embodiments, the data file is dynamic measurement results or control data, including assignment procedure operation
In the data of change, the data that change in assigned operation parameter；
The assigned operation parameter is the data in set point, process and hardware alarm and event.
In certain embodiments, in the mode that data encryption in step sl uploads, the data file of encryption is carried out
Data access controls (DAC), and the protections of data, data sharing and complete are carried out using public-key cryptosystem and symmetric cryptosystem
Whole property verification.
In certain embodiments, it is described also to include：Share of data file person is decrypted down by the way of data deciphering download
Carry data.
In certain embodiments, it is described also to include：After share of data file person updates the data file, data file institute is obtained
The private key for the person of having, with re-encrypted finger print information part, so as to which the data file of encryption is stored in data source nodes.
By the present invention, user easily can manage independently and update the data in network system, and realize number
According to file security and improve data renewal with management efficiency.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, it is various other the advantages of and benefit it is common for this area
Technical staff will be clear understanding.Accompanying drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And in whole accompanying drawing, identical part is denoted by the same reference numerals.In the accompanying drawings：
Accompanying drawing 1 is shown realizes that user data manages the network system knot with renewal independently according to embodiment of the present invention
A kind of schematic diagram of structure.
Accompanying drawing 2 is shown realizes that user data manages the network system frame with renewal independently according to embodiment of the present invention
A kind of schematic diagram of structure.
Accompanying drawing 3 shows the physical format of the data file according to embodiment of the present invention.
Accompanying drawing 4 shows the logical architecture schematic diagram according to the data file system of embodiment of the present invention.
Embodiment
The illustrative embodiments of the disclosure are more fully described below with reference to accompanying drawings.Although this public affairs is shown in accompanying drawing
The illustrative embodiments opened, it being understood, however, that may be realized in various forms the disclosure without the reality that should be illustrated here
The mode of applying is limited.Conversely, there is provided these embodiments are to be able to be best understood from the disclosure, and can be by this public affairs
The scope opened completely is communicated to those skilled in the art.
Google is a kind of important and widely used big data in the MapReduce parallel computation frames proposed in 2014
Handle solution.MapReduce is that user shields many bottoms by map and the simple DLLs of reduce two
Parallelization handles details, so as to significantly simplify the development difficulty of data-intensive applications.In addition, MapReduce frameworks are also
A lot of other key properties, including load balancing, elastic expansible and System Error-tolerance Property etc. are provided, these characteristics cause
MapReduce becomes parallelization Computational frame that is a kind of easy to maintain and using.Hadoop opens as MapReduce one kind
Source is realized, is widely used and is studied in industrial quarters and academia.
In order to ensure the safety of user data and efficiency, realize that user data is managed independently and updated we have proposed one kind
Network system and method.It is described realize user data manage independently with update network system include：
The terminal node of multiple users, for Various types of data caused by on-site collection business, and stored or updated and arrived
Data source nodes；
The terminal node of the user is physically under the jurisdiction of different subnets, and the user inside the subnet is physically
Connected using higher bandwidth；
One or more data source nodes, the Various types of data collected for storing user in terminal node；
One or more connecting nodes, are made up of some terminal node in subnet or several terminal nodes divide
A part of memory space is not contributed and is set up " shared node "；
The connecting node is connected with data source nodes, when user to user data are operated, it is only necessary to place
The data in connecting node inside the identical subnet are operated, and just complete the behaviour of the data to being stored in data source nodes
Make.
In certain embodiments, terminal node can perform physical function with the field apparatus of control process.It is for example, golden
Melt the counter terminal of mechanism, the controller in communication system, the on-site terminal of data acquisition equipment.They are distributed in multiple differences
The remote or near geographical position of distance, and physically and be in logic connected with corresponding data source nodes.The number
Can be private server, service station or all types of main frames of special storage data according to source node.
Each terminal node rate collection to generate, create, receive or otherwise observe local data respectively
Local data, and collected local data is directed or through into connecting node and is stored in respective corresponding data source node
In (for example, embedded big data holder, storage server) in, such as save as local, historization big data.It is distributed
, the big data of localization is collected and analysis allows more timely feeding back to the potentially harmful situation that occurs at the scene.
For example, in exemplary scene, controller is to the portion as the control loop being included in the process plant of production specific products
The set (for example, field apparatus and optional miscellaneous equipment) of the process control equipment divided is controlled.Thing in control loop
Certain combination of part causes the product quality of poor quality (when finally later (for example, several small after the combination of event occurs
When) generation product when).Controller using its big data analyzer come to event combination generation when or in the near future (for example,
When the data corresponding with the generation of event are sent into big data holder (storage server)) given birth to by the combination of event
Into process data automatically analyzed, rather than the product quality inferior of a few houres and overhauled after being detected and determined
To determine the basic reason of product quality inferior (as currently conducted in known Process Control System).Big data point
Parser can carry out to generate the learned knowledge of prediction product quality inferior based on these events, and/or can be automatically
Adjust in real time or change one or more parameters or process to mitigate the influence of the combination of event (if they occur in future).
For example, the value quilt that big data analyzer can determine the set point being corrected or the parameter value being corrected and this is corrected
Controller use is preferably to adjust and manage control loop.
One of application scenarios of the present invention are as shown in Figure 1.In the network system, there are three nodes to be protected as user data
The data source deposited and managed, data are provided to other nodes:Node G, H and I.Node A, B, C, D are physically under the jurisdiction of some
Subnet, such as, some LAN.There is higher bandwidth connection between A, B, C, D；Node E, F are physically under the jurisdiction of another
Subnet.Node A, B, C, D, E, F data are stored in one or more of three data sources respectively, from three data sources
One or more extraction data, carry out processing locality (as shown in the figure).After the completion of processing locality, it is also necessary to store data into
Data source.
If the connection bandwidth not between the node in same subnet is relatively low, it is evident that, come from same subnet
Different nodes respectively from data source extraction data will expend substantial amounts of bandwidth resources.If data can be extracted son first
Some in net or some nodes, other nodes in same subnet directly can extract data from these nodes, due to son
Net is internal to possess higher bandwidth connection, so so processing can may greatly save bandwidth resources.
Therefore, we are improved the network system shown in Fig. 1, increase several " connecting nodes " wherein, such as Fig. 2
Shown, Vl, V2, V3 are represented " connecting node " in respective subnet respectively.They can be some node in subnet, can also
It is that several nodes are contributed a part of memory space and set up " shared node " respectively.Solid arrow represent data send and
Transmission in receive process, dotted line connection represent the data transfer in subnet.Each node may also be exactly connecting node in itself
A part, so the connection between them is represented by dashed line.In addition, data source nodes H data are due in node A, B, C, D
Place subnet is only used by node B, so not needing connecting node to deposit its data.Then, the number in data source nodes G, H, I
According to the subnet being only communicated to where node A, B, C, D once；Data in data source nodes I are sent to where node E, F
Subnet once.And the node in subnet all obtains data inside subnet.Compared to the network in Fig. 1, the number in node G, H, I
According to being sent to A, B, C respectively, the subnet where D 2,1,3 times, the data in node I are sent to the subnet where node E, F
2 times.Based on the hypothesis of " bandwidth is higher in subnet, and the outer bandwidth of subnet is relatively low ", the band that can be made full use of in subnet is configured so that
Wide resource and greatly reduce outside bandwidth consumption.In fig. 2, data source nodes G be by connecting node V1 and node A, B,
Subnet where C, D is connected, as long as being sent to connecting node V1 so as to data source nodes G data, then node A, B, C, and D institutes
User's can in subnet directly obtains data from connecting node V1, after handling data, it is only necessary to by data
Connecting node V1 is stored in, then the data syn-chronization by operation can be stored in data source G by connecting node V1 automatically.Data source section
Point H is to be connected by the terminal node B of user with subnet where node A, B, C, D, as long as the data so as to data source nodes H
Terminal node B is sent to, then node A, B, C, user's can in subnet where D directly obtain data from node B, right
After data are handled, it is only necessary to data are stored in into node B, then node B automatically can deposit the data syn-chronization by operation
To data source H.Data source nodes I is connected by subnet where connecting node V3 and node E, F, so as to data source nodes I
As long as data be sent to connecting node V3, then user's can in subnet where node E, F is directly from connecting node V3
Obtain data, after handling data, it is only necessary to data are stored in connecting node V3, then connecting node V3 can automatically by
Data syn-chronization by operation is stored in data source I.
It is noted that in the network system environment of reality, each node is typically not aware that other sections in system
The situation of point one by one they do not know the presence of subnet, they do not know with oneself which similar node and they have it is similar
Inquiry request (identical data storage source), do not know yet those with itself have similar inquiry request node whether and from
Oneself is close.On the other hand, after a connecting node is established, this connecting node just as data source nodes, turns into
It is available for one of resource that other nodes use.The user node newly added directly can obtain data from connecting node, without
Access data source nodes.
In cloud era, Hadoop can not only utilize its distributed data files system as a distributed Open Source Platform
Storage environments of the system HDFS as big data (Big Data), but also support the distributed volumes of MapReduce that Google is proposed
Journey mode, nowadays it has been widely used in distributed and Distributed Computing Platform.But by the use of Hadoop as greatly
The storage environment of data (Big Data), the confidentiality of data, integrality and data access control (DAC) be equally worth research and
Thinking.
As an improvement, the present invention proposes a kind of improved distributed data files storage mode, to realizing user data
Manage independently and carry out data access control (DAC) with the data file stored in the network system of renewal, at the same it is close using public key
Code system and symmetric cryptosystem carry out protection, data sharing and the completeness check of data.
The encryption and decryption of data file be using user as core, only validated user could carry out data file upload and
Download, and whether specified data file shares to other users.Data File owner uses share of data file to other
Family can specify following three kinds of access rights:Read-only (R), only write it is (W) and readable writeable (RW).
Data file logical format
Next the logical construction of data file is will be described in detail, explains the symbol of correlation first:H [] is represented
Hash function, conventional hash function has MD5, SHA-1 etc., commonly used to calculate eap-message digest；E [] represents symmetric cryptography；
EPUn [] represents the public key encryption of the asymmetric cryptographic key centering using user n；EPRn [] represents Mi Lang pairs of asymmetric encryption
In private key encryption.
The logical format of data file storage, is mainly made up of three parts:Data file head, data file head summary info
With the encryption part of actual data files, wherein data file head summary employs hash function, and content data file encryption is adopted
With symmetric encipherment algorithm, as shown in table 1.
Table 1：Data file storage format
| Data file head | H [data file head] | E [content data file] |
Data file head can be subdivided into following components again：Data file ID, Data Filename, data file own
Person, data file description, creation time and the finger print information part with data File owner's private key encryption.As shown in table 2.
Table 2：Data file head form
Finger print information part can be subdivided into following three parts:Data file AES, user profile and data text
Part synopsis.AES part specifies the symmetric encipherment algorithm used；User profile part, it is divided into data file and owns
Person and several share of data file person's item of information, each item include their user name, data file access rights and with public
Symmetrical close Hu after steel encryption.Content data file summary part is to carry out Hash calculation to content data file, such as the institute of table 3
Show.
Table 3:Data file finger print information form
Data file physical format
Realize user data manage independently with renewal network system data file storage system in, data file is
Stored in the form of stream data file.Enter in order to facilitate to the shared information preserved in data file, content data file
Row management and access, employ physical format of the new form as data file, and specific form is as shown in Figure 3, wherein:
● the content data file after data file head, data file head summary, encryption is 1. 8. 9. represented respectively
● 2. represent data file ID；3. represent Data Filename；4. represent data File owner；5. represent data text
Part describes；6. represent creation time；7. represent finger print information
● represent AES；(b) content data file summary is represented；(C) user profile is represented；(d) data text is represented
The part owner；(e) share of data file person is represented
● (1) represent user name；(2) authority is represented；(3) key after encryption is represented
When encryption data is put into storage server by data File owner, by the way of data encryption upload.
After data file is encrypted, data File owner can specify the users to share data file, and assign corresponding
Three kinds of authorities (R, W, RW), its implementation process is as follows:
1) user uploads data file, and generates symmetric cryptography key key at random；
2) the recording data files owner and corresponding authority _, with the public key encryption key of data File owner；
3) if sharing user, the then shared user specified according to data File owner and authority, record is each respectively
Title, the authority of individual user, with the public key encryption key of sharer, form user profile item；
4) recording data files AES title, user profile and content data file summary, form finger print information；
5) recording data files ID, Data Filename, data file master, data file description, creation time and use data
The private key encryption finger print information of file master obtains encryption information, forms data file head；
6) recording data files header, the hashed value of data file head and with key key data file encryption contents, shape
Into storage document format data, IO is output to disk；
When user i decrypts downloading data, by the way of data deciphering download.
Data File owner and the share of data file person for possessing readable (R, RW) authority can enter to data file
Row decryption is downloaded.Its implementation process is as follows:
1) data file for reading in new form is parsed, and warning information " R1 " is provided if it can not correctly parse；
Otherwise enter in next step；
2) data file header is calculated hashed value, if inconsistent with H (data file head) partial content, provided
Warning information " R2 "；Otherwise enter in next step；
3) with the public key decryptions of the data File owner data file, warning information is provided if can not decrypt "
R3"；Otherwise enter in next step；
4) download user right to judge, if downloading user is data File owner or possesses readable (R or RW)
The share of data file person of authority then can normally download, into next step；Otherwise warning information " R4 " is provided；
5) with the symmetrical key for the private key decryption public key encryption for downloading user, warning information is provided if it can not decrypt
“R5”；Otherwise symmetric key key is obtained after decrypting, into next step；
6) with the symmetric encipherment algorithm and symmetric key key specified, ciphertext data file content part, if can not be normal
Decryption, then provide warning information " R6 ", otherwise enters in next step；
7) the content data file part after decryption is found hash value, entered with the data file summary part in finger print information
Row compares, and warning information " R7 " is provided if differing, and otherwise correct decryption obtains content data file.Above-mentioned steps can root
According to adjustment execution sequence before and after being actually needed, or execution step is deleted, rather than must be according to described tandem.
The warning information symbol description table of table 2. 4
| Symbol | Explanation |
| R1 | Document format data parsing failure, data file are tampered |
| R2 | Data file header is tampered |
| R3 | Data File owner's information errors, data file source are insincere |
| R4 | Insufficient permission (lack can read right) |
| R5 | Unsymmetrical key is to mismatching |
| R6 | Data file decryption failure |
| R7 | Content data file is tampered |
Data owner can be updated to data file.Data File owner updates the data file of itself, only
Data file, including re-encrypted data file need to be Resealed, regenerates the content data file summary in finger print information
Part, with private key encryption finger print information.And for share of data file person, if possess can write permission, updating the data file
The private key for needing to obtain data File owner afterwards comes re-encrypted finger print information part, in order to ensure the safeguard protection of private key,
The work of re-encrypted can transfer to key distribution center (KDC) to coordinate to complete.
Share of data file person updates the data document flow：
File-sharing person sends file modification request to key distribution center (KDC)；
File modification request is transmitted to file owner by key distribution center (KDC)；
If file owner disagrees modification, request terminates；
If file owner agrees to modification, transmission, which replies message, gives key distribution center (KDC)；
Key distribution center (KDC) will reply message again returns to file-sharing person；
File-sharing person will send modification content and be sent to key distribution center (KDC)；
Key distribution center (KDC) will send modification content and is transmitted to file owner again；
Amended file content is write file by file owner；
After write-in terminates, then request terminates.
In certain embodiments, the data file system used in the terminal node of user, connecting node, data source nodes
The logical architecture of system is as shown in Figure 4.Data File owner the inquiry of file, upload, retrieval and download during all
Need to carry out authentication.By way of authentication, using the method for file block encryption so that data file is in various behaviour
During work, play a part of secret protection.The data text that different data source nodes form on by geographical position
In part system, the encryption of file uploads, encrypts retrieval, decrypts and download all using the document format data being described in detail below, makes
Data file is obtained during real-time update is shared, is capable of the integrality of the protection data of safety.By one or more numbers
In data file system according to source node composition, the stable operation of system is realized by file system background process, and by different
Various abnormal conditions are handled by often detection program.
The terminal node of user can collect dynamic measuring data and control data and each other types of data,
The information that any user for collecting which data is provided without mark in advance or instruction.That is, the configuration of user
Eliminate to collect for the measurement data and control data of historization and the data of various other types at user
Identity any instruction.In current known Process Control System, operator or user generally have to the terminal to user
Node is configured (and in certain embodiments, described by specifying will be collected or preserve by identifying which data
Data are by the time for being collected or preserving or frequency) capture measurement data and control data.Data to be collected identity (with
And alternatively, time/frequency) it is included in the configuration of process control equipment.On the contrary, user was not necessarily configured to the phase
Hope the measurement data collected and the identity of control data and its time/frequency collected.In fact, in embodiment, automatically
Collect being directly generated by user and/or all measurement data and control data that are directly received in user and it is all its
The data of its type.
Each data text for supporting the present invention can be included with the network system of renewal by realizing that user data is managed independently
The part system and multiple nodes or equipment that are connected by computer network are (for example, can be the terminal node of user, connecting node
And/or data source nodes).It is (all that various types of data are collected locally and store at each equipment that can be in multiple equipment
Such as, the related data of the related data of real-time process, factory, the data of financial business, game data and other types of
Data).At each equipment in multiple equipment, locally-stored data can be locally analyzed at equipment to create
Or generation description is across time and/or significant relation, pattern, the correlation across at least some of data set in various data sets
The learned knowledge of property, trend etc..It is at least some of in collected data and/or the learned knowledge generated in embodiment
It can be transmitted between the node of computer network and equipment, for example, for improving the control to process in real time.
In some configurations, at least some of node or equipment in the node or equipment of computer network away from distributed big data equipment and
Set.
One can be supported with the network system of renewal and/or the subnet included in it by realizing that user data is managed independently
Or multiple appropriate Routing Protocols, it may for example comprise the agreement in Internet protocol (IP) group is (for example, UPD (user datagrams
Agreement), TCP (transmission control protocol), Ethernet etc.), or other appropriate Routing Protocols.Generally, it is included in distributed big
In the shared computer network system of data real-time exchange each equipment or node (for example, can be user terminal node,
Connecting node and/or data source nodes) all support to be supported by computer network at least the one of one or more Routing Protocols
Individual application layer (also, for some equipment, extra play).In embodiment, each equipment or node are realizing that user data is only
Standpipe is managed with for example being uniquely identified in the network system of renewal by unique network address.
Furthermore, it is possible to realize user data manage independently with renewal network system at least some of terminal node at
Locally collect, analyze and store the Data highway business relevant with the trunk of each other communication networks and network management
Data.It can collect, analyze the data related to storage user, it is such as relevant with customer service, login attempt, inquiry and instruction
Data.Can collect, analyze and stored text data (for example, daily record, operation sequence, handbook etc.), spatial data (for example,
Location-based data) and multi-medium data (for example, closed circuit TV, video clipping etc.).
In said system, terminal node can be the operating main body that different types of big data is directed in different field,
Can be the mechanism of people or such as electronic equipment etc, the mechanism is to contain processor, memory, bus, power supply
The device of the basic handling function such as circuit, it is preferable that the mechanism can also have such as keyboard, keypad, touch-screen as needed
Etc input equipment, can also have such as graphic user interface etc display device.Different field include it is existing and
The various fields developed later, it might even be possible to while including multiple fields or crossing domain.Definition to data depends on user
Requirement.The mode for obtaining data is arbitrary, can use various modes that are existing and developing later.Similarly, integrate and/
Or check that the mode of data is also any.The mode tested result, verified, disposed and updated is also arbitrary, can be used
Various modes that are existing and developing later.
Whole system can be divided into authentication, data file block encryption, digital protection, the encryption of data file upload,
Decryption is downloaded, data file encrypted query, and the distributed data files system process performed on backstage, abnormality detection are soft
Part.
User is the owner of data file, and they are distributed in different geographical position, and they are their all encryptions
Data are sent to connecting node, and encryption data is synchronously stored in data source nodes by connecting node again.
Various types of data can be managed independently and each terminal in the network system of renewal realizing user data
Joint position automatically drawing is collected and locally saves as big data.For example, dynamic automatically can be collected and stored at terminal node j
Measurement result and control data.The example of dynamic measurement results and control data can include the change in assignment procedure operation
Data, assigned operation parameter (such as, the note of set point, process and hardware alarm and event (such as, download and communication failure)
Record etc.) in change data.In any embodiment in these embodiments, all types of measurement data and control data
Big data is automatically captured as in a device.In addition, when a change is detected or when controller or other entities are initially
When being added into big data network, static data, such as controller configuration, batch recipe, police can be acquiescently transmitted automatically
Report and event.
According to an aspect of the present invention, the key used in data encryption process is sent to number by data File owner
According to file-sharing person, so that share of data file person uses when accessing data.
In addition, in some scenes, when detecting the change in metadata, realize user data manage independently with more
Capture describes or identified at least some of static metadata in dynamic control and measurement data in new network system.For example, such as
Fruit is changed in user to data, then data source nodes can be by the associated metadata of connecting node automatic capture more
Newly.In some cases, acquiescently automatic capture comes from external system or external source (for example, gold with being used to buffer in a device
Melt system, public service, communication common carrier etc.) data the associated parameter of special module.Additionally or alternatively, Ke Yi
Automatic capture Monitoring Data and/or other types of monitoring data in equipment.
According to an aspect of the present invention, encryption data is stored in data source nodes by data File owner or connection saves
Point, stored using improved distributed data collection.Distributed big data is using a kind of efficiently based on distributed memory
Abstract data object:We are referred to as improved distributed data collection.From the perspective of user, improved distributed data collection exists
It can be regarded as an array in form.The difference of it and common array is the data in improved distributed data collection object
It is physically to be stored in a manner of subregion (Partition), the data of different subregions can be distributed on different machines,
Handled in bottom by parallel computation.Put it briefly, core of the improved distributed data collection as big data processing computation schema
Abstraction interface, realize following function:
1) improved distributed data collection is subregion on cluster, immutable data acquisition system.Improved distributed data
Collection can only generate from file system or internal memory, or pass through " conversion " on existing improved distributed data collection
(transformation) operation such as map, flatMap produce, and by " action " (action) as count, collect,
Save etc., the actual triggering calculating process of improved distributed data collection of big data processing computation schema simultaneously return to a result,
Or the storage of improved distributed data collection into storage system.
2) improved distributed data collection can control its persistent storage rank.User can explicitly specify improved point
The persistent storage level of cloth data set.
3) conversion of improved distributed data collection and motion action are all coarsenesses.Improved distributed data collection
One operation can be applied in all data, and is not only on fraction data set.
4) Lazy computation (Lazy Computing), conversion (transformation) behaviour of improved distributed data collection
All it is lazy evaluation, only trigger action (action) operates, and big data processing computation schema just hold by the real distribution of meeting
Row calculates.
Improved distributed data collection is the core of big data processing computation schema programming, and all parallelizations of user calculate
Operation is expressed by the operate interface of improved distributed data collection.By using improved distributed data collection
As programming core, big data handle computation schema in technical elements in the following way:
1) versatility:The various computings defined around improved distributed data collection are MapReduce supersets, can be with complete
All computings that can be done into MapReduce.
2) internal memory calculates:Improved distributed data collection can make full use of while data distribution locality is taken into account
Cluster memory, by the way that frequently-used data set cache in internal memory, to be reached to the purpose for accelerating complicated iterative type and interactive to calculate；Phase
For Hadoop, big data processing computation schema can often reach to the speed-up ratio of complicated Class of Iterative and interactive calculating task
One to two orders of magnitude.
3) Thread-Level Parallelism:So that task scheduling delay is able to be down to submicrosecond level, be Spark Streaming it is such with
Stream calculation based on micro- batch processing (Micro Batching) is had laid a good foundation.
4) DAG calculates flow graph optimization:Similar with the DAG computing systems such as Dryad, improved distributed data collection has rich
The DAG that rich computing collection can easily express complexity is calculated, it is no longer necessary to as MapReduce is each step operation scheduling one
Individual single operation.Flow graph optimization is calculated along with being aided with operation inside each stage, even if not enabling memory cache, is held
Line efficiency is often also several times as much as Hadoop.
5) based on the fault-tolerant of pedigree:The immutableness that improved distributed data integrates allows to using data partition as granularity
Follow the trail of the history of data.When the node in cluster delays machine, the responsible improved distributed data collection of malfunctioning node need to be only followed the trail of
The pedigree of subregion, the subregion of loss can be recalculated, and whole error recovery procedure (ERP) can perform parallel.Data are towering remaining
Acceleration is only served in data recovery procedure.
6) data sharing is abstracted:Improved distributed data collection preferably solves each link in big data analysis process
Data sharing problem, avoid frequently distributed file system I/O operation.
7) more computation schemas are supported:Because bottom frame provides preferable versatility and efficiency guarantee, it is able on upper strata
Realize simultaneously and batch processing is provided, stream process, data query, iterative type calculate, internal memory calculates and scheme a variety of calculating such as calculating
Pattern.And realize that the component of each computation schema need to only focus on respective problem domain, solved without repetition in bottom frame
The common problem such as distributed, fault-tolerant, data sharing, it is achieved thereby that integrated big data processing function.
In certain embodiments, terminal node is referred to as " data user interface node ", " big number interchangeably herein
According to user interface facilities ", " user interface node " or " user interface facilities ".Equipment includes each, and there is integrated form user to connect
The one or more nodes or equipment of mouth, user or operator can hand over via the integrated form user interface and Process Control System
Mutually with perform relevant activity (for example, configure, check, monitor, test, diagnose, sort, plan, dispatch, annotate, and/or its
Its activity).The example of these user interface nodes or equipment includes mobile or static computing device, work station, hand-held and set
Standby, tablet device, surface computing device and any other calculating with processor, memory and integrated form user interface
Equipment.Integrated user interface can include screen, keyboard, keypad, mouse, button, touch-screen, touchpad, biometric
Interface, loudspeaker and microphone, camera, and/or any other user interface techniques.Each user interface facilities can include
The integrated user interface of one or more.User interface node or equipment can include to realize user data manage independently with more
New network system is directly connected to or can included for example via access point or gateway to realizing that user data manages independently
With being indirectly connected with for the network system of renewal.User interface facilities in a wired fashion and/or wirelessly can communicatedly connect
It is connected to the network system for realizing that user data is managed independently with renewal.In certain embodiments, user interface facilities can be with each
Kind is communicatively connected to realize the network system that user data is managed independently with renewal.
In cloud era, Hadoop can not only utilize its distributed data files system as a distributed Open Source Platform
Storage environments of the system HDFS as big data (Big Data), but also support the distributed volumes of MapReduce that Google is proposed
Journey mode, nowadays it has been widely used in distributed and Distributed Computing Platform.But by the use of Hadoop as greatly
The storage environment of data (Big Data), the confidentiality of data, integrality and data access control (DAC) be equally worth research and
Thinking.
The defects of in order to overcome Job execution performance, can be in the data source nodes of the present invention using a kind of improved big
Data processing computing system, the improved big data processing computing system compatibility MapReduce frameworks, at improved big data
The whole execution flow of reason computing system operation can chronologically be divided into preparation, operation and complete three phases.When an operation
Be submitted to data source nodes (operation can be for example, read-only (R), only write (W) and readable writeable (RW) operation), it is follow-up
It is as follows to perform flow:
1) preparatory stage:One operation performs since START condition, can enter prepare .INITIALIZING shapes first
State simultaneously completes some initial works, including from HDFS read input data burst information and generate the Map of corresponding number
With Reduce tasks.Then, entitled Setup Task special duty will be scheduled to a TaskTracker first and be held
Go to set the performing environment of whole operation.At this moment, the Job execution state, which turns into, prepares .SETUP.As the SetupTask
After successful execution terminates, whole operation will enter the operation phase.
2) operation phase:In this stage, operation since being run .RUN_WAIT states, its task wait for by
The scheduling of MapReduce frameworks performs.When there is a task to be scheduled for TaskTracker execution in operation, whole operation
State will be switched to operation operations _ TASKS.In a state, all Map/Reduce tasks will be all dispatched to successively
Performed on TaskTracker.After once all Map/Reduce tasks carryings are completed, whole operation will enter operation .SUC_
WAIT states, operation phase also reach coda.
3) stage is completed:In this stage, another special duty for being named as Cleanup Task will be scheduled for one
TaskTracker is performed, to clear up the running environment of the MapReduce operations., should after this Cleanup Task is completed
Operation is up to SUCCEEDED states, and whole operation also completes with regard to successful execution.
In any one state prepared and in the operation phase, operation can be terminated by the user hence into KILLED shapes
State, or failed always hence into FAILED states because some operations perform.
According to the proposed by the invention improved big data processing computing system, the scheduling of operation and execution it is whole
Handling process can be divided into following steps：
1) when task creation, task tracker can be that each task generates a TaskInProcess example.This is former
Business is still in unallocated state.
2) each job trace device performs task by sending heartbeat message to the application of task tracker.Responded as heartbeat
Information, task tracker can be that each job trace device distributes one or more tasks.The dispatching distribution of task is to pass through two-wheeled
Heartbeat communication is completed, and is often taken turns the time interval that heartbeat is sent and is defaulted as 3 seconds.
3) after a task is received, job trace device can proceed as follows:One is created first
TaskTracker.TaskInProgress examples, an independent Child JVM is then run to perform the task, and will make
The execution state of the industry tracker task is run instead.
4) status information of task is reported to task tracker by each job trace device, and then task tracker is by task
State be updated to run.This process needs to complete by other one wheel heartbeat communication.
5) after running after a while, task performs completion in Child JVM environment.Then, operation by this
The state of business makes COMMIT_PENDING into.Task will wait the permission from task tracker in this state, to submit
(commit) task.
6) change information of this task status will also be delivered to task tracker by next round heartbeat.As response,
The task status that oneself is safeguarded also is updated to COMMIT_PENDING by task tracker, and allows job trace device to submit
(commit) result of task.
7) after the submission for receiving task tracker is permitted, job trace device submits the implementing result of task, then
The state of task is updated to SUCCEEDED.
Hereafter, 8) job trace device is communicated by next round heartbeat is changed to task status SUCCEEDED message hair
Deliver to task tracker.Then, the status information for the task that oneself is safeguarded can be also labeled as by task tracker
SUCCEEDED.So far, the execution flow of a task just finishes.
Above-mentioned steps can be according to adjusting execution sequence before and after being actually needed, or deletes execution step, rather than must press
According to described tandem.
Based on the key management of share of data file group, in distributed data files system, data File owner has
The necessary key to shared data file is periodically changed, the purpose for the arrangement is that can ensure user addition/from
Ensure the safeguard protection of data when opening.The key includes the key that symmetric cryptography key key, public key encryption key etc. are used.
Current way is similar to and uses key distribution center KDC methods, allows control node to carry out the negotiation of key, makes number
Some symmetric key is all shared according to the owner of file and the sharer of data file, and with the encrypted data file,
Encryption key carries out the transmission of key using the mode of the public key encryption of user, but this method cannot be guaranteed sharer add or
The safeguard protection of key after shared group is left, it is therefore necessary to key is regularly changed.
Data File owner to key management can based on share of data file group key change, it is relatively effective
It is to utilize key tree construction to store key method, such as uses logic key hierarchical method LKH, and being safeguarded by data File owner should
Key tree, and after it have changed key, change information is reported to public key server.And merely using LKH storage key management
The storage overhead that method group controller GC when key is changed stores key is linear substantially with membership, and key is repaiied
Change the communication overhead brought with membership into logarithm proportionate relationship, but in distributed data platform, the frequency of mass users
Numerous addition or key modification expense is also very huge caused by exiting group, so the present invention combines Chebyshev multinomials
Periodicity and LKH key trees key management characteristic, it is proposed that one kind based on the polynomial periodicity keys of Chebyshev more
New method CKPS (Cyclic Key Update Scheme).The characteristics of this method is the modification and transmission of key, without
Key, directly by the multicast message of the owner, the direct local computing of sharer, calculating performance is improved, while can be effective
Traffic during key modification is reduced, and reduces the key storage of each node (including root node).In this way
Afterwards, the key that the effect of the key of the intermediate node storage in LKH logics key tree is only intended in key subtree exchanges.It is worth
Illustrating, this method is relatively adapted to the transmission of session key, is not particularly suitable for for data file this kind of " static resource ",
Because after key modification, the data file after encryption needs re-encrypted, but the action, can offline or delay progress.
CKPS methods are described in detail as follows:
(A) initial phase
● data File owner's initial construction Chebyshev multinomials Tn(x) (mod N), and determination triple (x, N,
T) value, t ∈ { 0,1,2,3 ... } are the timestamps for periodically changing multicast key, and N is natural number, and x is real number.
● data File owner is according to the file-sharing group membership U itself storediKeyIt is encryptedIt is sent to file-sharing group membership Ui。
● file-sharing group membership is according to receivingDecryption obtains Tn(x) (mod N) is more
Item formula and triple (x, N, t) and timestamp, group membership can calculate the polynomial value C of Chebyshevt=Tt(x)(mod
N),CtAs current multicast key, original state t=0.
(B) the periodic modification stage
The periodic modification stage is " freshness " in order to ensure multicast key within a period of time, can so ensure to share
The safeguard protection of data.
Data File owner carries out regularly key and changed, and can generate triple at random, and the triple of encryption is sent out
Give each file-sharing member node.
Group membership goes out current multicast key C by the Chebyshev polynomial computations of triple and storaget。
(C) file-sharing member adds or left the stage
File-sharing member adds, and the nearest idle leaf node of data File owner's chosen distance tree root adds composition
Member；File-sharing member leaves, and data File owner deletes file-sharing member node and corresponding redundancy intermediate node.
Data File owner generates triple at random, and the triple of encryption, which is sent to each file-sharing member, to be saved
Point.
File-sharing group membership goes out current multicast key by the Chebyshev polynomial computations of triple and storage
Ct。
CKPS methods can obtain the shared of share of data file group faster by the polynomial computation in internal memory
Key, the communication overhead brought and encryption and decryption operation are exchanged so as to avoid more secondary keys, and Key Exposure can be prevented,
There is obvious advantage in distributed data platform.
According to an aspect of the present invention, data processing function module library can be established, for data encryption upload and
Decryption is downloaded：Language (such as SQL, Scala, Java and R etc.) can perform by data processing using distributed big data system
Basic function and algorithm packaging are into each function element module in each flow, and give each Functional Unit module assignment one exclusive
Chinese；Function element module is stored to corresponding sub-function module storehouse；A data processing function element module storehouse is established,
Function element module is stored by the way of object storage.According to the handling process of data, by data processing function member mould
Block storehouse is divided into several sub-function modules such as data acquisition, importing, conversion, cleaning, fusion, analysis, excavation and machine learning
Storehouse.
Distributed Storage is for data source:There is the difference of isomorphism and isomery, theirs can be unified with xml modes
Form.For handling distributed data collection, following three kinds of processing modes can be used:
1) central host is arrived by each data source data is all centrally stored, then carry out data sharing.Advantage is can be straight
Connect using the uniprocessor algorithm in data mining, shortcoming is not utilize distributed characteristic, the flow of network is surged, while is also added
The operation processing burden of central host is weighed.
2) each host of data sources is first handled respective data set to obtain pattern or rule, then again by these moulds
Formula or rule, are aggregated into central host, and processing obtains global schema or rule.Advantage is to make use of distributed nature, is reduced
Network traffics, shortcoming are that the conclusion drawn is reported by mistake sometimes.
3) above two mode is compromise.Advantage is both to make use of distributed nature, reduces wrong report again.Except distribution
Correlation rule, the distributed sorting algorithm based on decision tree is also applied, distributed and multi-layer correlation rule can also be used, point
Cloth cluster analysis, distributed sequence analysis.
In certain embodiments, end node devices are process controllers, and process control interface is used to be controlled
The configuration (for example, from work station) of device, and/or obtain and be sent to the field apparatus for being connected to controller or set from the scene
The standby data received are with control process in real time.Received data can be stored in controller and/or can be by controlling
Device processed is using to perform at least a portion of control function or control loop.
In another embodiment, end node devices are to provide the I/O being connected between controller and field apparatus and set
It is standby.In this embodiment, process control interface include field device interface with field apparatus exchange process control data, and
Control unit interface with controller exchange process control data.Field device interface is connected to control unit interface, to allow to
Data are sent to controller and receive data from field apparatus via I/O equipment.
Many traditional serialization machine learning algorithms are difficult to complete the processing meter to big data within the acceptable time
Calculate, so as to be worked in practical application scene.Therefore, existing all serialization machine learning algorithms can be carried out simultaneously
Rowization designs.The parallelization thinking of common machine learning algorithm includes data parallel and model parallelization.
The foregoing is only a preferred embodiment of the present invention, but protection scope of the present invention be not limited thereto,
Any one skilled in the art the invention discloses technical scope in, the change or replacement that can readily occur in,
It should all be included within the scope of the present invention.Therefore, protection scope of the present invention should the protection model with claim
Enclose and be defined.
Claims (10)
1. a kind of realize that user data manages the network system with renewal independently, suitable for being managed to the data file of user
With renewal, it is described realize user data manage independently with update network system include：
The terminal node of multiple users, for Various types of data caused by on-site collection business, and stored or updated and arrive data
Source node；
The terminal node of the user is physically under the jurisdiction of different subnets, and the user inside the subnet physically uses
Higher bandwidth connection, realize the high-speed transfer of data；
One or more data source nodes, the Various types of data collected for storing user in terminal node；
Characterized in that, also include：
One or more connecting nodes, it is made up of some terminal node in subnet, or several terminal nodes are contributed respectively
A part of memory space and set up " shared node "；
The connecting node is connected with data source nodes, when user to user data file is operated, it is only necessary to place
The data file in connecting node inside the identical subnet is operated, and just completes the data to being stored in data source nodes
The operation of file；
The stable operation of system is realized by file system background process, and by abnormality detecting program, to various abnormal conditions
Handled.
2. as claimed in claim 1 realize that user data manages the network system with renewal independently, it is characterised in that：
Data file includes data file head, and data file head is subdivided into following components：Data file ID, creation time
With the finger print information part with data File owner's private key encryption.
3. as claimed in claim 1 realize that user data manages the network system with renewal independently, it is characterised in that：
The business is the related business of the related business of real-time process, factory, financial business, game service and other
The business of type.
4. the shared computer network system of distributed big data real-time exchange as claimed in claim 1, it is characterised in that：
Stored using improved distributed data set pair data file, the data text in improved distributed data collection object
Part is stored in a manner of subregion (Partition), and the data of different subregions can be distributed in different machines
On, handled in bottom by parallel computation.
5. the shared computer network system of distributed big data real-time exchange as claimed in claim 4, it is characterised in that：
Data File owner is that all share of data file person members share key.
6. a kind of be suitable to as claimed in claim 1 realizing that user data managed independently with running in the network system of renewal
Method, the encryption for realizing data update and shared, it is characterised in that comprise the following steps：
The data file of encryption is stored in data source nodes by data File owner by the way of data encryption upload；
Data File owner specifies the users to share data file；
When data File owner needs to update the data file of itself, data file is Resealed, using in data encryption
The data file of encryption is stored in data source nodes by the mode of biography.
7. as claimed in claim 6 realize user data manage independently with the method run in the network system of renewal, its
It is characterised by：
The data file is the data of the change in dynamic measurement results or control data, including assignment procedure operation, specified
The data changed in operating parameter；
The assigned operation parameter is the data in set point, process and hardware alarm and event.
8. as claimed in claim 6 realize user data manage independently with the method run in the network system of renewal, its
It is characterised by：
In the mode that data encryption in step sl uploads, data access control (DAC), profit are carried out to the data file of encryption
The protection, data sharing and completeness check of data are carried out with public-key cryptosystem and symmetric cryptosystem.
9. as claimed in claim 8 realize user data manage independently with the method run in the network system of renewal, its
It is characterised by, in addition to：
Share of data file person decrypts downloading data by the way of data deciphering download.
10. as claimed in claim 9 realize user data manage independently with the method run in the network system of renewal, its
It is characterised by, in addition to：
After share of data file person updates the data file, the private key of data File owner is obtained, with re-encrypted finger print information
Part, so as to which the data file of encryption is stored in data source nodes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711011779.7A CN107770276A (en) | 2017-10-26 | 2017-10-26 | It is a kind of to realize that user data manages the network system and method with renewal independently |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711011779.7A CN107770276A (en) | 2017-10-26 | 2017-10-26 | It is a kind of to realize that user data manages the network system and method with renewal independently |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107770276A true CN107770276A (en) | 2018-03-06 |
Family
ID=61270011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711011779.7A Pending CN107770276A (en) | 2017-10-26 | 2017-10-26 | It is a kind of to realize that user data manages the network system and method with renewal independently |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107770276A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108536795A (en) * | 2018-04-02 | 2018-09-14 | 深圳市华傲数据技术有限公司 | A kind of method, medium and equipment improving diagram data interactive efficiency |
| CN110414245A (en) * | 2018-04-28 | 2019-11-05 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for managing encrypted key within the storage system |
| CN112035421A (en) * | 2020-11-02 | 2020-12-04 | 杭州优云科技有限公司 | IDC scheduling optimization system based on data accumulation |
| CN113168478A (en) * | 2018-07-10 | 2021-07-23 | 柯拉松简化股份公司 | Scalable server architecture providing access to data content |
| CN114444986A (en) * | 2022-04-11 | 2022-05-06 | 成都数之联科技股份有限公司 | Product analysis method, system, device and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188569A (en) * | 2006-11-16 | 2008-05-28 | 饶大平 | Method for constructing data quanta space in network and distributed file storage system |
| CN102685148A (en) * | 2012-05-31 | 2012-09-19 | 清华大学 | Method for realizing secure network backup system under cloud storage environment |
| US9275059B1 (en) * | 2011-11-07 | 2016-03-01 | Emc Corporation | Genome big data indexing |
| CN105450750A (en) * | 2015-12-01 | 2016-03-30 | 成都汇合乾元科技有限公司 | Secure interaction method for intelligent terminal |
| CN106528717A (en) * | 2016-10-26 | 2017-03-22 | 中国电子产品可靠性与环境试验研究所 | Data processing method and system |
-
2017
- 2017-10-26 CN CN201711011779.7A patent/CN107770276A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188569A (en) * | 2006-11-16 | 2008-05-28 | 饶大平 | Method for constructing data quanta space in network and distributed file storage system |
| US9275059B1 (en) * | 2011-11-07 | 2016-03-01 | Emc Corporation | Genome big data indexing |
| CN102685148A (en) * | 2012-05-31 | 2012-09-19 | 清华大学 | Method for realizing secure network backup system under cloud storage environment |
| CN105450750A (en) * | 2015-12-01 | 2016-03-30 | 成都汇合乾元科技有限公司 | Secure interaction method for intelligent terminal |
| CN106528717A (en) * | 2016-10-26 | 2017-03-22 | 中国电子产品可靠性与环境试验研究所 | Data processing method and system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108536795A (en) * | 2018-04-02 | 2018-09-14 | 深圳市华傲数据技术有限公司 | A kind of method, medium and equipment improving diagram data interactive efficiency |
| CN110414245A (en) * | 2018-04-28 | 2019-11-05 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for managing encrypted key within the storage system |
| CN110414245B (en) * | 2018-04-28 | 2023-09-22 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for managing encryption keys in a storage system |
| CN113168478A (en) * | 2018-07-10 | 2021-07-23 | 柯拉松简化股份公司 | Scalable server architecture providing access to data content |
| CN112035421A (en) * | 2020-11-02 | 2020-12-04 | 杭州优云科技有限公司 | IDC scheduling optimization system based on data accumulation |
| CN114444986A (en) * | 2022-04-11 | 2022-05-06 | 成都数之联科技股份有限公司 | Product analysis method, system, device and medium |
| CN114444986B (en) * | 2022-04-11 | 2022-06-03 | 成都数之联科技股份有限公司 | Product analysis method, system, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107800787B (en) | Distributed big data real-time exchange sharing computer network system | |
| Zhang et al. | Federated learning for the internet of things: Applications, challenges, and opportunities | |
| CN107770276A (en) | It is a kind of to realize that user data manages the network system and method with renewal independently | |
| Li et al. | Federated learning: Challenges, methods, and future directions | |
| Niknam et al. | Federated learning for wireless communications: Motivation, opportunities, and challenges | |
| Elmisery et al. | A new computing environment for collective privacy protection from constrained healthcare devices to IoT cloud services | |
| US20230039182A1 (en) | Method, apparatus, computer device, storage medium, and program product for processing data | |
| CN109729168A (en) | A kind of data share exchange system and method based on block chain | |
| CN107850882B (en) | Automation and control distributed | |
| Zhu et al. | Dynamic audit services for outsourced storages in clouds | |
| CN104365058B (en) | For the system and method in multinuclear and group system high speed caching SNMP data | |
| Burkhalter et al. | {TimeCrypt}: Encrypted data stream processing at scale with cryptographic access control | |
| CA3093718C (en) | Method, apparatus, and computer program product for encryption key management within a group-based communication system | |
| CN110543464A (en) | Big data platform applied to smart park and operation method | |
| CN107771320A (en) | For improving security socket layer（SSL）The system and method for communications security | |
| Song et al. | Networking systems of AI: On the convergence of computing and communications | |
| US10185607B1 (en) | Data statement monitoring and control | |
| Rahman et al. | EduChain: CIA-compliant blockchain for intelligent cyber defense of microservices in education industry 4.0 | |
| CN114153920A (en) | Big data edge platform and method | |
| CN112835977A (en) | Database management method and system based on block chain | |
| CN103957174A (en) | Method for semantic switch loose coupling system to process information | |
| Toyoda et al. | Performance evaluation of ethereum-based on-chain sensor data management platform for industrial IoT | |
| CN103957173A (en) | Semantic switch | |
| Sultanov et al. | Development of a centralized system for data storage and processing on operation modes and reliability indicators of power equipment | |
| CN116208340A (en) | Trusted data flow platform system method based on privacy calculation and blockchain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180306 |