US10516669B2 - Methods and systems relating to network based storage - Google Patents
Methods and systems relating to network based storage Download PDFInfo
- Publication number
- US10516669B2 US10516669B2 US15/346,094 US201615346094A US10516669B2 US 10516669 B2 US10516669 B2 US 10516669B2 US 201615346094 A US201615346094 A US 201615346094A US 10516669 B2 US10516669 B2 US 10516669B2
- Authority
- US
- United States
- Prior art keywords
- data
- user
- cloud storage
- data sources
- sources
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000003860 storage Methods 0.000 title claims abstract description 102
- 238000000034 method Methods 0.000 title claims description 101
- 230000015654 memory Effects 0.000 claims description 31
- 238000004891 communication Methods 0.000 claims description 17
- 230000037406 food intake Effects 0.000 claims description 11
- 230000008859 change Effects 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims 1
- 230000008569 process Effects 0.000 description 40
- 238000010586 diagram Methods 0.000 description 15
- 230000009471 action Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 230000001413 cellular effect Effects 0.000 description 8
- 230000009286 beneficial effect Effects 0.000 description 7
- 238000013500 data storage Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 208000016169 Fish-eye disease Diseases 0.000 description 2
- 241001112258 Moca Species 0.000 description 2
- 230000002730 additional effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000001650 pulsed electrochemical detection Methods 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000003466 anti-cipated effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000003339 best practice Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000013479 data entry Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- PQYUGUXEJHLOIL-UHFFFAOYSA-N diethoxysilyl triethyl silicate Chemical compound C(C)O[SiH](O[Si](OCC)(OCC)OCC)OCC PQYUGUXEJHLOIL-UHFFFAOYSA-N 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000012377 drug delivery Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010339 medical test Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000926 neurological effect Effects 0.000 description 1
- 229920001690 polydopamine Polymers 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 235000014347 soups Nutrition 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 239000004753 textile Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0605—Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
- G06F3/0622—Securing storage systems in relation to access
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0637—Permissions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Definitions
- This invention relates to network and/or cloud based storage and more particularly to methods of writing data to network and/or cloud storage and of providing efficient human interfaces for knowledge workers within network and/or cloud storage solutions.
- Cloud storage has evolved in the last decade from a model being presented as a solution to evolving data storage needs to the main storage form for many enterprises, organizations and individuals.
- a single social network, FacebookTM alone stored approximately 400 Petabytes of data.
- Cloud storage represents a data storage model where data is stored in logical pools, the physical storage spans multiple servers and often locations, and the physical environment is typically owned and managed by a hosting company and/or service provider. These cloud storage providers are responsible for keeping the data available and accessible, and the physical environment protected and running. People and organizations buy or lease storage capacity from the providers to store user, organization, or application data.
- Cloud storage services may be accessed through a co-located cloud computer service, a web service application programming interface (API) or by applications that utilize the API, such as cloud desktop storage, a cloud storage gateway or Web-based content management systems.
- API application programming interface
- cloud storage is based on a highly virtualized infrastructure and is like the broader concept of cloud computing in terms of accessible interfaces, near-instant elasticity and scalability, multi-tenancy, and metered resources.
- Cloud storage a form of network based storage, is made up of many distributed resources, but still acts as one (often referred to as federated storage clouds), is highly fault tolerant through redundancy and distribution of data, highly durable through the creation of versioned copies, and generally what is known as “eventually consistent” with regard to data replicas.
- cloud storage also comes with some drawbacks and limitations in how this information is uploaded (or ingested) and how a user or users access this information subsequently in comparison to the management tools etc. that the user is typically used to using.
- tools such as Microsoft One Drive offer individual users functionality similar to Microsoft Explorer for managing files and integrate to software applications such as Microsoft's own Word, Excel and PowerPoint there is a lack of automated tools for managing tens, hundreds and thousands of users within enterprises and organizations. Migrating to the cloud for these is a massive undertaking.
- knowledge workers e.g. users
- a human interface e.g. a graphical user interface on their electronic devices
- a method or computer readable instructions stored upon a non-volatile non-transitory storage medium which when executed by a microprocessor perform the method, the method comprising linking a cloud storage repository to a user directory service and maintaining a record of user identities, wherein when a data source connector traverses and captures data from a third-party system it submits access control information to the cloud storage repository it at least one of resolves all access control members to user identities and stores this information relationally within the record of user identities and updates the user directory service.
- a graphical user interface comprising computer readable instructions stored upon a non-volatile non-transitory storage medium which when executed by a microprocessor perform a method, the method comprising:
- a first module for resolving access control information to knowledge worker identities with respect to a plurality of knowledge workers accessing data stored upon a remote storage system
- a second module for generating and assembling data map reports for each knowledge worker of the plurality of knowledge workers
- a fourth module synchronizing changes and updating the data map reports as changes are captured from third-party systems.
- a computer-implemented method for centrally managing delta comparisons and ingestion logic with respect to a data storage network comprising the steps of:
- a data source connector associated with a repository of data, the data source connector:
- the data storage network supports at least one of directly and indirectly through other cloud storage repositories all of the policies and other capture logic, the user interface(s), and associated system(s) of record.
- each policy block represents logical groupings of item versions from the policy's data set and is stored as a separate record in a policy block table.
- FIG. 1 depicts a network environment within which embodiments of the invention may be employed
- FIG. 2 depicts a wireless portable electronic device supporting communications to a network such as depicted in FIG. 1 and as supporting embodiments of the invention
- FIGS. 3 and 4 depict flow diagrams in respect of providing a knowledge worker with a cloud view of their entitlements from third party data sources according to an embodiment of the invention
- FIG. 5 depicts a flow diagram for incremental synchronised connector writing to a cloud data repository according to an embodiment of the invention
- FIG. 6 depicts a flow diagram for initial policy creation and execution according to an embodiment of the invention
- FIG. 7 depicts a flow diagram for incremental policy execution and update of statistics according to an embodiment of the invention.
- FIG. 8 depicts schematically a process for writing to cloud storage without exposing cloud storage API security according to an embodiment of the invention.
- FIG. 9 depicts a flow diagram for writing to cloud storage without exposing cloud storage API security as depicted by the process in FIG. 8 according to an embodiment of the invention.
- the present invention is directed to network and/or cloud based storage and more particularly to methods of writing data to network and/or cloud storage and of providing efficient human interfaces for knowledge workers within network and/or cloud storage solutions.
- a “portable electronic device” refers to a wireless device used for communications and other applications that requires a battery or other independent form of energy for power. This includes devices, but is not limited to, such as a cellular telephone, smartphone, personal digital assistant (PDA), portable computer, pager, portable multimedia player, portable gaming console, laptop computer, tablet computer, and an electronic reader.
- PDA personal digital assistant
- a “fixed electronic device” refers to a wireless and/or wired device used for communications and other applications that requires connection to a fixed interface to obtain power. This includes, but is not limited to, a laptop computer, a personal computer, a computer server, a kiosk, a gaming console, a digital set-top box, an analog set-top box, an Internet enabled appliance, an Internet enabled television, and a multimedia player.
- An “application” (commonly referred to as an “app”) as used herein may refer to, but is not limited to, a “software application”, an element of a “software suite”, a computer program designed to allow an individual to perform an activity, a computer program designed to allow an electronic device to perform an activity, and a computer program designed to communicate with local and/or remote electronic devices.
- An application thus differs from an operating system (which runs a computer), a utility (which performs maintenance or general-purpose chores), and a programming tools (with which computer programs are created).
- an application is generally presented in respect of software permanently and/or temporarily installed upon a PED and/or FED.
- a “social network” or “social networking service” as used herein may refer to, but is not limited to, a platform to build social networks or social relations among people who may, for example, share interests, activities, backgrounds, or real-life connections. This includes, but is not limited to, social networks such as U.S. based services such as Facebook, Google+, Tumblr and Twitter; as well as Nexopia, Badoo, Bebo, VKontakte, Delphi, Hi5, Hyves, iWiW, Nasza-Klasa, Soup, Glocals, Skyrock, The Sphere, StudiVZ, Tagged, Tuenti, XING, Orkut, Mxit, Cyworld, Mixi, renren, weibo and Wretch.
- U.S. based services such as Facebook, Google+, Tumblr and Twitter
- Nexopia Badoo, Bebo, VKontakte, Delphi, Hi5, Hyves, iWiW, Nasza-Klasa, Soup,
- Social media or “social media services” as used herein may refer to, but is not limited to, a means of interaction among people in which they create, share, and/or exchange information and ideas in virtual communities and networks. This includes, but is not limited to, social media services relating to magazines, Internet forums, weblogs, social blogs, microblogging, wikis, social networks, podcasts, photographs or pictures, video, rating and social bookmarking as well as those exploiting blogging, picture-sharing, video logs, wall-posting, music-sharing, crowdsourcing and voice over IP, to name a few.
- Social media services may be classified, for example, as collaborative projects (for example, Wikipedia); blogs and microblogs (for example, TwitterTM); content communities (for example, YouTube and DailyMotion); social networking sites (for example, FacebookTM); virtual game-worlds (e.g., World of WarcraftTM); and virtual social worlds (e.g. Second LifeTM).
- collaborative projects for example, Wikipedia
- blogs and microblogs for example, TwitterTM
- content communities for example, YouTube and DailyMotion
- social networking sites for example, FacebookTM
- virtual game-worlds e.g., World of WarcraftTM
- virtual social worlds e.g. Second LifeTM
- An “enterprise” as used herein may refer to, but is not limited to, a provider of a service and/or a product to a user, customer, or consumer. This includes, but is not limited to, a retail outlet, a store, a market, an online marketplace, a manufacturer, an online retailer, a charity, a utility, and a service provider. Such enterprises may be directly owned and controlled by a company or may be owned and operated by a franchisee under the direction and management of a franchiser.
- a “service provider” as used herein may refer to, but is not limited to, a third party provider of a service and/or a product to an enterprise and/or individual and/or group of individuals and/or a device comprising a microprocessor. This includes, but is not limited to, a retail outlet, a store, a market, an online marketplace, a manufacturer, an online retailer, a utility, an own brand provider, and a service provider wherein the service and/or product is at least one of marketed, sold, offered, and distributed by the enterprise solely or in addition to the service provider.
- a ‘third party’ or “third party provider” as used herein may refer to, but is not limited to, a so-called “arm's length” provider of a service and/or a product to an enterprise and/or individual and/or group of individuals and/or a device comprising a microprocessor wherein the consumer and/or customer engages the third party but the actual service and/or product that they are interested in and/or purchase and/or receive is provided through an enterprise and/or service provider.
- a “user” may refer to, but is not limited to, an individual or group of individuals who may, but not limited to, monitor, acquire, store, transmit, process and analyse either locally or remotely to the user data within one or more databases. This includes, but is not limited to, private individuals, employees of organizations and/or enterprises, members of community organizations, members of charity organizations, men, women, children, and teenagers. In its broadest sense the user may further include, but not be limited to, software systems, mechanical systems, robotic systems, android systems, etc. that may, but not limited to, monitor, acquire, store, transmitt, process and analyse either locally or remotely to the user data within one or more databases.
- a “wearable device” or “wearable sensor” relates to miniature electronic devices that are worn by the user including those under, within, with or on top of clothing and are part of a broader general class of wearable technology which includes “wearable computers” which in contrast are directed to general or special purpose information technologies and media development.
- Such wearable devices and/or wearable sensors may include, but not be limited to, smartphones, smart watches, e-textiles, smart shirts, activity trackers, smart glasses, environmental sensors, medical sensors, biological sensors, physiological sensors, chemical sensors, ambient environment sensors, position sensors, neurological sensors, drug delivery systems, medical testing and diagnosis devices, and motion sensors.
- Electronic content (also referred to as “content” or “digital content”) as used herein may refer to, but is not limited to, any type of content that exists in the form of digital data as stored, transmitted, received and/or converted wherein one or more of these steps may be analog although generally these steps will be digital.
- Digital content include, but are not limited to, information that is digitally broadcast, streamed or contained in discrete files.
- types of digital content include popular media types such as MP3, JPG, AVI, TIFF, AAC, TXT, RTF, HTML, XHTML, PDF, XLS, SVG, WMA, MP4, FLV, and PPT, for example, as well as others, see for example hap://en.wikipedia.org/wiki/List_of_file_formats.
- digital content may include any type of digital information, e.g. digitally updated weather forecast, a GPS map, an eBook, a photograph, a video, a VineTM, a blog posting, a FacebookTM posting, a TwitterTM tweet, online TV, etc.
- the digital content may be any digital data that is at least one of generated, selected, created, modified, and transmitted in response to a user request, said request may be a query, a search, a trigger, an alarm, and a message for example.
- content information may refer to, but is not limited to, any combination of content features, content serving constraints, information derivable from content features or content serving constraints (referred to as “content derived information”), and/or information related to the content (referred to as “content related information”), as well as an extension of such information (e.g., information derived from content related information).
- a document may be a file, a combination of files, one or more files with embedded links to other files, etc.
- the files may be of any type, such as text, audio, image, video, etc.
- Parts of a document to be rendered to an end user can be thought of as “content” of the document.
- a document may include “structured data” containing both content (words, pictures, etc.) and some indication of the meaning of that content (for example, e-mail fields and associated data, HTML tags and associated data, etc.).
- a common document is a Web page.
- Web pages often include content and may include embedded information (such as meta-information, hyperlinks, etc.) and/or embedded instructions (such as Javascript, etc.).
- a document has a unique, addressable, storage location and can therefore be uniquely identified by this addressable location such as a universal resource locator (URL) for example used as a unique address used to access information on the Internet.
- URL universal resource locator
- references to a “human interface” or “user interface” as used herein may refer to, but is not limited to, any interface presenting to a type of interface that allows users to interact with electronic devices and/or systems directly and/or remotely.
- UI user interface
- GUI graphical user interface
- references to a “cloud” as used herein may refer to, but is not limited to, a model of ubiquitous, convenient, on-demand access to a shared pool of configurable computing resources via a network such as the Internet.
- the “cloud” may refer to an application, platform, and/or infrastructure remotely hosted from the user but accessible through a network.
- the cloud may refer to, but not be limited to, cloud computing, cloud servers, cloud (operating system), and cloud storage.
- a “thick UI”, “thick client” etc. may refer to, but is not limited to, an element, e.g. a UI or computer (client), which typically provides rich functionality independent of a central and/or remote server to which the UI and/or client relates.
- a “thin UI”, “thin client” etc. may refer to, but is not limited to, an element, e.g. a UI or computer (client), which typically has high dependency upon a central and/or remote server to which the UI and/or client relates in order to provide rich functionality.
- Connector may refer to, but is not limited to, a technology solution for connecting application servers and information systems etc. Accordingly, a “Connector” may define a software application compliant with a standard for connecting an application server to an information system or a software application forming part of another application in execution upon a server, client, information system etc. that operates by a defined standard a standard set of system-level contracts, for example between an application server and a resource adapter.
- Such connector standards including, but not limited to, Java Connector Architecture, Java Database Connectivity, and Java EE Connector Architecture.
- FIG. 1 there is depicted a network environment 100 within which embodiments of the invention may be employed supporting remote storage systems, applications, and platforms (RS-SAPs) according to embodiments of the invention.
- RS-SAPs for example supporting multiple channels and dynamic content.
- first and second user groups 100 A and 100 B respectively interface to a telecommunications network 100 .
- a remote central exchange 180 communicates with the remainder of a telecommunication service providers network via the network 100 which may include for example long-haul OC-48/OC-192 backbone elements, an OC-48 wide area network (WAN), a Passive Optical Network, and a Wireless Link.
- WAN wide area network
- Passive Optical Network a Wireless Link
- the central exchange 180 is connected via the network 100 to local, regional, and international exchanges (not shown for clarity) and therein through network 100 to first and second cellular APs 195 A and 195 B respectively which provide Wi-Fi cells for first and second user groups 100 A and 100 B respectively. Also connected to the network 100 are first and second Wi-Fi nodes 110 A and 110 B, the latter of which being coupled to network 100 via router 105 . Second Wi-Fi node 110 B is associated with first and second Enterprises 160 A and 160 B respectively, such as General ElectricTM or MicrosoftTM for example, within which other first and second user groups 100 A and 100 B are disposed.
- Second user group 100 B may also be connected to the network 100 via wired interfaces including, but not limited to, DSL, Dial-Up, DOCSIS, Ethernet, G.hn, ISDN, MoCA, PON, and Power line communication (PLC) which may or may not be routed through a router such as router 105 .
- wired interfaces including, but not limited to, DSL, Dial-Up, DOCSIS, Ethernet, G.hn, ISDN, MoCA, PON, and Power line communication (PLC) which may or may not be routed through a router such as router 105 .
- PLC Power line communication
- first group of users 100 A may employ a variety of PEDs including for example, laptop computer 155 , portable gaming console 135 , tablet computer 140 , smartphone 150 , cellular telephone 145 as well as portable multimedia player 130 .
- second group of users 100 B which may employ a variety of FEDs including for example gaming console 125 , personal computer 115 and wireless/Internet enabled television 120 as well as cable modem 105 .
- First and second cellular APs 195 A and 195 B respectively provide, for example, cellular GSM (Global System for Mobile Communications) telephony services as well as 3G and 4G evolved services with enhanced data transport support.
- GSM Global System for Mobile Communications
- Second cellular AP 195 B provides coverage in the exemplary embodiment to first and second user groups 100 A and 100 B.
- first and second user groups 100 A and 100 B may be geographically disparate and access the network 100 through multiple APs, not shown for clarity, distributed geographically by the network operator or operators.
- First cellular AP 195 A as show provides coverage to first user group 100 A and environment 170 , which comprises second user group 100 B as well as first user group 100 A.
- the first and second user groups 100 A and 100 B may according to their particular communications interfaces communicate to the network 100 through one or more wireless communications standards such as, for example, IEEE 802.11, IEEE 802.15, IEEE 802.16, IEEE 802.20, UMTS, GSM 850, GSM 900, GSM 1800, GSM 1900, GPRS, ITU-R 5.138, ITU-R 5.150, ITU-R 5.280, and IMT-1000.
- GSM services such as telephony and SMS and Wi-Fi/WiMAX data transmission, VOIP and Internet access.
- portable electronic devices within first user group 100 A may form associations either through standards such as IEEE 802.15 and Bluetooth as well in an ad-hoc manner.
- SOCNETS Social Networks
- cloud document service provider 170 A e.g. US Government open data portal Data.gov
- first and second business networks 170 B and 170 C respectively, e.g. LinkedInTM and ViadeoTM
- first to second online gaming communities 170 D and 170 E respectively e.g. Call of DutyTM ghosts and World of WarcraftTM
- first and second servers 190 A and 190 B which together with others, not shown for clarity.
- first and second cloud storage service providers 175 A and 175 B e.g. Microsoft One Drive and Google docs
- residential service provider 175 C e.g. ConEdisonTM
- an online multimedia distributor 175 D e.g. AppleTM
- telecom service provider 175 E e.g. AT&T.
- a user employing one or more RS-SAPs may through their avatar and/or avatar characteristics interact with one or more such providers, enterprises, and third parties.
- First and second servers 190 A and 190 B may host according to embodiments of the inventions multiple services associated with a provider of publishing systems and publishing applications/platforms (RS-SAPs); a provider of a SOCNET or Social Media (SOME) exploiting RS-SAP features; a provider of a SOCNET and/or SOME not exploiting RS-SAP features; a provider of services to PEDS and/or FEDS; a provider of one or more aspects of wired and/or wireless communications; an Enterprise 160 exploiting RS-SAP features; license databases; content databases; image databases; content libraries; customer databases; websites; and software applications for download to or access by FEDs and/or PEDs exploiting and/or hosting RS-SAP features.
- First and second primary content servers 190 A and 190 B may also host for example other Internet services such as a search engine, financial services, third party applications and other Internet based services.
- a user may exploit a PED and/or FED within an Enterprise 160 , for example, and access one of the first or second primary content servers 190 A and 190 B respectively to perform an operation such as accessing/downloading an application which provides RS-SAP features according to embodiments of the invention; execute an application already installed providing RS-SAP features; execute a web based application providing RS-SAP features; or access content.
- a user may undertake such actions or others exploiting embodiments of the invention exploiting a PED or FED within first and second user groups 100 A and 100 B respectively via one of first and second cellular APs 195 A and 195 B respectively and first Wi-Fi nodes 110 A.
- Electronic device 204 may, for example, be a PED and/or FED and may include additional elements above and beyond those described and depicted.
- the protocol architecture is depicted within the electronic device 204 that includes an electronic device 204 , such as a smartphone 155 , an access point (AP) 206 , such as first AP 110 , and one or more network devices 207 , such as communication servers, streaming media servers, and routers for example such as first and second servers 190 A and 190 B respectively.
- AP access point
- network devices 207 such as communication servers, streaming media servers, and routers for example such as first and second servers 190 A and 190 B respectively.
- Network devices 207 may be coupled to AP 206 via any combination of networks, wired, wireless and/or optical communication links such as discussed above in respect of FIG. 1 as well as directly as indicated.
- Network devices 207 are coupled to network 100 and therein Social Networks (SOCNETS) 165 , Also connected to the network 100 are Social Networks (SOCNETS) 165 , cloud document service provider 170 A, e.g. US Government open data portal Data.gov, first and second business networks 170 B and 170 C respectively, e.g. LinkedlnTM and ViadeoTM, first to second online gaming communities 170 D and 170 E respectively, e.g. Call of DutyTM ghosts and World of WarcraftTM.
- SOCNETS Social Networks
- first and second cloud storage service providers 175 A and 175 B e.g. Microsoft One Drive and Google docs
- residential service provider 175 C e.g. ConEdisonTM
- online multimedia distributor 175 D e.g. AppleTM
- telecom service provider 175 E e.g. AT&T.
- the electronic device 204 includes one or more processors 210 and a memory 212 coupled to processor(s) 210 .
- AP 206 also includes one or more processors 211 and a memory 213 coupled to processor(s) 210 .
- a non-exhaustive list of examples for any of processors 210 and 211 includes a central processing unit (CPU), a digital signal processor (DSP), a reduced instruction set computer (RISC), a complex instruction set computer (CISC) and the like.
- any of processors 210 and 211 may be part of application specific integrated circuits (ASICs) or may be a part of application specific standard products (ASSPs).
- ASICs application specific integrated circuits
- ASSPs application specific standard products
- memories 212 and 213 includes any combination of the following semiconductor devices such as registers, latches, ROM, EEPROM, flash memory devices, non-volatile random access memory devices (NVRAM), SDRAM, DRAM, double data rate (DDR) memory devices, SRAM, universal serial bus (USB) removable memory, and the like.
- semiconductor devices such as registers, latches, ROM, EEPROM, flash memory devices, non-volatile random access memory devices (NVRAM), SDRAM, DRAM, double data rate (DDR) memory devices, SRAM, universal serial bus (USB) removable memory, and the like.
- Electronic device 204 may include an audio input element 214 , for example a microphone, and an audio output element 216 , for example, a speaker, coupled to any of processors 210 .
- Electronic device 204 may include a video input element 218 , for example, a video camera or camera, and a video output element 220 , for example an LCD display, coupled to any of processors 210 .
- Electronic device 204 also includes a keyboard 215 and touchpad 217 which may for example be a physical keyboard and touchpad allowing the user to enter content or select functions within one of more applications 222 . Alternatively, the keyboard 215 and touchpad 217 may be predetermined regions of a touch sensitive element forming part of the display within the electronic device 204 .
- the one or more applications 222 that are typically stored in memory 212 and are executable by any combination of processors 210 .
- Electronic device 204 also includes accelerometer 260 providing three-dimensional motion input to the process 210 and GPS 262 which provides geographical location information to processor 210 .
- Protocol stack 224 includes an IEEE 802.11-compatible PHY module 226 that is coupled to one or more Front-End Tx/Rx & Antenna 228 , an IEEE 802.11-compatible MAC module 230 coupled to an IEEE 802.2-compatible LLC module 232 .
- Protocol stack 224 includes a network layer IP module 234 , a transport layer User Datagram Protocol (UDP) module 236 and a transport layer Transmission Control Protocol (TCP) module 238 .
- UDP User Datagram Protocol
- TCP Transmission Control Protocol
- Protocol stack 224 also includes a session layer Real Time Transport Protocol (RTP) module 240 , a Session Announcement Protocol (SAP) module 242 , a Session Initiation Protocol (SIP) module 244 and a Real Time Streaming Protocol (RTSP) module 246 .
- Protocol stack 224 includes a presentation layer media negotiation module 248 , a call control module 250 , one or more audio codecs 252 and one or more video codecs 254 .
- Applications 222 may be able to create maintain and/or terminate communication sessions with any of devices 207 by way of AP 206 . Typically, applications 222 may activate any of the SAP, SIP, RTSP, media negotiation and call control modules for that purpose.
- information may propagate from the SAP, SIP, RTSP, media negotiation and call control modules to PHY module 226 through TCP module 238 , IP module 234 , LLC module 232 and MAC module 230 .
- elements of the electronic device 204 may also be implemented within the AP 206 including but not limited to one or more elements of the protocol stack 224 , including for example an IEEE 802.11-compatible PHY module, an IEEE 802.11-compatible MAC module, and an IEEE 802.2-compatible LLC module 232 .
- the AP 206 may additionally include a network layer IP module, a transport layer User Datagram Protocol (UDP) module and a transport layer Transmission Control Protocol (TCP) module as well as a session layer Real Time Transport Protocol (RTP) module, a Session Announcement Protocol (SAP) module, a Session Initiation Protocol (SIP) module and a Real Time Streaming Protocol (RTSP) module, media negotiation module, and a call control module.
- a network layer IP module a transport layer User Datagram Protocol (UDP) module and a transport layer Transmission Control Protocol (TCP) module
- RTP Real Time Transport Protocol
- SAP Session Announcement Protocol
- SIP Session Initiation Protocol
- RTSP Real Time Streaming Protocol
- Portable and fixed electronic devices represented by electronic device 204 may include one or more additional wireless or wired interfaces in addition to the depicted IEEE 802.11 interface which may be selected from the group comprising IEEE 802.15, IEEE 802.16, IEEE 802.20, UMTS, GSM 850, GSM 900, GSM 1800, GSM 1900, GPRS, ITU-R 5.138, ITU-R 5.150, ITU-R 5.280, IMT-1000, DSL, Dial-Up, DOCSIS, Ethernet, G.hn, ISDN, MoCA, PON, and Power line communication (PLC).
- PLC Power line communication
- FIGS. 1 and 2 and below in respect of FIGS. 3 to 9 are merely illustrative and are not intended to limit the scope of the present invention.
- Such computer systems may be connected to other devices that are not illustrated, including through one or more networks such as the Internet or via the World Wide Web (WWW).
- WWW World Wide Web
- the functionality provided by the illustrated components may in some embodiments be combined in fewer components or distributed in additional components.
- the functionality of some of the illustrated components may not be provided and/or other additional functionality may be available.
- filters and software components are illustrated as being stored in memory while being used, these items or portions of them can be transferred between memory and other storage devices for purposes of memory management and data integrity. Similarly, items illustrated as being present on storage while being used can instead be present in memory and transferred between storage and memory. Alternately, in other embodiments some or all of the software modules may execute in memory on another device.
- a “client” or “server” computing device may comprise any combination of hardware or software that can interact, including computers, network devices, internet appliances, PDAs, wireless phones, pagers, electronic organizers, television-based systems and various other consumer products that include inter-communication capabilities. Accordingly, the present invention may be practiced with other computer system configurations.
- GUI graphical user interface
- Such third-party systems may relate to an organization and/or a plurality of organizations, a service provider and/or service providers, an enterprise and/or plurality of enterprises and/or other third-party databases, data repositories, etc.
- the knowledge worker e.g. user, may access the human interface upon a PED, FED, wearable device or other electronic device.
- a system or software executing a method according to an embodiment of the invention operates by linking to one or multiple user directory services and maintaining a record of user identities.
- a data source connector traverses and captures data from a third-party system it submits access control information to the cloud storage repository, which then resolves all access control members to user identities and stores this information relationally.
- the data source connector may also map the data captured from the third-party system to the user directory service.
- the invention assembles and maintains a data map report for each knowledge worker in the system.
- the knowledge workers view the data map report and manually select any of the folder locations they wish to pin to their human interface for immediate visibility and access.
- future synchronization between the third-party system and the cloud storage repository occurs the knowledge worker will automatically see the latest information for these pinned folder locations.
- embodiments of the invention automatically recreate the content source logs and structure of content ingested and the user is then provided with a view of their access.
- storing content for access by a user is achieved by either caching data or searching.
- the anticipated data required by the user is retrieved from the remote storage and cached in memory of the user's electronic device but this is expensive and difficult where the volumes of data are large. The latter is slow and expensive in respect of storing current searches etc.
- the system pushes some data to the data source connector and the user is then involved to associate certain locations as favorites, for example by entering a path as a favorite or running an initial search and then selecting favorites.
- FIGS. 3 and 4 there are depicted flow diagrams in respect of providing a knowledge worker with a cloud view of their entitlements from third party data sources according to an embodiment of the invention. Accordingly referring to FIG. 3 the process comprises steps 305 to 350 comprising:
- Rendering the data map within the web service UI to the user may be through methods as known in the prior art as well as those subsequently envisioned for providing a user with a visualization of locations of individual files, folders, directories, servers, networks, etc.
- a Connector and submitted to Hub Stor in the cloud.
- the location and permissions information are essentially inherited or synchronized context relating to the Blobs (data content) written to the cloud archive. Accordingly, one of the uses of this context is to provide individual knowledge workers with a familiar view of their data, organized in the same folders as the user had them in the original data source. However, this would be very inefficient to subsequently query the full list of permissioned locations for a user each time they access their Web UI to view and/or access their Hub Stor cloud archive.
- the process flow described and depicted in respect of FIG. 5 addresses that through the concept of favorites such that a user accessing their Web UI is presented with the result of a query to their favorite locations rapidly whilst either the remainder are built and displayed subsequently or held pending a user request to view/access.
- Such a methodology of minimizing data transfer upon an initial user access being particularly beneficial in mobile based user access.
- the favorite location(s) concept for a given user is also exposed in the RS-RAP's Connector Application Programming Interface (API).
- API Connector Application Programming Interface
- a Connector has the ability to programmatically pass through the ConnectorService that a particular location or sub-location that it is submitting should be a favorite. Any locations marked by a Connector as a favorite will automatically appear for the user when they login to the End-User Web UI. For example, many locations such as a user's mailbox may be a favorite.
- RS-RAP based analytics may establish a location or locations as favorite(s) based upon the user's access and usage patterns.
- a data source connector requires its own (i) user interface, (ii) record of what has been previously captured from the data source, and (iii) policy and other capture logic.
- embodiments of the invention provide a generic interface that completely removes the connector's traditional requirements. Accordingly, the generic interface is agnostic in that it may be easily supported and/or installed within any electronic device, such as a PED, FED or wearable device, supporting the coded generic interface.
- the invention operates through provisioning three elements within the data source connector, these being:
- the connector as a minimum functional set should interface with a data source and be able to handle for this data source any calls from a cloud ingestion service.
- the cloud storage repository supports directly or through other cloud storage repositories all of the policies and other capture logic, the user interface(s), and associated system(s) of record.
- a software development kit (SDK) and/or a dynamic link library (DLL) are executed at the client side, e.g. the user's electronic device, to put the monitoring/maintenance engine on their system and then push the incremental changes to the remote network storage, e.g. cloud storage.
- SDK software development kit
- DLL dynamic link library
- the inventors refer to this as a data source connector SDK or “Connector SDK.”
- this API creates the embedded database (DB) behind the scenes and tracks all state information within it, i.e. the contents of each data source.
- DB embedded database
- Any Connector simply uses the API to pass information about what is currently in a data source's location and the API implementation does the rest including, but not limited to, find incremental differences, pushing incremental differences up to the RS-RAP cloud storage, and saving the updated information within the embedded database.
- FIG. 5 there is depicted a flow diagram for incremental synchronised connector writing to a cloud data repository according to an embodiment of the invention comprising steps 505 to 570 respectively. These steps being
- Cloud based storage Users of network based storage, and particularly cloud storage may, as a result of a single query or search, access millions, hundreds of millions and even billions of entries within the database(s) or repository(ies) accessed. Similarly, these records may be subject to thousands or millions of concurrent searches/queries. Accordingly, it is important for the cloud based storage to provide the users with an efficient means of initially querying and subsequently maintaining statistics for a large data set (item version counts in the multiple billions). Accordingly, embodiments of the invention provide for a responsive and computationally light method of refreshing statistics and getting query results for policies that evaluate one or more clauses against a cloud storage repository and provide statistics on their resulting data set.
- embodiments of the invention may operate by segmenting a policy's data set into a series of policy blocks. These policy blocks representing logical groupings of item versions from the policy's data set, where each policy block is stored as a separate record in a policy block table. As a result, this initial segmentation supports subsequent computational work being performed incrementally.
- a policy is configured with two major attributes, a query and an action.
- Query defineds the item version(s) that meet the policy and accordingly is based, typically, upon ItemVersion fields. Ultimately these are translated into a single, possibly complex, WHERE clause at policy evaluation time. This may also have potential additional elements, commonly referred to as joins, to other policies and/or data, relating to location/custodian restrictions, for example.
- Actions defineds an optional list of actions (e.g. full-text index, PII detect, deletion, etc.) that are to be applied to item versions that match the query.
- policies record overall statistics and aggregation statistics for the item versions that meet the query.
- a policy may exist strictly in order to track statistics for a subset of data, but often will exist to have associated actions to manipulate item version that meet the query.
- FIG. 6 there is depicted a flow diagram for initial policy creation and execution according to an embodiment of the invention comprising steps 605 to 650 , wherein these comprise:
- step 650 Upon a negative determination the process proceeds to step 650 and ends, otherwise it proceeds to step 645 which comprises:
- a policy may be re-run wherein the RS-RAP must handle the instances of changes to the query result set, resulting from either changes in the corpus of data being queried or in the policy's query filter criteria.
- policies record overall statistics and aggregation statistics for the item versions that meet the query, and the item counts may be very large (e.g. a billion items) it would be efficient to only update statistics incrementally where changes have occured.
- FIG. 7 An embodiment of this is depicted in FIG. 7 for an incremental policy execution and update of statistics according to an embodiment of the invention comprising steps 705 to 740 respectively. As such the process steps comprise:
- step 725 Upon a positive determination in step 725 the process proceeds to step 730 , wherein
- a dirty block is a region of ItemVersionIDs, a database of content with associated metadata relating to the content, in which at least one of the item versions has undergone a metadata change or location change.
- Block.LastUpdate in order to make the system thread-safe. Whenever anything changes to make a block dirty, Block.Dirty is simply set to 1 and the LastUpdate set to the current timestamp in the database. After a dirty block has been re-evaluated then Block.Dirty is set to 0 only if the LastUpdate hasn't changed since the worker thread started processing the given block. This means if a change is made to an item version in the block while it is being processed, it stays Dirty and must be processed again. In this manner no changes are missed. If a complete reevaluation of the policy is required, both for action queuing and statistics, then it is only necessary to simply mark all blocks as dirty.
- FIG. 8 there is depicted schematically a process for writing to cloud storage without exposing cloud storage API security according to an embodiment of the invention.
- the process comprises first to ninth flows 801 to 809 respectively from an on-premises ConnectorService 810 (assumed to be behind a firewall) with a WebApp 820 and a Staging Container 830 whilst some of the flows are then between the WebApp 820 and an Index 840 , Blob Archive Container 850 , and the Staging Container 830 . Accordingly, these flows are:
- ConnectorService 810 obtains a time-limited write-only token to the Staging Container 830 from the WebApp 820 .
- This ensures the ability to read any content in any of the containers (staging included) never leaves the WebApp 820 whilst the use of a separate Staging Container 830 also ensures that malicious writes do not affect the production blob within the Blob Archive Container and can be easily detected and deleted. Indeed, there is a separate maintenance task that periodically cleans up any lingering content in the Staging Container 830 that should not be there.
- ConnectorService 810 calls the WebApp 820 writing method for passing the blob locations in the Staging Container 830 .
- Fourth flow 804 where the WebApp 820 uses a MD5 hash, for example, of the blobs to check for duplicates and, if there is a duplicate, single instance stores the blob after doing a full byte-by-byte comparison to ensure no MD5 collision occurred. If there is a collision an error is logged and the write operation fails.
- a MD5 hash for example, of the blobs to check for duplicates and, if there is a duplicate, single instance stores the blob after doing a full byte-by-byte comparison to ensure no MD5 collision occurred. If there is a collision an error is logged and the write operation fails.
- Ninth flow 809 where subsequently the policy engine will detect the dirty block(s) and re-evaluate for the dirty block and, as appropriate take action against the item (i.e. full text index, update policy stats, etc.).
- FIG. 9 there is depicted a flow diagram for writing to cloud storage without exposing cloud storage API security as depicted by the process in FIG. 8 according to an embodiment of the invention.
- the process comprises steps 905 to 990 respectively.
- the process steps comprise:
- step 935 if the determination is negative then the process proceeds to steps 955 to 975 before ending at step 990 .
- steps 955 to 975 these comprise:
- step 975 the process proceeds to step 990 and ends. If in step 935 the determination is positive, then the process proceeds to step 940 wherein a byte-by-byte comparison is performed to make sure there are no collisions. From step 940 the process proceeds to step 945 wherein a determination regarding the collision is made. If the determination is negative the process proceeds from step 945 to 980 and thereafter step 975 before ending in step 990 . In step 980 the WebApp performs a single instance storage operation. If the determination is positive in step 945 then the process proceeds to step 950 and thereafter step 985 before ending in step 990 . Steps 950 and 985 comprise:
- FIGS. 3 to 9 and the Figures embodiments of the invention have been described from the viewpoint of a user accessing data/content that has been stored within a network and/or cloud storage system such as, for example, when an enterprise transfers its historical physical electronic data repositories into cloud based storage for a single location, single team, multiple locations, etc. and the user is accessing/storing/reading/writing content.
- the embodiments of the invention render a data map within the web service UI to the user through methods as known in the prior art as well as those subsequently envisioned for providing a user with a visualization of locations of individual files, folders, directories, servers, networks, etc.
- today one such standard format is that provided through MicrosoftTM WindowsTM Explorer.
- the content stored may already be visualized through a web based UI such as Microsoft EdgeTM or Google ChromeTM for example, wherein a user may already have multiple cloud storage stores such as Microsoft OneDriveTM DropBoxTM, and Google Drive which they wish to view in a consolidated manner through a single web UI or actually consolidate within the cloud as well as virtually within the web UI.
- embodiments of the invention may allow a user to exploit similar functionality in respect of “favorites” through a web based UI for a plurality of social networks and/or social media that the user accesses. For example, a user may wish to access on a regular basis elements of a corporate Facebook, a product support webpage, multiple supplier Facebook pages, Twitter accounts etc. but each time they login the system only accesses the “favorites” initially.
- Implementation of the techniques, blocks, steps and means described above may be done in various ways. For example, these techniques, blocks, steps and means may be implemented in hardware, software, or a combination thereof.
- the processing units may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above and/or a combination thereof.
- ASICs application specific integrated circuits
- DSPs digital signal processors
- DSPDs digital signal processing devices
- PLDs programmable logic devices
- FPGAs field programmable gate arrays
- processors controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above and/or a combination thereof.
- the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged.
- a process is terminated when its operations are completed, but could have additional steps not included in the figure.
- a process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
- embodiments may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages and/or any combination thereof.
- the program code or code segments to perform the necessary tasks may be stored in a machine readable medium, such as a storage medium.
- a code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures and/or program statements.
- a code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters and/or memory content. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
- the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein.
- Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein.
- software codes may be stored in a memory.
- Memory may be implemented within the processor or external to the processor and may vary in implementation where the memory is employed in storing software codes for subsequent execution to that when the memory is employed in executing the software codes.
- the term “memory” refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.
- the term “storage medium” may represent one or more devices for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information.
- ROM read only memory
- RAM random access memory
- magnetic RAM magnetic RAM
- core memory magnetic disk storage mediums
- optical storage mediums flash memory devices and/or other machine readable mediums for storing information.
- machine-readable medium includes, but is not limited to portable or fixed storage devices, optical storage devices, wireless channels and/or various other mediums capable of storing, containing or carrying instruction(s) and/or data.
- the methodologies described herein are, in one or more embodiments, performable by a machine which includes one or more processors that accept code segments containing instructions. For any of the methods described herein, when the instructions are executed by the machine, the machine performs the method. Any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine are included.
- a typical machine may be exemplified by a typical processing system that includes one or more processors.
- Each processor may include one or more of a CPU, a graphics-processing unit, and a programmable DSP unit.
- the processing system further may include a memory subsystem including main RAM and/or a static RAM, and/or ROM.
- a bus subsystem may be included for communicating between the components.
- the processing system requires a display, such a display may be included, e.g., a liquid crystal display (LCD).
- a display e.g., a liquid crystal display (LCD).
- the processing system also includes an input device such as one or more of an alphanumeric input unit such as a keyboard, a pointing control device such as a mouse, and so forth.
- the memory includes machine-readable code segments (e.g. software or software code) including instructions for performing, when executed by the processing system, one of more of the methods described herein.
- the software may reside entirely in the memory, or may also reside, completely or at least partially, within the RAM and/or within the processor during execution thereof by the computer system.
- the memory and the processor also constitute a system comprising machine-readable code.
- the machine operates as a standalone device or may be connected, e.g., networked to other machines, in a networked deployment, the machine may operate in the capacity of a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer or distributed network environment.
- the machine may be, for example, a computer, a server, a cluster of servers, a cluster of computers, a web appliance, a distributed computing environment, a cloud computing environment, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
- the term “machine” may also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
- the specification may have presented the method and/or process of the present invention as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. As one of ordinary skill in the art would appreciate, other sequences of steps may be possible. Therefore, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. In addition, the claims directed to the method and/or process of the present invention should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the present invention.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Transfer Between Computers (AREA)
- Human Computer Interaction (AREA)
Abstract
Description
-
- 1.The RS-RAP (e.g. HubStor™) cloud data repository, which houses the data stored within the cloud by the user, parties associated with a user, third parties etc.
- 2. A ConnectorService which is an operating system (e.g. Microsoft™ Windows™, Linux™, MAC OS™) service that runs processing for one or more connectors which are identify by machine name. Within an embodiment of the invention each ConnectorService connects to the RS-RAP (HubStor™) in the cloud and has its own “thick” UI which is used to configure all settings and all connectors used by the ConnectorService (wherein this configuration is still stored in the RS-RAP (e.g. HubStor™) cloud database.
- 3. Connector(s) which are individually each a single instance capable of scanning and synchronising a data source (i.e. Exchange, SharePoint, file servers, etc.) to the RS-RAP on a policy based basis. Accordingly, connectors are focused on providing very basic functionality whilst the requisite logic is in the ConnectorService. Within an embodiment of the invention the basic functionality the Connector needs to support includes, but is not limited to, get root location, get sub-locations, get items in location, and get an identifier, e.g. HSItem, for an item.
-
- a cloud storage repository;
- a user directory synchronization service;
- at least one data source connector; and
- a human interface.
-
- Step 305 wherein a user seeking to access cloud based content logs into the RS-RAP's (e.g. Hub Stor™ as defined within the specification and figures) end user web based user interface (UI);
- Step 310 wherein the user requests a data map report from the RS-RAP;
- Step 315 where the user's request is sent to the RS-RAP (HubStor™, subsequently referred to as HubStors) web service;
- Step 320 wherein the RS-RAP web service queries the user's location data map;
- Step 325 wherein the RS-RAP web service returns the user's data map information;
- Step 330 wherein the RS-RAP's end user UI renders the data map (data map information) into a report presented to the user;
- Step 335 wherein the user selects one or more favorite locations from the rendered data map (which they may customise in terms of rendering to a format of their personal preference rather than a RS-RAP default option);
- Step 340 wherein the user saves and exits the report provided by the RS-RAP end user UI;
- Step 345 where the RS-RAP end user UI sends identification information relating to the user selected favorite locations to the RS-RAP web service; and
- Step 350 where the RS-RAP web service updates the favorite locations for the user within a RS-RAP cloud database.
-
- Step 405 wherein the user starts by logging into the RS-RAP's End User Web UI;
- Step 410 wherein the RS-RAP Web Service fetches favorite locations associated with the user;
- Step 415 wherein the RS-RAP's End User Web UI receives the favorite locations; and
- Step 420 wherein the RS-RAP End User Web UI renders for display the favorite locations for the user to consume.
-
- a cloud ingestion service for a cloud repository, wherein the cloud repository comprises at least a database and a set of policies;
- an Application Programming Interface (API); and
- at least one data source connector.
-
- 1) Acquisition of root level locations;
- 2) Acquisition of sub locations; and
- 3) Acquisition of a list of items.
-
- Step 505 wherein the job is started from ConnectorService;
- Step 510 wherein the job obtains Connector Information from a ConnectorService Database;
- Step 515 wherein the job invoke calls to a Connector starting with “Get RootLocations”;
- Step 520 wherein a response from the Connector is received providing the requested data to the ConnectorService;
- Step 525 wherein the ConnectorService stores the latest data with respect to the data source;
- Step 530 wherein the ConnectorService computes a difference and calls a policy to determine whether there are necessary operations or not; and
- Step 535 wherein the process based upon determining whether additional operations are required either proceeds forward to step 570 and ends for a negative determination or proceeds to
steps 540 to 565 upon a positive determination.
-
- Step 540 wherein the ConnectorService passes to the Connector a list of items to capture;
- Step 545 wherein the Connector fetches applicable data blocks, permissions, and item metadata from the data source;
- Step 550 where the Connector submits them to the ConnectorService;
- Step 555 where the ConnectorService submits the items to the RS-RAP cloud storage, e.g. HubStor™ Cloud Data Repository;
- Step 560 wherein the RS-RAP cloud storage sends a commit response to ConnectorService; and
- Step 565 where the ConnectorService stores the commit status in its database.
-
- Step 605 where a policy is created;
- Step 610 where any policy filter criteria for the policy are defined;
- Step 615 wherein any policy actions are assigned to the policy;
- Step 620 wherein the resulting policy is run;
- Step 625 where based upon the policy execution records are created, such as within PolicyBlock and Block tables, relating to the policy execution;
- Step 630 where statistics of the policy are computed;
- Step 635 wherein the computed statistics for the policy are stored; and
- Step 640 wherein a determination of additional actions is made.
-
- Step 645 wherein additional actions are performed, such as updating an ItemVersionAction table for example.
-
- Step 705 wherein a trigger or user login or other action initiates an update policy statistics job execution;
- Step 710 wherein the process establishes a list of BlockIDs with Block.Dirty=1;
- Step 715 wherein the process re-computes the policy statistics;
- Step 720 wherein the latest policy statistics are stored;
- Step 725 wherein a determination is made as to whether the last update has changed since the user thread started processing.
-
- Step 730 wherein the process leaves impacted BlockIDs as Block.Dirty=1;
-
- Step 735 wherein the process resets impacted BlockIDs as Block.Dirty=0.
-
- Step 905 wherein the job is started from the ConnectorService;
- Step 910 wherein the ConnectorService calls the WebApp to obtain a write-only token to the Staging Container;
- Step 915 where the WebApp Generates the Token and passes it to the ConnectorService;
- Step 920 wherein the ConnectorService uses the received token to write blobs to the Staging Container;
- Step 925 wherein the ConnectorService provides data to the WebApp concerning the Blobs written to the Staging Container;
- Step 930 wherein the WebApp uses an MD5 Hash to check for duplicates;
-
- Step 955 wherein the new item is written to the Blob Archive Container;
- Step 960 wherein the write integrity is verified;
- Step 965 wherein the relevant blocks within the database are marked as dirty;
- Step 970 wherein the WebApp proceeds to delete the relevant blobs from the Staging Container;
- Step 975 wherein the WebApp returns the item identifiers to the ConnectorService.
-
- Step 950 wherein the error is logged in respect of the failed write operation; and
- Step 985 wherein the WebApp returns the error to the ConnectorService.
Claims (12)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/346,094 US10516669B2 (en) | 2015-11-12 | 2016-11-08 | Methods and systems relating to network based storage |
US16/681,033 US11575674B2 (en) | 2015-11-12 | 2019-11-12 | Methods and systems relating to network based storage |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201562254258P | 2015-11-12 | 2015-11-12 | |
US15/346,094 US10516669B2 (en) | 2015-11-12 | 2016-11-08 | Methods and systems relating to network based storage |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/681,033 Continuation US11575674B2 (en) | 2015-11-12 | 2019-11-12 | Methods and systems relating to network based storage |
Publications (2)
Publication Number | Publication Date |
---|---|
US20170142113A1 US20170142113A1 (en) | 2017-05-18 |
US10516669B2 true US10516669B2 (en) | 2019-12-24 |
Family
ID=58691713
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/346,094 Active 2037-03-06 US10516669B2 (en) | 2015-11-12 | 2016-11-08 | Methods and systems relating to network based storage |
US16/681,033 Active US11575674B2 (en) | 2015-11-12 | 2019-11-12 | Methods and systems relating to network based storage |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/681,033 Active US11575674B2 (en) | 2015-11-12 | 2019-11-12 | Methods and systems relating to network based storage |
Country Status (1)
Country | Link |
---|---|
US (2) | US10516669B2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11134085B2 (en) * | 2018-10-08 | 2021-09-28 | Sonrai Security Inc. | Cloud least identity privilege and data access framework |
US11630744B2 (en) | 2017-05-18 | 2023-04-18 | Veritas Technologies Llc | Methods and systems relating to network based storage retention |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11331019B2 (en) | 2017-08-07 | 2022-05-17 | The Research Foundation For The State University Of New York | Nanoparticle sensor having a nanofibrous membrane scaffold |
CN108156175B (en) * | 2018-01-22 | 2021-05-14 | 中证鹏元资信评估股份有限公司 | Method for accessing shared storage information under cloud computing platform |
US10749771B2 (en) * | 2018-05-18 | 2020-08-18 | Microsoft Technology Licensing, Llc | Extensible, secure and efficient monitoring and diagnostic pipeline for hybrid cloud architecture |
CN110245204A (en) * | 2019-06-12 | 2019-09-17 | 桂林电子科技大学 | A kind of intelligent recommendation method based on positioning and knowledge mapping |
US20200394225A1 (en) * | 2019-06-14 | 2020-12-17 | Salesforce.Com, Inc. | Prepackaged data ingestion from various data sources |
CN118069658B (en) * | 2024-04-19 | 2024-06-25 | 北京威斯盾网络科技有限公司 | Control method of database system and database system |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7275244B1 (en) * | 2003-03-18 | 2007-09-25 | Microsoft Corporation | System and method for incrementally saving web files to a web server using file hash values |
US8527549B2 (en) * | 2010-02-22 | 2013-09-03 | Sookasa Inc. | Cloud based operating and virtual file system |
US20140026182A1 (en) * | 2012-07-19 | 2014-01-23 | Box, Inc. | Data loss prevention (dlp) methods by a cloud service including third party integration architectures |
US20140115329A1 (en) * | 2012-08-31 | 2014-04-24 | Pkware, Inc. | Systems and methods for data verifcation and replay prevention |
US20150319185A1 (en) * | 2013-12-13 | 2015-11-05 | Palerra, Inc. | Systems and Methods for Contextual and Cross Application Threat Detection and Prediction in Cloud Applications |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9678981B1 (en) * | 2010-05-03 | 2017-06-13 | Panzura, Inc. | Customizing data management for a distributed filesystem |
US9870398B1 (en) * | 2012-12-31 | 2018-01-16 | Teradata Us, Inc. | Database-table sampling-percentage selection |
US20150281292A1 (en) * | 2014-03-25 | 2015-10-01 | PlusAmp, Inc. | Data File Discovery, Visualization, and Actioning |
EP3040886A1 (en) * | 2015-01-05 | 2016-07-06 | Iguazio Systems Ltd. | Service oriented data management and architecture |
US10380108B2 (en) * | 2015-06-22 | 2019-08-13 | International Business Machines Corporation | Partition access method for query optimization |
-
2016
- 2016-11-08 US US15/346,094 patent/US10516669B2/en active Active
-
2019
- 2019-11-12 US US16/681,033 patent/US11575674B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7275244B1 (en) * | 2003-03-18 | 2007-09-25 | Microsoft Corporation | System and method for incrementally saving web files to a web server using file hash values |
US8527549B2 (en) * | 2010-02-22 | 2013-09-03 | Sookasa Inc. | Cloud based operating and virtual file system |
US20140026182A1 (en) * | 2012-07-19 | 2014-01-23 | Box, Inc. | Data loss prevention (dlp) methods by a cloud service including third party integration architectures |
US20140115329A1 (en) * | 2012-08-31 | 2014-04-24 | Pkware, Inc. | Systems and methods for data verifcation and replay prevention |
US20150319185A1 (en) * | 2013-12-13 | 2015-11-05 | Palerra, Inc. | Systems and Methods for Contextual and Cross Application Threat Detection and Prediction in Cloud Applications |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11630744B2 (en) | 2017-05-18 | 2023-04-18 | Veritas Technologies Llc | Methods and systems relating to network based storage retention |
US11134085B2 (en) * | 2018-10-08 | 2021-09-28 | Sonrai Security Inc. | Cloud least identity privilege and data access framework |
Also Published As
Publication number | Publication date |
---|---|
US11575674B2 (en) | 2023-02-07 |
US20170142113A1 (en) | 2017-05-18 |
US20200084215A1 (en) | 2020-03-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11575674B2 (en) | Methods and systems relating to network based storage | |
US11789828B2 (en) | Methods and systems relating to network based storage | |
US11100042B2 (en) | Methods and systems relating to network based storage | |
US20200128073A1 (en) | Synchronization of permissioned content in cloud-based environments | |
US10515058B2 (en) | Unified file and object data storage | |
US20200401554A1 (en) | Selective data migration and sharing | |
US9866508B2 (en) | Aggregating and presenting recent activities for synchronized online content management systems | |
Oussalah et al. | A software architecture for Twitter collection, search and geolocation services | |
US9268716B2 (en) | Writing data from hadoop to off grid storage | |
US20140304384A1 (en) | Uploading large content items | |
US10872097B2 (en) | Data resolution system for management of distributed data | |
CN104092770A (en) | Inner-enterprise address book management method and system based on cloud computing | |
US10642862B2 (en) | System and method for referencing master data in a multi-tenant application-platform-as-a-service architecture | |
JP7538949B2 (en) | Inter-application data exchange via a group-based communication system that triggers user intervention | |
US11630744B2 (en) | Methods and systems relating to network based storage retention | |
US20150365497A1 (en) | Providing access to information across multiple computing devices | |
US20090234902A1 (en) | System, method and apparatus for making content available over multiple devices | |
CN105162697A (en) | Microblog system and data control method thereof | |
JP2016503202A (en) | Create tasks based on newsfeed user entries | |
US11848994B2 (en) | Communication protocols for an online content management system | |
US20170180285A1 (en) | Import content items from email | |
Göndör et al. | The current state of interoperability in decentralized online social networking services | |
US10021167B2 (en) | Accessing mobile documents | |
US12069058B2 (en) | Security mechanisms for content management systems | |
Karakostas | A cloud architecture for dynamic virtual organisations in transport logistics |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HUBSTOR INC., CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BOURGEOIS, GEOFFREY;CAMPBELL, GREG;REEL/FRAME:040255/0737 Effective date: 20161103 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: VERITAS TECHNOLOGIES LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HUBSTOR INC.;REEL/FRAME:058665/0587 Effective date: 20210201 Owner name: VERITAS STORAGE (IRELAND) LTD., IRELAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HUBSTOR INC.;REEL/FRAME:058163/0954 Effective date: 20210201 Owner name: VERITAS TECHNOLOGIES LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:VERITAS STORAGE (IRELAND) LTD.;REEL/FRAME:058135/0116 Effective date: 20211028 |
|
AS | Assignment |
Owner name: BANK OF AMERICA, N.A., ILLINOIS Free format text: PATENT SECURITY AGREEMENT SUPPLEMENT;ASSIGNOR:VERITAS TECHNOLOGIES LLC;REEL/FRAME:059247/0656 Effective date: 20220218 |
|
AS | Assignment |
Owner name: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT, DELAWARE Free format text: PATENT SECURITY AGREEMENT SUPPLEMENT;ASSIGNOR:VERITAS TECHNOLOGIES LLC;REEL/FRAME:059337/0159 Effective date: 20220225 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |