SE1751210A1 - Distributed data set storage and retrieval - Google Patents

Distributed data set storage and retrieval Download PDF

Info

Publication number
SE1751210A1
SE1751210A1 SE1751210A SE1751210A SE1751210A1 SE 1751210 A1 SE1751210 A1 SE 1751210A1 SE 1751210 A SE1751210 A SE 1751210A SE 1751210 A SE1751210 A SE 1751210A SE 1751210 A1 SE1751210 A1 SE 1751210A1
Authority
SE
Sweden
Prior art keywords
data
data set
node devices
available
map
Prior art date
Application number
SE1751210A
Other languages
English (en)
Other versions
SE540908C2 (en
Inventor
Payton Bowman Brian
E Krueger Steven
Todd KNIGHT Richard
Ho Chih-Wei
Original Assignee
Sas Inst Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sas Inst Inc filed Critical Sas Inst Inc
Publication of SE1751210A1 publication Critical patent/SE1751210A1/sv
Publication of SE540908C2 publication Critical patent/SE540908C2/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G06F16/137—Hash-based
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/0292—User address space allocation, e.g. contiguous or non contiguous base addressing using tables or multilevel address translation means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • G06F16/1827—Management specifically adapted to NAS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1016—Performance improvement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1056—Simplification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/15—Use in a specific computing environment
    • G06F2212/154—Networked environment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/26—Using a specific storage system architecture
    • G06F2212/261—Storage comprising a plurality of storage devices
    • G06F2212/262—Storage comprising a plurality of storage devices configured as RAID
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/26—Using a specific storage system architecture
    • G06F2212/263—Network storage, e.g. SAN or NAS

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Multi Processors (AREA)
SE1751210A 2015-07-27 2016-07-27 Distributed data set storage and retrieval SE540908C2 (en)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201562197519P 2015-07-27 2015-07-27
US201562197514P 2015-07-27 2015-07-27
US15/220,192 US9703789B2 (en) 2015-07-27 2016-07-26 Distributed data set storage and retrieval
US15/220,182 US9811524B2 (en) 2015-07-27 2016-07-26 Distributed data set storage and retrieval
US15/220,034 US9619148B2 (en) 2015-07-27 2016-07-26 Distributed data set storage and retrieval
PCT/US2016/044309 WO2017019794A1 (en) 2015-07-27 2016-07-27 Distributed data set storage and retrieval

Publications (2)

Publication Number Publication Date
SE1751210A1 true SE1751210A1 (sv) 2017-09-29
SE540908C2 SE540908C2 (en) 2018-12-18

Family

ID=57882504

Family Applications (3)

Application Number Title Priority Date Filing Date
SE1751211A SE540909C2 (en) 2015-07-27 2016-07-27 Distributed data set storage and retrieval
SE1751212A SE541999C2 (en) 2015-07-27 2016-07-27 Distributed data set storage and retrieval
SE1751210A SE540908C2 (en) 2015-07-27 2016-07-27 Distributed data set storage and retrieval

Family Applications Before (2)

Application Number Title Priority Date Filing Date
SE1751211A SE540909C2 (en) 2015-07-27 2016-07-27 Distributed data set storage and retrieval
SE1751212A SE541999C2 (en) 2015-07-27 2016-07-27 Distributed data set storage and retrieval

Country Status (8)

Country Link
US (4) US9619148B2 (sv)
CN (1) CN108701076B (sv)
CA (1) CA2981521C (sv)
DE (1) DE112016001075B4 (sv)
HK (1) HK1259159A1 (sv)
IL (1) IL254745B (sv)
SE (3) SE540909C2 (sv)
WO (1) WO2017019794A1 (sv)

Families Citing this family (98)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9619148B2 (en) 2015-07-27 2017-04-11 Sas Institute Inc. Distributed data set storage and retrieval
US9990367B2 (en) 2015-07-27 2018-06-05 Sas Institute Inc. Distributed data set encryption and decryption
US10929417B2 (en) * 2015-09-11 2021-02-23 International Business Machines Corporation Transforming and loading data utilizing in-memory processing
US10784901B2 (en) 2015-11-12 2020-09-22 Qualcomm Incorporated Puncturing for structured low density parity check (LDPC) codes
US10152527B1 (en) 2015-12-28 2018-12-11 EMC IP Holding Company LLC Increment resynchronization in hash-based replication
US10324782B1 (en) 2016-03-24 2019-06-18 Emc Corporation Hiccup management in a storage array
US10101934B1 (en) 2016-03-24 2018-10-16 Emc Corporation Memory allocation balancing for storage systems
US10705907B1 (en) 2016-03-24 2020-07-07 EMC IP Holding Company LLC Data protection in a heterogeneous random access storage array
US10469104B2 (en) 2016-06-14 2019-11-05 Qualcomm Incorporated Methods and apparatus for compactly describing lifted low-density parity-check (LDPC) codes
US10250450B2 (en) 2016-06-29 2019-04-02 Nicira, Inc. Distributed network troubleshooting using simultaneous multi-point packet capture
US20180089324A1 (en) 2016-09-26 2018-03-29 Splunk Inc. Dynamic resource allocation for real-time search
US11620336B1 (en) 2016-09-26 2023-04-04 Splunk Inc. Managing and storing buckets to a remote shared storage system based on a collective bucket size
US10977260B2 (en) 2016-09-26 2021-04-13 Splunk Inc. Task distribution in an execution node of a distributed execution environment
US11567993B1 (en) * 2016-09-26 2023-01-31 Splunk Inc. Copying buckets from a remote shared storage system to memory associated with a search node for query execution
US11586627B2 (en) 2016-09-26 2023-02-21 Splunk Inc. Partitioning and reducing records at ingest of a worker node
US10956415B2 (en) 2016-09-26 2021-03-23 Splunk Inc. Generating a subquery for an external data system using a configuration file
US11604795B2 (en) 2016-09-26 2023-03-14 Splunk Inc. Distributing partial results from an external data system between worker nodes
US11562023B1 (en) 2016-09-26 2023-01-24 Splunk Inc. Merging buckets in a data intake and query system
US10726009B2 (en) 2016-09-26 2020-07-28 Splunk Inc. Query processing using query-resource usage and node utilization data
US11163758B2 (en) 2016-09-26 2021-11-02 Splunk Inc. External dataset capability compensation
US11580107B2 (en) 2016-09-26 2023-02-14 Splunk Inc. Bucket data distribution for exporting data to worker nodes
US10984044B1 (en) 2016-09-26 2021-04-20 Splunk Inc. Identifying buckets for query execution using a catalog of buckets stored in a remote shared storage system
US11663227B2 (en) 2016-09-26 2023-05-30 Splunk Inc. Generating a subquery for a distinct data intake and query system
US11599541B2 (en) 2016-09-26 2023-03-07 Splunk Inc. Determining records generated by a processing task of a query
US11294941B1 (en) 2016-09-26 2022-04-05 Splunk Inc. Message-based data ingestion to a data intake and query system
US11222066B1 (en) 2016-09-26 2022-01-11 Splunk Inc. Processing data using containerized state-free indexing nodes in a containerized scalable environment
US11314753B2 (en) 2016-09-26 2022-04-26 Splunk Inc. Execution of a query received from a data intake and query system
US11860940B1 (en) 2016-09-26 2024-01-02 Splunk Inc. Identifying buckets for query execution using a catalog of buckets
US11550847B1 (en) 2016-09-26 2023-01-10 Splunk Inc. Hashing bucket identifiers to identify search nodes for efficient query execution
US11874691B1 (en) 2016-09-26 2024-01-16 Splunk Inc. Managing efficient query execution including mapping of buckets to search nodes
US11416528B2 (en) 2016-09-26 2022-08-16 Splunk Inc. Query acceleration data store
US11593377B2 (en) 2016-09-26 2023-02-28 Splunk Inc. Assigning processing tasks in a data intake and query system
US10353965B2 (en) 2016-09-26 2019-07-16 Splunk Inc. Data fabric service system architecture
US11232100B2 (en) 2016-09-26 2022-01-25 Splunk Inc. Resource allocation for multiple datasets
US11250056B1 (en) 2016-09-26 2022-02-15 Splunk Inc. Updating a location marker of an ingestion buffer based on storing buckets in a shared storage system
US11126632B2 (en) 2016-09-26 2021-09-21 Splunk Inc. Subquery generation based on search configuration data from an external data system
US11269939B1 (en) 2016-09-26 2022-03-08 Splunk Inc. Iterative message-based data processing including streaming analytics
US10776355B1 (en) 2016-09-26 2020-09-15 Splunk Inc. Managing, storing, and caching query results and partial query results for combination with additional query results
US11023463B2 (en) 2016-09-26 2021-06-01 Splunk Inc. Converting and modifying a subquery for an external data system
US11003714B1 (en) 2016-09-26 2021-05-11 Splunk Inc. Search node and bucket identification using a search node catalog and a data store catalog
US10795884B2 (en) 2016-09-26 2020-10-06 Splunk Inc. Dynamic resource allocation for common storage query
US11243963B2 (en) 2016-09-26 2022-02-08 Splunk Inc. Distributing partial results to worker nodes from an external data system
US11281706B2 (en) 2016-09-26 2022-03-22 Splunk Inc. Multi-layer partition allocation for query execution
US11442935B2 (en) 2016-09-26 2022-09-13 Splunk Inc. Determining a record generation estimate of a processing task
US11615104B2 (en) 2016-09-26 2023-03-28 Splunk Inc. Subquery generation based on a data ingest estimate of an external data system
US11321321B2 (en) 2016-09-26 2022-05-03 Splunk Inc. Record expansion and reduction based on a processing task in a data intake and query system
US11106734B1 (en) 2016-09-26 2021-08-31 Splunk Inc. Query execution using containerized state-free search nodes in a containerized scalable environment
US11461334B2 (en) 2016-09-26 2022-10-04 Splunk Inc. Data conditioning for dataset destination
US10152371B1 (en) * 2016-09-30 2018-12-11 EMC IP Holding Company LLC End-to-end data protection for distributed storage
US10223008B1 (en) 2016-09-30 2019-03-05 EMC IP Holding Company LLC Storage array sizing for compressed applications
US10255172B1 (en) 2016-09-30 2019-04-09 EMC IP Holding Company LLC Controlled testing using code error injection
US10447763B2 (en) * 2016-12-08 2019-10-15 Nanning Fugui Precision Industrial Co., Ltd. Distributed storage method and system
US10909173B2 (en) * 2016-12-09 2021-02-02 The Nielsen Company (Us), Llc Scalable architectures for reference signature matching and updating
US10921991B1 (en) 2016-12-20 2021-02-16 Amazon Technologies, Inc. Rule invalidation for a block store management system
US10809920B1 (en) 2016-12-20 2020-10-20 Amazon Technologies, Inc. Block store management for remote storage systems
US10268593B1 (en) 2016-12-20 2019-04-23 Amazon Technologies, Inc. Block store managamement using a virtual computing system service
US10185507B1 (en) * 2016-12-20 2019-01-22 Amazon Technologies, Inc. Stateless block store manager volume reconstruction
US11507283B1 (en) 2016-12-20 2022-11-22 Amazon Technologies, Inc. Enabling host computer systems to access logical volumes by dynamic updates to data structure rules
CA3061047A1 (en) * 2017-04-27 2018-11-01 Financial & Risk Organisation Limited Systems and methods for distributed data mapping
TWI644229B (zh) * 2017-05-04 2018-12-11 慧榮科技股份有限公司 採加密技術之數據中心與數據中心操作方法
US9870508B1 (en) * 2017-06-01 2018-01-16 Unveiled Labs, Inc. Securely authenticating a recording file from initial collection through post-production and distribution
US10312939B2 (en) 2017-06-10 2019-06-04 Qualcomm Incorporated Communication techniques involving pairwise orthogonality of adjacent rows in LPDC code
CA3169394C (en) * 2017-06-14 2024-01-09 Sas Institute Inc. Distributed data set encryption and decryption
US11921672B2 (en) 2017-07-31 2024-03-05 Splunk Inc. Query execution at a remote heterogeneous data store of a data fabric service
US11989194B2 (en) 2017-07-31 2024-05-21 Splunk Inc. Addressing memory limits for partition tracking among worker nodes
US11151137B2 (en) 2017-09-25 2021-10-19 Splunk Inc. Multi-partition operation in combination operations
US10896182B2 (en) 2017-09-25 2021-01-19 Splunk Inc. Multi-partitioning determination for combination operations
US11144613B2 (en) * 2017-09-29 2021-10-12 Blackberry Limited Platform-independent publish-subscribe
US11170000B2 (en) * 2017-10-20 2021-11-09 Intuit Inc. Parallel map and reduce on hash chains
US10877757B2 (en) * 2017-11-14 2020-12-29 Nvidia Corporation Binding constants at runtime for improved resource utilization
US11334543B1 (en) 2018-04-30 2022-05-17 Splunk Inc. Scalable bucket merging for a data intake and query system
US11138376B2 (en) * 2018-05-21 2021-10-05 State Street Corporation Techniques for information ranking and retrieval
WO2019231761A1 (en) * 2018-05-31 2019-12-05 Secturion Systems, Inc. Locally-stored remote block data integrity
CN108776634B (zh) * 2018-06-07 2021-10-12 北京云测信息技术有限公司 机柜环境异常处理方法及装置
US10552739B1 (en) * 2018-10-15 2020-02-04 Sas Institute Inc. Intelligent data curation
US11604786B2 (en) * 2019-04-26 2023-03-14 EMC IP Holding Company LLC Method and system for processing unstable writes in a clustered file system
WO2020220216A1 (en) 2019-04-29 2020-11-05 Splunk Inc. Search time estimate in data intake and query system
US11715051B1 (en) 2019-04-30 2023-08-01 Splunk Inc. Service provider instance recommendations using machine-learned classifications and reconciliation
WO2020250206A1 (en) * 2019-06-14 2020-12-17 Ailia Sa Method for the execution of an instance of a smart contract by means of a blockchain
US10826801B1 (en) 2019-07-31 2020-11-03 Bank Of America Corporation Multi-level data channel and inspection architectures
US11115310B2 (en) 2019-08-06 2021-09-07 Bank Of America Corporation Multi-level data channel and inspection architectures having data pipes in parallel connections
US11470046B2 (en) 2019-08-26 2022-10-11 Bank Of America Corporation Multi-level data channel and inspection architecture including security-level-based filters for diverting network traffic
US11494380B2 (en) 2019-10-18 2022-11-08 Splunk Inc. Management of distributed computing framework components in a data fabric service system
CN110837516A (zh) * 2019-11-07 2020-02-25 恩亿科(北京)数据科技有限公司 数据割接方法、装置、计算机设备和可读存储介质
WO2021102673A1 (en) 2019-11-26 2021-06-03 Citrix Systems, Inc. Document storage and management
US11221973B2 (en) * 2019-12-20 2022-01-11 Micron Technology, Inc. Parallel iterator for machine learning frameworks
CN111241195B (zh) * 2020-01-02 2023-03-21 腾讯云计算（北京）有限责任公司 分布式系统的数据库处理方法、装置、设备及存储介质
US11922222B1 (en) 2020-01-30 2024-03-05 Splunk Inc. Generating a modified component for a data intake and query system using an isolated execution environment image
CN111522846B (zh) * 2020-04-09 2023-08-22 浙江邦盛科技股份有限公司 一种基于时序中间态数据结构的数据聚合方法
WO2021243594A1 (zh) * 2020-06-03 2021-12-09 铨鸿资讯有限公司 基于集体验证的部分数据验证方法
CN114254034B (zh) * 2020-09-22 2024-05-28 中车株洲电力机车研究所有限公司 基于Redis集群的风电监控对象信息和实时数据存储管理方法
US11704313B1 (en) 2020-10-19 2023-07-18 Splunk Inc. Parallel branch operation using intermediary nodes
CN112751926B (zh) * 2020-12-29 2023-05-26 北京浪潮数据技术有限公司 一种集群中工作节点的管理方法、系统及相关装置
US11483412B2 (en) 2020-12-30 2022-10-25 Blackberry Limited Method for marshalling events in a publish-subscribe system
WO2022250547A1 (en) * 2021-05-28 2022-12-01 Xero Limited Methods and systems for web-based data presentation
CN114024979A (zh) * 2021-10-25 2022-02-08 深圳市高德信通信股份有限公司 一种分布式边缘计算数据存储系统
CN114840608B (zh) * 2022-07-04 2022-11-01 深圳市茗格科技有限公司 分布式数据访问方法、设备及存储介质
US11792262B1 (en) 2022-07-20 2023-10-17 The Toronto-Dominion Bank System and method for data movement

Family Cites Families (70)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5295258A (en) 1989-12-22 1994-03-15 Tandem Computers Incorporated Fault-tolerant computer system with online recovery and reintegration of redundant components
US5271013A (en) 1990-05-09 1993-12-14 Unisys Corporation Fault tolerant computer system
WO1993009494A1 (en) 1991-10-28 1993-05-13 Digital Equipment Corporation Fault-tolerant computer processing using a shadow virtual processor
WO1994018634A1 (en) 1993-02-01 1994-08-18 Lsc, Inc. Archiving file system for data servers in a distributed network environment
US6119151A (en) 1994-03-07 2000-09-12 International Business Machines Corp. System and method for efficient cache management in a distributed file system
US5621885A (en) 1995-06-07 1997-04-15 Tandem Computers, Incorporated System and method for providing a fault tolerant computer program runtime support environment
US5802265A (en) 1995-12-01 1998-09-01 Stratus Computer, Inc. Transparent fault tolerant computer system
US5796934A (en) 1996-05-31 1998-08-18 Oracle Corporation Fault tolerant client server system
US6029168A (en) 1998-01-23 2000-02-22 Tricord Systems, Inc. Decentralized file mapping in a striped network file system in a distributed computing environment
US6173293B1 (en) 1998-03-13 2001-01-09 Digital Equipment Corporation Scalable distributed file system
DE19836347C2 (de) 1998-08-11 2001-11-15 Ericsson Telefon Ab L M Fehlertolerantes Computersystem
US6393583B1 (en) 1998-10-29 2002-05-21 International Business Machines Corporation Method of performing checkpoint/restart of a parallel program
EP1020797B1 (en) 1999-01-11 2004-05-06 Koken Co., Ltd. Fault tolerant computer system
US6556998B1 (en) 2000-05-04 2003-04-29 Matsushita Electric Industrial Co., Ltd. Real-time distributed file system
US6904599B1 (en) 1999-11-29 2005-06-07 Microsoft Corporation Storage management system having abstracted volume providers
US6993764B2 (en) 2000-06-30 2006-01-31 The Regents Of The University Of California Buffered coscheduling for parallel programming and enhanced fault tolerance
US7406484B1 (en) 2000-09-12 2008-07-29 Tbrix, Inc. Storage allocation in a distributed segmented file system
GB2369695B (en) 2000-11-30 2005-03-16 Indigo One Technologies Ltd Database
US6779063B2 (en) 2001-04-09 2004-08-17 Hitachi, Ltd. Direct access storage system having plural interfaces which permit receipt of block and file I/O requests
WO2003013675A1 (en) 2001-08-07 2003-02-20 Rebel Arts Llc Distributed and fault tolerant server system and method
EP1365538A1 (en) 2002-05-23 2003-11-26 Benoit Marchand Implementing a scalable, dynamic, fault-tolerant, multicast based file transfer and asynchronous file replication protocol
US7305585B2 (en) 2002-05-23 2007-12-04 Exludus Technologies Inc. Asynchronous and autonomous data replication
JP4457581B2 (ja) 2003-05-28 2010-04-28 日本電気株式会社 耐障害システム、プログラム並列実行方法、耐障害システムの障害検出装置およびプログラム
US7475274B2 (en) 2004-11-17 2009-01-06 Raytheon Company Fault tolerance and recovery in a high-performance computing (HPC) system
JP4411602B2 (ja) 2004-12-16 2010-02-10 日本電気株式会社 フォールトトレラント・コンピュータシステム
US7657578B1 (en) 2004-12-20 2010-02-02 Symantec Operating Corporation System and method for volume replication in a storage environment employing distributed block virtualization
WO2006121990A2 (en) 2005-05-06 2006-11-16 Marathon Technologies Corporation Fault tolerant computer system
US20060253726A1 (en) 2005-05-06 2006-11-09 Vikas Kukshya Fault-tolerant architecture for a distributed control system
US7484032B2 (en) 2005-06-30 2009-01-27 Smith Roger A Fault tolerant computer system
US8209363B2 (en) 2007-10-09 2012-06-26 Cleversafe, Inc. File system adapted for use with a dispersed data storage network
US7899989B2 (en) 2005-11-04 2011-03-01 Oracle America, Inc. Method and system for using a block allocation policy
US7360063B2 (en) 2006-03-02 2008-04-15 International Business Machines Corporation Method for SIMD-oriented management of register maps for map-based indirect register-file access
US7437373B2 (en) 2006-03-06 2008-10-14 The Real Time Matrix Corporation Method and system for correlating information
US7941404B2 (en) 2006-03-08 2011-05-10 International Business Machines Corporation Coordinated federated backup of a distributed application environment
US7571347B2 (en) 2006-03-20 2009-08-04 Sun Microsystems, Inc. Method and apparatus for providing fault-tolerance in parallel-processing systems
US7657787B2 (en) 2006-04-11 2010-02-02 Hewlett-Packard Development Company, L.P. Method of restoring communication state of process
US8726242B2 (en) 2006-07-27 2014-05-13 Commvault Systems, Inc. Systems and methods for continuous data replication
US8291419B2 (en) 2006-09-26 2012-10-16 International Business Machines Corporation Fault tolerant system for execution of parallel jobs
US7783847B2 (en) 2006-10-31 2010-08-24 Oracle America Inc. Method and system for reallocating blocks in a storage pool
US20080235497A1 (en) 2006-11-26 2008-09-25 Tomblin Jimmy J Parallel Data Output
US20080228771A1 (en) 2006-12-22 2008-09-18 Commvault Systems, Inc. Method and system for searching stored data
US7610510B2 (en) 2007-02-16 2009-10-27 Symantec Corporation Method and apparatus for transactional fault tolerance in a client-server system
US8335813B2 (en) 2007-05-01 2012-12-18 Illinois Institute Of Technology Systems, methods, and protocols for process migration and group membership management
US7779298B2 (en) 2007-06-11 2010-08-17 International Business Machines Corporation Distributed job manager recovery
US20090019258A1 (en) 2007-07-09 2009-01-15 Shi Justin Y Fault tolerant self-optimizing multi-processor system and method thereof
US8239486B2 (en) 2008-03-19 2012-08-07 Oracle International Corporation Direct network file system
US8731190B2 (en) 2009-06-09 2014-05-20 Emc Corporation Segment deduplication system with encryption and compression of segments
EP2275952A1 (en) 2009-07-01 2011-01-19 Thomson Telecom Belgium Method for accessing files of a file system according to metadata and device implementing the method
US9900150B2 (en) 2009-10-30 2018-02-20 International Business Machines Corporation Dispersed storage camera device and method of operation
US20110202329A1 (en) 2010-02-12 2011-08-18 James Howard Goodnight Scenario State Processing Systems And Methods For Operation Within A Grid Computing Environment
BR112012023325A2 (pt) 2010-03-22 2016-05-24 Graco Minnesota Inc módulo de controle de rf para ferramentas de gesso acartonado de alimentação contínua
US8874961B2 (en) 2010-03-22 2014-10-28 Infosys Limited Method and system for automatic failover of distributed query processing using distributed shared memory
CN102222077A (zh) 2010-04-14 2011-10-19 中兴通讯股份有限公司 一种语音频数据的存储方法和装置
US8250405B2 (en) 2010-05-27 2012-08-21 International Business Machines Corporation Accelerating recovery in MPI environments
US9411517B2 (en) 2010-08-30 2016-08-09 Vmware, Inc. System software interfaces for space-optimized block devices
US8458413B2 (en) 2010-10-27 2013-06-04 International Business Machines Corporation Supporting virtual input/output (I/O) server (VIOS) active memory sharing in a cluster environment
US8473783B2 (en) 2010-11-09 2013-06-25 International Business Machines Corporation Fault tolerance in distributed systems
US8271537B2 (en) 2010-11-15 2012-09-18 Sas Institute Inc. Grid computing system alongside a distributed database architecture
US9104326B2 (en) 2010-11-15 2015-08-11 Emc Corporation Scalable block data storage using content addressing
US8984328B2 (en) 2011-03-11 2015-03-17 Microsoft Technology Licensing, Llc Fault tolerance in a parallel database system
US9009567B2 (en) 2011-12-12 2015-04-14 Cleversafe, Inc. Encrypting distributed computing data
US20130291080A1 (en) 2012-04-26 2013-10-31 Appsense Limited Systems and methods for data access protection
US9268796B2 (en) 2012-05-29 2016-02-23 Sas Institute Inc. Systems and methods for quantile estimation in a distributed data system
US9794135B2 (en) * 2013-11-11 2017-10-17 Amazon Technologies, Inc. Managed service for acquisition, storage and consumption of large-scale data streams
US9720989B2 (en) * 2013-11-11 2017-08-01 Amazon Technologies, Inc. Dynamic partitioning techniques for data streams
US9483378B2 (en) * 2014-05-21 2016-11-01 Dynatrace Llc Method and system for resource monitoring of large-scale, orchestrated, multi process job execution environments
US9722974B1 (en) 2014-12-18 2017-08-01 AbeBooks Inc. Automated data re-encryption process in multi-tiered encryption system
GB2540220A (en) 2015-07-06 2017-01-11 Barclays Bank Plc Distributed encryption system and method
US9619148B2 (en) 2015-07-27 2017-04-11 Sas Institute Inc. Distributed data set storage and retrieval
TWI611349B (zh) 2015-12-11 2018-01-11 財團法人資訊工業策進會 檢測系統及其方法

Also Published As

Publication number Publication date
SE540908C2 (en) 2018-12-18
CN108701076A (zh) 2018-10-23
CA2981521A1 (en) 2017-02-02
HK1259159A1 (zh) 2019-11-29
US9619148B2 (en) 2017-04-11
US20170031599A1 (en) 2017-02-02
CN108701076B (zh) 2020-02-21
SE541999C2 (en) 2020-02-11
CA2981521C (en) 2019-07-09
US20170031937A1 (en) 2017-02-02
WO2017019794A1 (en) 2017-02-02
SE540909C2 (en) 2018-12-18
SE1751211A1 (sv) 2017-09-29
IL254745B (en) 2018-02-28
US9703789B2 (en) 2017-07-11
DE112016001075T5 (de) 2018-01-04
US20170031936A1 (en) 2017-02-02
SE1751212A1 (sv) 2017-09-29
US9811524B2 (en) 2017-11-07
IL254745A0 (en) 2017-11-30
US20180075051A1 (en) 2018-03-15
DE112016001075B4 (de) 2024-05-23
US10185721B2 (en) 2019-01-22

Similar Documents

Publication Publication Date Title
SE1751210A1 (sv) Distributed data set storage and retrieval
GB2550800A (en) Managing data in storage according to a log structure
PH12020550738A1 (en) Blockchain-based transaction processing method and apparatus, and electronic device
WO2015066719A3 (en) Use of solid state storage devices and the like in data deduplication
GB2544903A (en) Storing and transferring application data between devices
WO2016113636A3 (en) Secure distributed backup for personal device and cloud data
CL2016000958A1 (es) Métodos de procesamiento de datos de video de múltiples capas para facilitar el acceso aleatorio y el cambio de capa que comprenden generar un archivo que comprende una caja de pista que contiene metadatos, en donde unos datos de medios para la pista comprenden una secuencia de muestras; y dispositivos de video.
EP2905709A3 (en) Method and apparatus for replication of files and file systems using a deduplication key space
CR20150552A (es) Entorno de aprendizaje de idiomas
WO2016109435A3 (en) Encrypted file storage
BR112016021485A2 (pt) Pesquisa de encriptador com base em hash para cópia intrabloco
AR103493A1 (es) Métodos y sistemas para determinar rendimientos en la agricultura
EP3113043A4 (en) Method, device and host for updating metadata stored in columns in distributed file system
BR112014030245A2 (pt) métodos e aparelhos para monitorar apresentações de mídia
CL2017003158A1 (es) Generación de señal de banda alta
GB2510762A (en) A method and device to distribute code and data stores between volatile memory and non-volatile memory
GB2522373A (en) Storage system and method of storing and managing data
SG11201809083TA (en) Routing configuration method for view file, and storage medium, terminal device and apparatus
JP2018133080A5 (ja) データ管理装置、情報処理装置、コンテンツデータの出力方法、コンテンツデータｉｄの共有方法、プログラム、及びデータ構造
IN2013MU02918A (sv)
PH12017550112A1 (en) Techniques to manage bookmarks for media files
TW201612743A (en) Bit group interleave processors, methods, systems, and instructions
TWM476327U (en) Service data record system and point of sale using the same
GB2532667A (en) Memory management
AU2014410705A1 (en) Data processing method and apparatus