EP3811324A4 - Gpu based server in a distributed file system - Google Patents

Gpu based server in a distributed file system Download PDF

Info

Publication number
EP3811324A4
EP3811324A4 EP19822947.8A EP19822947A EP3811324A4 EP 3811324 A4 EP3811324 A4 EP 3811324A4 EP 19822947 A EP19822947 A EP 19822947A EP 3811324 A4 EP3811324 A4 EP 3811324A4
Authority
EP
European Patent Office
Prior art keywords
file system
distributed file
based server
gpu based
gpu
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP19822947.8A
Other languages
German (de)
French (fr)
Other versions
EP3811324A2 (en
Inventor
Maor BEN DAYAN
Omri Palmon
Liran Zvibel
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Weka Io Ltd
Original Assignee
Weka Io Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Weka Io Ltd filed Critical Weka Io Ltd
Publication of EP3811324A2 publication Critical patent/EP3811324A2/en
Publication of EP3811324A4 publication Critical patent/EP3811324A4/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/004—Error avoidance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G06F11/108—Parity data distribution in semiconductor storages, e.g. in SSD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4204—Bus transfer protocol, e.g. handshake; Synchronisation on a parallel bus
    • G06F13/4221—Bus transfer protocol, e.g. handshake; Synchronisation on a parallel bus being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/1847—File system types specifically adapted to static storage, e.g. adapted to flash memory or SSD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/188—Virtual file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2211/00—Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/10—Indexing scheme relating to G06F11/10
    • G06F2211/1002—Indexing scheme relating to G06F11/1076
    • G06F2211/1028—Distributed, i.e. distributed RAID systems with parity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0026—PCI express

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
EP19822947.8A 2018-06-19 2019-06-05 Gpu based server in a distributed file system Pending EP3811324A4 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201862686964P 2018-06-19 2018-06-19
US16/273,749 US11074668B2 (en) 2018-06-19 2019-02-12 GPU based server in a distributed file system
PCT/IB2019/000742 WO2019243892A2 (en) 2018-06-19 2019-06-05 Gpu based server in a distributed file system

Publications (2)

Publication Number Publication Date
EP3811324A2 EP3811324A2 (en) 2021-04-28
EP3811324A4 true EP3811324A4 (en) 2022-03-02

Family

ID=68839444

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19822947.8A Pending EP3811324A4 (en) 2018-06-19 2019-06-05 Gpu based server in a distributed file system

Country Status (4)

Country Link
US (9) US11074668B2 (en)
EP (1) EP3811324A4 (en)
CN (1) CN112262407A (en)
WO (1) WO2019243892A2 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11042661B2 (en) * 2018-06-08 2021-06-22 Weka.IO Ltd. Encryption for a distributed filesystem
US11074668B2 (en) * 2018-06-19 2021-07-27 Weka.IO Ltd. GPU based server in a distributed file system
US11169961B2 (en) * 2018-06-29 2021-11-09 Weka.IO Ltd. Data migration in a distributive file system
CN111104056B (en) 2018-10-25 2021-12-31 华为技术有限公司 Data recovery method, system and device in storage system
US20200319812A1 (en) * 2020-06-03 2020-10-08 Intel Corporation Intermediary for storage command transfers
CN112181916B (en) * 2020-09-14 2024-04-09 北京星辰天合科技股份有限公司 File pre-reading method and device based on user space file system FUSE, and electronic equipment
US11470012B2 (en) 2021-02-25 2022-10-11 Capital One Services, Llc System and method for queue management

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170052847A1 (en) * 2015-08-22 2017-02-23 Weka.IO LTD Distributed Erasure Coded Virtual File System
US20170147516A1 (en) * 2015-11-19 2017-05-25 HGST Netherlands B.V. Direct interface between graphics processing unit and data storage unit

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6138126A (en) * 1995-05-31 2000-10-24 Network Appliance, Inc. Method for allocating files in a file system integrated with a raid disk sub-system
US6487636B1 (en) 2000-04-24 2002-11-26 Hewlett-Packard Co. Method and apparatus for mapping data in a heterogeneous disk array storage system
US7681072B1 (en) 2004-08-13 2010-03-16 Panasas, Inc. Systems and methods for facilitating file reconstruction and restoration in data storage systems where a RAID-X format is implemented at a file level within a plurality of storage devices
US7447839B2 (en) * 2005-09-13 2008-11-04 Yahoo! Inc. System for a distributed column chunk data store
US7793146B1 (en) 2007-08-07 2010-09-07 Panasas, Inc. Methods for storing data in a data storage system where a RAID-X format or formats are implemented at a file level
US8645749B2 (en) 2009-02-04 2014-02-04 Micron Technology, Inc. Systems and methods for storing and recovering controller data in non-volatile memory devices
US8458287B2 (en) 2009-07-31 2013-06-04 Microsoft Corporation Erasure coded storage aggregation in data centers
US10838646B2 (en) * 2011-07-28 2020-11-17 Netlist, Inc. Method and apparatus for presearching stored data
US9430324B2 (en) 2013-05-24 2016-08-30 Rambus Inc. Memory repair method and apparatus based on error code tracking
US11182284B2 (en) * 2013-11-07 2021-11-23 Netlist, Inc. Memory module having volatile and non-volatile memory subsystems and method of operation
US10635316B2 (en) * 2014-03-08 2020-04-28 Diamanti, Inc. Methods and systems for data storage using solid state drives
US10372685B2 (en) * 2014-03-31 2019-08-06 Amazon Technologies, Inc. Scalable file storage service
US10264071B2 (en) * 2014-03-31 2019-04-16 Amazon Technologies, Inc. Session management in distributed storage systems
US9274713B2 (en) 2014-04-03 2016-03-01 Avago Technologies General Ip (Singapore) Pte. Ltd. Device driver, method and computer-readable medium for dynamically configuring a storage controller based on RAID type, data alignment with a characteristic of storage elements and queue depth in a cache
US9087012B1 (en) 2014-06-04 2015-07-21 Pure Storage, Inc. Disaster recovery at high reliability in a storage cluster
US9582363B2 (en) 2014-06-09 2017-02-28 International Business Machines Corporation Failure domain based storage system data stripe layout
US9921910B2 (en) 2015-02-19 2018-03-20 Netapp, Inc. Virtual chunk service based data recovery in a distributed data storage system
US20170004131A1 (en) * 2015-07-01 2017-01-05 Weka.IO LTD Virtual File System Supporting Multi-Tiered Storage
US20170337103A1 (en) 2016-05-19 2017-11-23 Intel Corporation Method and apparatus to provide data redundancy in a solid-state drive
TWI616759B (en) * 2016-08-10 2018-03-01 創義達科技股份有限公司 Apparatus assigning controller and apparatus assigning method
US10496319B2 (en) * 2017-02-28 2019-12-03 Sap Se Lifecycle management for data in non-volatile memory including blocking creation of a database savepoint and associating non-volatile memory block identifiers with database column fragments
US10489881B2 (en) * 2017-06-30 2019-11-26 H3 Platform Inc. Direct memory access for co-processor memory
US11074668B2 (en) * 2018-06-19 2021-07-27 Weka.IO Ltd. GPU based server in a distributed file system
US10929229B2 (en) * 2018-06-21 2021-02-23 International Business Machines Corporation Decentralized RAID scheme having distributed parity computation and recovery
US10825477B2 (en) * 2018-08-02 2020-11-03 Western Digital Technologies, Inc. RAID storage system with logical data group priority
US11150805B2 (en) * 2019-05-02 2021-10-19 Vast Data Ltd. System and method for using free space to improve erasure code locality

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170052847A1 (en) * 2015-08-22 2017-02-23 Weka.IO LTD Distributed Erasure Coded Virtual File System
US20170147516A1 (en) * 2015-11-19 2017-05-25 HGST Netherlands B.V. Direct interface between graphics processing unit and data storage unit

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "NVIDIA GPUDirect | NVIDIA Developer", 8 November 2017 (2017-11-08), pages 1 - 6, XP055883181, Retrieved from the Internet <URL:https://web.archive.org/web/20171108030424/https://developer.nvidia.com/gpudirect> [retrieved on 20220124] *
BHATOTIA PRAMOD KUMAR: "Incremental parallel and distributed systems", 1 April 2015 (2015-04-01), pages 1 - 193, XP055883403, Retrieved from the Internet <URL:https://www.mpi-sws.org/tr/2015-002.pdf> [retrieved on 20220125], DOI: 10.22028/d291-25419 *
ZVIBEL LIRAN: "Making Machine Learning Compute Bound Again", 20 February 2018 (2018-02-20), pages 1 - 34, XP055883180, Retrieved from the Internet <URL:https://www.hpcadvisorycouncil.com/events/2018/stanford-workshop/pdf/DayOne_Tues20Feb2018/Zvibel_WekaIO_ComputeBoundML_DayOne_20Feb2018.pdf> [retrieved on 20220124] *

Also Published As

Publication number Publication date
US20210248713A1 (en) 2021-08-12
US20230214956A1 (en) 2023-07-06
US11074668B2 (en) 2021-07-27
US11922537B2 (en) 2024-03-05
US20220076371A1 (en) 2022-03-10
US11205244B2 (en) 2021-12-21
US11544812B2 (en) 2023-01-03
US20240046406A1 (en) 2024-02-08
US11151686B2 (en) 2021-10-19
WO2019243892A2 (en) 2019-12-26
US20210248711A1 (en) 2021-08-12
CN112262407A (en) 2021-01-22
US11798124B2 (en) 2023-10-24
US20210248712A1 (en) 2021-08-12
US20220335564A1 (en) 2022-10-20
US20240202863A1 (en) 2024-06-20
US20190385271A1 (en) 2019-12-19
WO2019243892A3 (en) 2020-06-11
EP3811324A2 (en) 2021-04-28
US11145026B2 (en) 2021-10-12

Similar Documents

Publication Publication Date Title
EP3811324A4 (en) Gpu based server in a distributed file system
EP3844963A4 (en) Improved attribute support in point cloud coding
EP3837674A4 (en) A cross reality system
EP3340573A4 (en) Distributed storage-based file delivery system and method
EP3606584A4 (en) Insulin-on-board accounting in an artificial pancreas system
EP3568777A4 (en) Data sharing in a multi-tenant database system
EP3710952A4 (en) Metadata journal in a distributed storage system
EP3332328A4 (en) Distributed erasure coded virtual file system
EP3817531C0 (en) Server system
EP3400550A4 (en) Methods and systems for securing data in the public cloud
EP3274986A4 (en) Virtual 3d methods, systems and software
EP3371731A4 (en) Digital content delivery system
EP3272114A4 (en) System for distributing metadata embedded in video
EP3155519A4 (en) System and method for optimizing the selection of cloud services based on price and performance
EP3394828A4 (en) Tone mastering system with creative intent metadata
EP3127019A4 (en) Namespace management in distributed storage systems
EP3665561A4 (en) A metadata control in a load-balanced distributed storage system
EP3710944A4 (en) File operations in a distributed storage system
EP3851971A4 (en) Verification system and verification server
EP3659094A4 (en) Technologies for automatically validating the functionality of offers in a cloud service brokerage system
EP3420449A4 (en) Managing updates in a computing system using multiple access methods
EP3726491A4 (en) Central server and dramatic performance system including same
EP3323078A4 (en) Method and system for accessing a file
EP3529721A4 (en) Cluster file replication
EP3467761A4 (en) Hearse dispatch server

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20201218

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20220202

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 16/182 20190101ALI20220127BHEP

Ipc: G06F 11/10 20060101ALI20220127BHEP

Ipc: G06F 16/188 20190101ALI20220127BHEP

Ipc: G06F 16/18 20190101ALI20220127BHEP

Ipc: G06F 11/00 20060101ALI20220127BHEP

Ipc: G06T 1/60 20060101AFI20220127BHEP

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230516

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230928