WO2015055502A3 - Method of partitioning storage in a distributed data storage system and corresponding device - Google Patents

Method of partitioning storage in a distributed data storage system and corresponding device Download PDF

Info

Publication number
WO2015055502A3
WO2015055502A3 PCT/EP2014/071658 EP2014071658W WO2015055502A3 WO 2015055502 A3 WO2015055502 A3 WO 2015055502A3 EP 2014071658 W EP2014071658 W EP 2014071658W WO 2015055502 A3 WO2015055502 A3 WO 2015055502A3
Authority
WO
WIPO (PCT)
Prior art keywords
partition
data items
partitioning
storage system
assigned
Prior art date
Application number
PCT/EP2014/071658
Other languages
French (fr)
Other versions
WO2015055502A2 (en
Inventor
Erwan Le Merrer
Gilles Tredan
Yizhong Liang
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Publication of WO2015055502A2 publication Critical patent/WO2015055502A2/en
Publication of WO2015055502A3 publication Critical patent/WO2015055502A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24532—Query optimisation of parallel queries

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A method and device for efficient partitioning of storage of big data in a distributed storage system, capable of handling input streams of data. Related data items that are received from the input data stream, are assigned to a partition if they are not already part of any partition; if any of the related data items is already in a partition, the other one of the related data items is assigned to the partition if there is enough place in the partition. If both related data items do not already exist in any partition, they are assigned to the partition in the distributed data storage system that has the lowest number of data items. Thus, on-line partitioning takes place as data items arrive, allowing the partitioning to be well-balanced and limiting the creation of edges between partitions.
PCT/EP2014/071658 2013-10-18 2014-10-09 Method of partitioning storage in a distributed data storage system and corresponding device WO2015055502A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
EP13306438 2013-10-18
EP13306438.6 2013-10-18
EP14305601.8 2014-04-24
EP14305601 2014-04-24

Publications (2)

Publication Number Publication Date
WO2015055502A2 WO2015055502A2 (en) 2015-04-23
WO2015055502A3 true WO2015055502A3 (en) 2015-06-18

Family

ID=51663209

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2014/071658 WO2015055502A2 (en) 2013-10-18 2014-10-09 Method of partitioning storage in a distributed data storage system and corresponding device

Country Status (1)

Country Link
WO (1) WO2015055502A2 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10635645B1 (en) 2014-05-04 2020-04-28 Veritas Technologies Llc Systems and methods for maintaining aggregate tables in databases
US10025804B2 (en) 2014-05-04 2018-07-17 Veritas Technologies Llc Systems and methods for aggregating information-asset metadata from multiple disparate data-management systems
US20220019370A1 (en) * 2020-07-16 2022-01-20 Micron Technology, Inc. Partial zone memory unit handling in a zoned namespace of a memory device
CN114581221B (en) * 2022-05-05 2022-07-29 支付宝(杭州)信息技术有限公司 Distributed computing system and computer device

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
ISABELLE STANTON ET AL: "Streaming graph partitioning for large distributed graphs", KNOWLEDGE DISCOVERY AND DATA MINING, ACM, 2 PENN PLAZA, SUITE 701 NEW YORK NY 10121-0701 USA, 12 August 2012 (2012-08-12), pages 1222 - 1230, XP058007826, ISBN: 978-1-4503-1462-6, DOI: 10.1145/2339530.2339722 *
KORANNE S: "A distributed algorithm for k-way graph partitioning", EUROMICRO CONFERENCE, 1999. PROCEEDINGS. 25TH MILAN, ITALY 8-10 SEPT. 1999, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, US, vol. 2, 8 September 1999 (1999-09-08), pages 446 - 448, XP010352293, ISBN: 978-0-7695-0321-9 *
WIKIPEDIA: "Distributed data store", 9 July 2013 (2013-07-09), XP002738277, Retrieved from the Internet <URL:http://en.wikipedia.org/w/index.php?title=Distributed_data_store&oldid=563562257> [retrieved on 20150410] *
YEN-CHUEN WEI ET AL: "RATIO CUT PARTITIONING FOR HIERARCHICAL DESIGNS", IEEE TRANSACTIONS ON COMPUTER AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 10, no. 7, 1 July 1991 (1991-07-01), pages 911 - 921, XP000240892, ISSN: 0278-0070, DOI: 10.1109/43.87601 *

Also Published As

Publication number Publication date
WO2015055502A2 (en) 2015-04-23

Similar Documents

Publication Publication Date Title
WO2016094785A3 (en) Multiple transaction logs in a distributed storage system
EP4286516A3 (en) Partition processing methods and systems
WO2014165439A3 (en) Automated storage and retrieval system and control system thereof
WO2013009503A3 (en) Query execution systems and methods
WO2015066061A3 (en) Systems, methods, and media for content management and sharing
WO2011115841A3 (en) Reorganization of data under continuous workload
WO2016018472A3 (en) Content-based association of device to user
WO2017117349A3 (en) Cell separation devices, systems, and methods
WO2016022822A3 (en) Knowledge automation system
WO2015119691A3 (en) Client-configurable security options for data streams
WO2015073771A3 (en) Methods, systems and computer program products for using a distributed associative memory base to determine data correlations and convergence therein
WO2016044692A3 (en) Storing and transferring application data between devices
WO2014047218A3 (en) Table format for map reduce system
WO2014025705A3 (en) Search result ranking and presentation
WO2016110453A8 (en) A crispr-cas system for a filamentous fungal host cell
EP3688596A4 (en) Computer program product, system, and method to manage access to storage resources from multiple applications
WO2011116087A3 (en) Highly scalable and distributed data de-duplication
WO2014179418A3 (en) Search intent for queries on online social networks
WO2015044696A8 (en) Computer architecture and processing method
WO2012068449A8 (en) Control node for a processing cluster
WO2014124336A3 (en) Partitioning and processing of analytes and other species
WO2011129874A3 (en) Boot partitions in memory devices and systems
WO2014137449A3 (en) A method and system for privacy preserving counting
WO2012054089A3 (en) Distributed processing pipeline and distributed layered application processing
WO2012082557A3 (en) Dynamic work partitioning on heterogeneous processing devices

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14781591

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14781591

Country of ref document: EP

Kind code of ref document: A2