CN110678845B - 用于大数据分析的分布式文件系统中的多租户数据服务 - Google Patents

用于大数据分析的分布式文件系统中的多租户数据服务 Download PDF

Info

Publication number
CN110678845B
CN110678845B CN201880035218.7A CN201880035218A CN110678845B CN 110678845 B CN110678845 B CN 110678845B CN 201880035218 A CN201880035218 A CN 201880035218A CN 110678845 B CN110678845 B CN 110678845B
Authority
CN
China
Prior art keywords
tenant
read
directory
node
computer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201880035218.7A
Other languages
English (en)
Chinese (zh)
Other versions
CN110678845A (zh
Inventor
郑勇
袁正才
冯添
王昕�
包小明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of CN110678845A publication Critical patent/CN110678845A/zh
Application granted granted Critical
Publication of CN110678845B publication Critical patent/CN110678845B/zh
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/188—Virtual file systems
    • G06F16/192—Implementing virtual folder structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45579—I/O management, e.g. providing access to device drivers or storage

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Multi Processors (AREA)
CN201880035218.7A 2017-06-29 2018-06-14 用于大数据分析的分布式文件系统中的多租户数据服务 Expired - Fee Related CN110678845B (zh)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US15/636,770 2017-06-29
US15/636,770 US20190005066A1 (en) 2017-06-29 2017-06-29 Multi-tenant data service in distributed file systems for big data analysis
US15/824,356 US20190005067A1 (en) 2017-06-29 2017-11-28 Multi-tenant data service in distributed file systems for big data analysis
US15/824,356 2017-11-28
PCT/IB2018/054378 WO2019003029A1 (en) 2017-06-29 2018-06-14 TENANT DATA SERVICE IN DISTRIBUTED FILE SYSTEMS FOR MEGADATA ANALYSIS

Publications (2)

Publication Number Publication Date
CN110678845A CN110678845A (zh) 2020-01-10
CN110678845B true CN110678845B (zh) 2023-05-12

Family

ID=64738079

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880035218.7A Expired - Fee Related CN110678845B (zh) 2017-06-29 2018-06-14 用于大数据分析的分布式文件系统中的多租户数据服务

Country Status (6)

Country Link
US (2) US20190005066A1 (enExample)
JP (1) JP7160442B2 (enExample)
CN (1) CN110678845B (enExample)
DE (1) DE112018001972T5 (enExample)
GB (1) GB2578077B (enExample)
WO (1) WO2019003029A1 (enExample)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11327992B1 (en) 2018-04-30 2022-05-10 Splunk Inc. Authenticating a user to access a data intake and query system
US11275733B1 (en) * 2018-04-30 2022-03-15 Splunk Inc. Mapping search nodes to a search head using a tenant identifier
US11892996B1 (en) 2019-07-16 2024-02-06 Splunk Inc. Identifying an indexing node to process data using a resource catalog
US11157497B1 (en) 2018-04-30 2021-10-26 Splunk Inc. Dynamically assigning a search head and search nodes for a query
CN110187838B (zh) * 2019-05-30 2023-06-20 北京百度网讯科技有限公司 数据io信息处理方法、分析方法、装置和相关设备
US11416465B1 (en) 2019-07-16 2022-08-16 Splunk Inc. Processing data associated with different tenant identifiers
CN110688674B (zh) * 2019-09-23 2024-04-26 中国银联股份有限公司 一种访问对接器、系统及应用该访问对接器的方法及装置
US11829415B1 (en) 2020-01-31 2023-11-28 Splunk Inc. Mapping buckets and search peers to a bucket map identifier for searching
US11449371B1 (en) 2020-07-31 2022-09-20 Splunk Inc. Indexing data at a data intake and query system based on a node capacity threshold
US11615082B1 (en) 2020-07-31 2023-03-28 Splunk Inc. Using a data store and message queue to ingest data for a data intake and query system
US12321396B1 (en) 2020-07-31 2025-06-03 Splunk Inc. Generating and storing aggregate data slices in a remote shared storage system
US11609913B1 (en) 2020-10-16 2023-03-21 Splunk Inc. Reassigning data groups from backup to searching for a processing node
US11809395B1 (en) 2021-07-15 2023-11-07 Splunk Inc. Load balancing, failover, and reliable delivery of data in a data intake and query system
KR20230070718A (ko) * 2021-11-15 2023-05-23 에스케이하이닉스 주식회사 테넌트를 식별하는 키-값 기반의 데이터 저장 장치 및 그 동작 방법
US12373414B1 (en) 2023-01-31 2025-07-29 Splunk Inc. Reassigning data groups based on activation of a processing node
US12164402B1 (en) 2023-01-31 2024-12-10 Splunk Inc. Deactivating a processing node based on assignment of a data group assigned to the processing node

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000148565A (ja) * 1998-11-13 2000-05-30 Hitachi Ltd 異種オペレーティングシステムファイル共用方法およびファイル共用システム
CN103617199A (zh) * 2013-11-13 2014-03-05 北京京东尚科信息技术有限公司 一种操作数据的方法和系统
CN104050201A (zh) * 2013-03-15 2014-09-17 伊姆西公司 用于多租户分布式环境中的数据管理的方法和设备
CN106202452A (zh) * 2016-07-15 2016-12-07 复旦大学 大数据平台的统一数据资源管理系统与方法

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007188209A (ja) * 2006-01-12 2007-07-26 Seiko Epson Corp ネットワークに接続される制御装置内に格納されたファイルに対するアクセス
JP4327869B2 (ja) * 2007-07-26 2009-09-09 株式会社日立製作所 分散ファイルシステム、分散ファイルシステムサーバ及び分散ファイルシステムへのアクセス方法
JP5365502B2 (ja) * 2009-12-24 2013-12-11 富士通株式会社 ファイル管理装置、ファイル管理プログラム、およびファイル管理方法
US8386431B2 (en) * 2010-06-14 2013-02-26 Sap Ag Method and system for determining database object associated with tenant-independent or tenant-specific data, configured to store data partition, current version of the respective convertor
RU2598812C2 (ru) * 2011-01-21 2016-09-27 Томсон Лайсенсинг Способ усовершенствования выполнения операции обратно совместимой агрегатной файловой системы и соответственное устройство
US20140007189A1 (en) * 2012-06-28 2014-01-02 International Business Machines Corporation Secure access to shared storage resources
US9348652B2 (en) * 2012-07-02 2016-05-24 Vmware, Inc. Multi-tenant-cloud-aggregation and application-support system
US9542400B2 (en) * 2012-09-07 2017-01-10 Oracle International Corporation Service archive support
US9727578B2 (en) * 2012-09-28 2017-08-08 International Business Machines Corporation Coordinated access to a file system's shared storage using dynamic creation of file access layout
TWI490716B (zh) * 2012-12-07 2015-07-01 Ind Tech Res Inst 多租戶應用程序的開發方法、資料存取方法與系統
US9069778B1 (en) * 2012-12-28 2015-06-30 Emc Corporation Cloud object store for archive storage of high performance computing data using decoupling middleware
US9619545B2 (en) * 2013-06-28 2017-04-11 Oracle International Corporation Naïve, client-side sharding with online addition of shards
US9571356B2 (en) * 2013-09-27 2017-02-14 Zettaset, Inc. Capturing data packets from external networks into high availability clusters while maintaining high availability of popular data packets
US10642800B2 (en) * 2013-10-25 2020-05-05 Vmware, Inc. Multi-tenant distributed computing and database
CN105052104B (zh) * 2013-12-02 2018-08-14 华为技术有限公司 文件的处理方法、装置及系统
DE102013114214A1 (de) * 2013-12-17 2015-06-18 Fujitsu Technology Solutions Intellectual Property Gmbh POSIX-kompatibles Dateisystem, Verfahren zum Erzeugen einer Dateiliste und Speichervorrichtung
US9661064B2 (en) * 2014-01-24 2017-05-23 Ca, Inc. Systems and methods for deploying legacy software in the cloud
JP2015219852A (ja) * 2014-05-21 2015-12-07 キヤノン株式会社 情報処理装置、情報処理装置の制御方法、及びプログラム
US9756135B2 (en) * 2014-07-31 2017-09-05 Ca, Inc. Accessing network services from external networks
US9721117B2 (en) * 2014-09-19 2017-08-01 Oracle International Corporation Shared identity management (IDM) integration in a multi-tenant computing environment
US9762672B2 (en) * 2015-06-15 2017-09-12 International Business Machines Corporation Dynamic node group allocation
US10212078B2 (en) * 2015-07-09 2019-02-19 International Business Machines Corporation Enabling network services in multi-tenant IAAS environment
US9811386B2 (en) * 2015-10-23 2017-11-07 Oracle International Corporation System and method for multitenant execution of OS programs invoked from a multitenant middleware application
US10884992B2 (en) * 2015-12-23 2021-01-05 EMC IP Holding Company LLC Multi-stream object-based upload in a distributed file system
CN109643242B (zh) * 2016-05-23 2023-06-27 摩根大通国家银行 用于多租户hadoop集群的安全设计和架构

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000148565A (ja) * 1998-11-13 2000-05-30 Hitachi Ltd 異種オペレーティングシステムファイル共用方法およびファイル共用システム
CN104050201A (zh) * 2013-03-15 2014-09-17 伊姆西公司 用于多租户分布式环境中的数据管理的方法和设备
CN103617199A (zh) * 2013-11-13 2014-03-05 北京京东尚科信息技术有限公司 一种操作数据的方法和系统
CN106202452A (zh) * 2016-07-15 2016-12-07 复旦大学 大数据平台的统一数据资源管理系统与方法

Also Published As

Publication number Publication date
JP7160442B2 (ja) 2022-10-25
US20190005067A1 (en) 2019-01-03
US20190005066A1 (en) 2019-01-03
GB2578077B (en) 2020-09-16
JP2020525909A (ja) 2020-08-27
GB202000838D0 (en) 2020-03-04
GB2578077A (en) 2020-04-15
CN110678845A (zh) 2020-01-10
DE112018001972T5 (de) 2019-12-24
WO2019003029A1 (en) 2019-01-03

Similar Documents

Publication Publication Date Title
CN110678845B (zh) 用于大数据分析的分布式文件系统中的多租户数据服务
US9244951B2 (en) Managing tenant-specific data sets in a multi-tenant environment
US9135342B2 (en) Comprehensive, relevant, and dynamic data searching in a virtualization environment
US11100106B1 (en) Query engine virtualization
WO2020132342A1 (en) Discovery of hyper-converged infrastructure devices
US10656972B2 (en) Managing idempotent operations while interacting with a system of record
US20220066837A1 (en) System and method for an efficient and scalable multitenant implementation for content management services platforms, including cloud deployed content management services platforms
US11716406B2 (en) Data caching for cloud services
US10262046B2 (en) Profile-enabled dynamic runtime environment for web application servers
WO2018119601A1 (zh) 数据转换方法及备份服务器
US10310881B2 (en) Compositing data model information across a network
CN105637471B (zh) 用于对存储环境进行监测和控制的方法及设备
US11016946B1 (en) Method and apparatus for processing object metadata
EP4122167B1 (en) Systems and methods for detecting and resolving conflicts in internet services
US9229753B2 (en) Autonomic customization of properties of a virtual appliance in a computer system
US12282484B2 (en) Federation of data during query time in computing systems
Schapranow et al. A federated in-memory database system for life sciences
US20250200012A1 (en) Network file system server proxy and protocol translation
US20220342720A1 (en) Method and system for managing elastic accelerator resource pools with a shared storage
Sharma et al. Accessing E-Infrastructures Using Cdac Scientific Cloud (Csc) Services

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20230512