US11080244B2 - Inter-version mapping of distributed file systems - Google Patents
Inter-version mapping of distributed file systems Download PDFInfo
- Publication number
- US11080244B2 US11080244B2 US14/288,506 US201414288506A US11080244B2 US 11080244 B2 US11080244 B2 US 11080244B2 US 201414288506 A US201414288506 A US 201414288506A US 11080244 B2 US11080244 B2 US 11080244B2
- Authority
- US
- United States
- Prior art keywords
- data
- lspf
- file system
- version
- distributed file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44536—Selecting among different versions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/541—Interprogram communication via adapters, e.g. between incompatible applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45579—I/O management, e.g. providing access to device drivers or storage
Definitions
- aspects of the disclosure are related to computing hardware and software technology, and in particular to inter-version mapping of distributed file systems.
- a virtualized environment In addition to the advances in distributed applications and frameworks, virtualization techniques have gained popularity and are now commonplace in data centers and other environments in which it is useful to increase the efficiency with which computing resources are used.
- one or more virtual machines are instantiated on an underlying computer (or another virtual machine) and share the resources of the underlying computer.
- a cluster may employ a plurality of virtual machines per computing device to more efficiently distribute the workload.
- a method to provide data includes receiving an access request from a LSPF node to access data in accordance with a version of a distributed file system. The method further includes, responsive to the access request, accessing the data for the LSPF node in accordance with a different version of the distributed file system, and presenting the data to the LSPF node in accordance with the version of the distributed file system used by the LSPF node.
- LSPF large-scale processing framework
- a computer apparatus to provide data to a LSPF node in a LSPF cluster includes processing instructions that direct a data service computing system, when executed by the data service computing system, to receive an access request from the LSPF node to access data in accordance with a version of a distributed file system.
- the processing instructions further direct the data service computing system to, responsive to the access request, access the data for the LSPF node in accordance with a different version of the distributed file system, and present the data to the LSPF node in accordance with the version of the distributed file system used by the LSPF node.
- the computer apparatus also includes one or more non-transitory computer readable media that store the processing instructions.
- a system to provide data to a LSPF node in a LSPF cluster includes the LSPF node configured to initiate an access request to access data in accordance with a version of a distributed file system.
- the system further includes a data service configured to identify the access request and, responsive to the access request, access the data for the LSPF node in accordance with a different version of the distributed file system.
- the data service is further configured to present the data to the LSPF node in accordance with the version of the distributed file system used by the LSPF node.
- FIG. 1 illustrates a computing environment for providing data to large-scale processing framework nodes in a large-scale processing framework cluster.
- FIG. 2 illustrates a method to provide data to a large-scale processing framework node in a large-scale processing framework cluster.
- FIG. 3 illustrates a virtual large-scale processing framework cluster.
- FIG. 4 illustrates an overview for providing data to a large-scale processing framework node in a large-scale processing framework cluster.
- FIG. 5 illustrates an overview for providing data to a large-scale processing framework node in a virtual large-scale processing framework cluster.
- FIG. 6 illustrates a computing system for providing data to a large-scale processing framework node in a large-scale processing framework cluster.
- FIG. 7 illustrates an overview for providing data to large-scale processing framework nodes in multiple virtual large-scale processing framework clusters.
- LSPF large-scale processing framework
- applications and frameworks have been developed to process vast amounts of data from storage volumes using one or more processing systems.
- processing systems may include real processing systems, such as server computers, desktop computers, and the like, as well as virtual machines within these real or host processing systems.
- one or more virtual machines are instantiated within a host environment.
- the virtual machines may be instantiated by a hypervisor running in the host environment, which may run with or without an operating system beneath it.
- the hypervisor may be implemented at a layer above the host operating system, while in other implementations the hypervisor may be integrated with the operating system.
- Other hypervisor configurations are possible and may be considered within the scope of the present disclosure.
- the virtual machines executed by the hypervisor may include various guest elements, such as a guest operating system and its components, guest applications, and the like, that consume and execute processes on data.
- the virtual machines may also include virtual representations of various computing components, such as guest memory, a guest storage system, and a guest processor.
- a guest element running within the virtual machine may require data for processing.
- This application or framework is used to access data from one or more storage volumes, and process the data in parallel with one or more other real or virtual computing nodes.
- a LSPF cluster may contain one or more nodes that are capable of processing data using a particular version of a distributed file system. Accordingly, when the nodes within the LSPF cluster require data, the nodes may initiate an access request using the particular distributed file system. Responsive to this request, a data service process may identify the access requests, and access the appropriate data using a different version of the distributed file system. In turn, the data service process may present the data to the LSPF nodes in accordance with the particular version of the distributed file system used by the nodes in the LSPF cluster.
- FIG. 1 illustrates a computing environment 100 to provide data to LSPF nodes in a LSPF cluster.
- Computing environment 100 includes data repository 110 , LSPF cluster 120 , and data service 160 .
- LSPF cluster 120 further includes LSPF nodes 121 - 123 , which use a first distributed file system (DFS) version to request data from data repository 110 .
- DFS distributed file system
- LSPF cluster 120 might include any number of nodes to support the parallel processing of large data sets.
- LSPF nodes 121 - 123 may represent real or virtual machines that are initiated by an administrator or some other management entity to process large amounts of data in parallel with the other nodes.
- each of LSPF nodes 121 - 123 may be configured with a processing or map-reduce framework, such as Hadoop, Spark, Disco, or some other data processing framework.
- the nodes may require data for processing from data repository 110 .
- the data within data repository 110 is accessed using a different distributed file system version than what is used by the various nodes in LSPF cluster 120 .
- data service 160 which is responsible for gathering the data for processing, provides the content in an acceptable form for the various nodes.
- FIG. 2 illustrates a method 200 to provide data to a LSPF node in a LSPF cluster.
- method 200 includes receiving an access request from a LSPF node to access data in accordance with a version of a distributed file system ( 201 ). Responsive to the request, the method includes accessing the data in accordance with a different version of the distributed file system ( 203 ), and presenting the data to the LSPF node in accordance with the version of the distributed file system used by the LSPF node ( 205 ).
- one or more LSPF nodes may be initiated within an LSPF cluster to process large data sets.
- the nodes may require data from a storage repository, such as data repository 110 .
- node 121 may request data using a first DFS version. Responsive to the request, data service 160 , or another similar process between node 121 and data repository 110 , accesses the requested data for node 121 .
- the distributed file system version used for the node is not equivalent to the distributed file system version used to access the data in data repository 110 . Accordingly, the data that is provided to the node must be presented or translated in accordance with the file system version for the node.
- LSPF node 121 uses the first DFS version to request a first data item. Responsive to the request, data service 160 identifies and accesses the first data item in accordance with a second DFS version. Once the item is accessed in accordance with the second DFS version, the first data item is provided to node 121 according to the first DFS version. Thus, although the data item is stored and accessed using a different DFS version in data repository 110 , data service 160 supplies the item to the node in accordance with the DFS version for that node.
- LSPF nodes 121 - 123 comprise real or virtual computing nodes capable of cooperative operation in LSPF cluster 120 .
- LSPF nodes 121 - 123 may each include real or virtual processing systems, storage systems, communication interfaces, user interfaces, as well as other computing elements.
- LSPF nodes 121 - 123 may operate on one or more host computing devices and hypervisors that are used to abstract the physical components of the host machine and provide them to virtual nodes. Accordingly, multiple LSPF nodes may be implemented on the same host machine, allowing the nodes to share the physical resources of the host system.
- Data service 160 comprises one or more computing systems capable of providing data from data repository 110 to LSPF nodes 121 - 123 . Although illustrated separate from LSPF nodes 121 - 123 , it should be understood that data service 160 might be implemented wholly or partially on the same devices responsible for LSPF nodes 121 - 123 . Data service 160 may comprise a processing system, a storage system, communication interfaces, user interfaces, or any other similar computing element.
- Data repository 110 includes any storage medium capable of storing data for processing in LSPF cluster 120 .
- Data repository 110 may include disk drives, solid state drives, or any other similar storage media.
- data repository 110 includes a variety of data stored using a variety of different distributed file system versions, such as versions of the Hadoop distributed file system, versions of the Google file system, versions of the Gluster file system, or any other distributed file system version—including combinations thereof. Accordingly, data service 160 may be used to provide content to nodes that use a different DFS version than data that is stored in data repository 110 .
- FIG. 3 illustrates a virtual LSPF cluster 300 according to one example.
- LSPF cluster 300 includes hosts 301 - 302 and data repository 380 .
- Hosts 301 - 302 further include data services 360 - 361 and hypervisor 350 - 351 , which are used to operate virtual machines (VMs) 321 - 324 .
- VMs 321 - 324 further include job 371 , Hadoop instances 331 - 334 and distributed file system (DFS) 340 .
- DFS distributed file system
- hypervisors 350 - 351 are used to abstract the components from hosts 301 - 302 and provide the components to VMs 321 - 324 , which allows VMs 321 - 324 to execute as an LSPF cluster.
- VMs 321 - 324 include Hadoop instances 331 - 334 that are used to provide map reduce functionality for job 371 within cluster 300 .
- Hadoop instances 331 - 334 use DFS 340 to identify data for processing within each of the individual nodes, although the data requested may be stored using an alternative version of the DFS.
- data services 360 - 361 receive access requests from VMs 321 - 324 to access data in accordance with the DFS version for DFS 340 . Responsive to the requests, data services 360 - 361 access the data in accordance with a second version of the DFS, and present the data to VMs 321 - 324 in accordance with the DFS version for DFS 340 .
- DFS 340 may use a first version of the Hadoop DFS, however, the data to be accessed may use a second version of Hadoop DFS.
- data services 360 - 361 may act as intermediaries to access and present the data to the VMs.
- an LSPF node may request data using a Hadoop DFS version, but the data service may access the data using a Google file system version, or some other distributed file system version.
- FIG. 4 illustrates an overview for providing data to LSPF nodes in an LSPF cluster.
- FIG. 4 includes data repository 410 , LSPF cluster 420 , and data service 460 .
- LSPF cluster 420 further includes LSPF nodes 421 - 423 that execute first Hadoop version instances 431 - 433 .
- Data repository 410 further includes second DFS version 415 that is used to store data 416 .
- LSPF cluster 420 In operation, an administrator or some other similar management process initiates LSPF cluster 420 to process data sets stored in data repository 410 .
- LSPF nodes 421 - 423 within LSPF cluster 420 include a first version of the Hadoop framework that accesses data using a first DFS version.
- data as it is stored in data repository 410 is accessible using a second DFS version 415 . Accordingly, access requests for the data must be identified in accordance with the first DFS version and the data must be accessed using the second DFS version.
- first Hadoop version instance 431 initiates a data access request using a first version of a distributed file system.
- data service 460 accesses the data using second DFS version 415 .
- data service 460 presents the data to LSPF node 421 in accordance with the first version of the DFS. Accordingly, although the file system within LSPF node 421 may manage the data in accordance with a first file system version, the actual data may be stored within data repository 410 in accordance with an alternative version of the file system.
- data service 460 may be used to supply data to virtual LSPF nodes.
- the data service may be located wholly or partially within the host to supply data from the data repository to each of the virtual LSPF nodes initiated on the host.
- a driver or some other process may be located within each of the LSPF nodes.
- the driver may identify the requests and notify data service 460 of the request in accordance with a first DFS version.
- data service 460 accesses the appropriate data and provides the data to the LSPF node using the first DFS version.
- the nodes may be initiated with a first framework and DFS version
- the data within a storage repository may be stored using an alternative DFS version.
- data repository 410 or other data repositories accessible by the LSPF nodes and data service 460 might store data using a variety of different DFS versions.
- data service 460 may be responsible for accessing data using second DFS version 415 and any other file system version that includes data for processing by LSPF nodes 421 - 423 .
- FIG. 5 illustrates an overview 500 for providing data to LSPF nodes in a virtual LSPF cluster.
- Overview 500 includes data repository 510 and host 501 , which further comprises VMs 521 - 523 .
- VMs 521 - 523 are configured with Hadoop instances 531 - 533 to process job 571 .
- VMs 521 - 523 further include distributed file system (DFS) 540 to access data located in data repository 510 .
- DFS distributed file system
- host 501 executes VMs 521 - 523 using hypervisor 550 to abstract the physical components of host 501 and provide the components to VMs 521 - 523 .
- Hypervisor 550 may be software, firmware, or hardware on host 501 that is used present the guest operating systems within VMs 521 - 523 with a virtual operating platform.
- Hadoop instances 531 - 533 process data supplied by data service 560 to complete job 571 .
- VMs 521 - 523 use distributed file system 540 to manage the parallel processing of data across the individual virtual machines.
- a Hadoop instance requires data, such as Hadoop instance 531
- an access request is initiated that requests data in accordance with the version of the file system associated with distributed file system 540 .
- data service 560 accesses the data using a second version of the distributed file system, and presents the data to VM 521 using the first version of the file system.
- a cluster may be initiated with a first version of a data processing framework that uses a first version a distributed file system, the cluster may access data that is stored using a data service to access the data in accordance with a second version of the distributed file system.
- FIG. 6 illustrates a computing system 600 for providing data to LSPF nodes in a LSPF cluster.
- Computing system 600 is an example of any computing system, device, or system of devices capable of implementing the data service described in FIGS. 1-5 .
- Computing system 600 comprises communication interface 601 , user interface 602 , and processing system 603 .
- Processing system 603 is linked to communication interface 601 and user interface 602 .
- Processing system 603 includes processing circuitry 605 and memory device 606 that stores operating software 607 .
- Communication interface 601 comprises components that communicate over communication links, such as network cards, ports, RF transceivers, processing circuitry and software, or some other communication devices.
- Communication interface 601 may be configured to communicate over metallic, wireless, or optical links.
- Communication interface 601 may be configured to use TDM, IP, Ethernet, optical networking, wireless protocols, communication signaling, or some other communication format—including combinations thereof.
- communication interface 601 may be configured to communicate with one or more real or virtual machines within an LSPF cluster, and may be further configured to communicate with a storage repository to provide data to the real or virtual machines.
- User interface 602 comprises components that interact with a user.
- User interface 602 may include a keyboard, display screen, mouse, touch pad, or some other user input/output apparatus.
- User interface 602 may be omitted in some examples.
- Processing circuitry 605 comprises microprocessor and other circuitry that retrieves and executes operating software 607 from memory device 606 .
- Memory device 606 comprises a non-transitory storage medium, such as a disk drive, flash drive, data storage circuitry, or some other memory apparatus.
- Operating software 607 comprises computer programs, firmware, or some other form of machine-readable processing instructions. Operating software 607 includes receipt module 608 , access module 609 , and present module 610 . Operating software 607 may further include an operating system, utilities, drivers, network interfaces, applications, or some other type of software. When executed by circuitry 605 , operating software 607 directs processing system 603 to operate computing system 600 as described herein.
- receipt module 608 is configured to receive or identify access requests from one or more LSPF nodes in an LSPF cluster to access data in accordance with a first version of a distributed file system. Responsive to the requests, access module 609 accesses the data in accordance with a second version of a distributed file system, and present module 610 presents the data to the LSPF nodes in accordance with the first version of the distributed file system used by the LSPF nodes.
- data may be stored in a storage repository using a particular version of a distributed file system that is associated with a map reduce framework.
- mapping new versions of the distributed file system may also be created. Accordingly, as new versions of the map reduce framework are initiated, the framework may no longer match to the older versions of the file system, and vice versa.
- computing system 600 may be used to present data from an older file system version to a newer map reduce framework version in the LSPF cluster, or present data from a newer file system version to an older map reduce framework version in the LSPF cluster.
- FIG. 7 illustrates an overview 700 for providing data to LSPF nodes in multiple LSPF clusters.
- Overview 700 includes data repository 710 , LSPF clusters 740 - 741 , and data service 760 .
- Data repository 710 stores data 716 using a third DFS version 715 .
- LSPF cluster 740 includes LSPF nodes 721 - 722 that initiate access requests using a first DFS version 731
- LSPF cluster 741 includes LSPF nodes 721 - 722 that initiate access requests using a second DFS version 732 .
- LSPF clusters 740 - 741 are initiated to process data that is stored in data repository 710 .
- the nodes initiate access requests using the local DFS present on each of the nodes.
- data service 760 accesses the appropriate data in accordance with the actual DFS version for the data storage, and presents the data to the LSPF nodes using the local DFS version for the node.
- LSPF node 721 includes first DFS version 731 for accessing the data necessary for processing. Accordingly, although LSPF node 721 may be configured with a first DFS version, the data that needs to be accessed may be stored in a distinctly different version of the DFS. As illustrated, LSPF node 721 initiates a data access request using the first version of the file system 731 . Responsive to the request, data service 760 accesses the data in accordance with a third version of the file system 715 and returns the data to LSPF node 721 in accordance with the first version of the file system 731 . Thus, although clusters 740 - 741 use different versions of the file system, data service 760 provides access to data items in data repository 710 using a third version of the file system.
- data service 760 might access data using a variety of other file system versions. Accordingly, if data were stored using a fourth file system version, data service 760 may be capable of receiving the data requests from clusters 740 - 741 , accessing the data using the fourth file system version, and presenting the data to the LSPF nodes in accordance with the file system that was used to make the request. For example, LSPF node 721 may initiate a request using a first version of the file system 731 . Responsive to the request, data service 760 would access the data using the fourth version of the file system, and present the data to LSPF node 721 in accordance with the first version of the file system 731 .
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (12)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/288,506 US11080244B2 (en) | 2014-05-28 | 2014-05-28 | Inter-version mapping of distributed file systems |
PCT/US2015/010200 WO2015183344A1 (en) | 2014-05-28 | 2015-01-05 | Inter-version mapping of distributed file systems |
CN201580028177.5A CN106796500B (en) | 2014-05-28 | 2015-01-05 | Inter-version mapping for distributed file systems |
EP15800015.8A EP3149574A4 (en) | 2014-05-28 | 2015-01-05 | Inter-version mapping of distributed file systems |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/288,506 US11080244B2 (en) | 2014-05-28 | 2014-05-28 | Inter-version mapping of distributed file systems |
Publications (2)
Publication Number | Publication Date |
---|---|
US20150347450A1 US20150347450A1 (en) | 2015-12-03 |
US11080244B2 true US11080244B2 (en) | 2021-08-03 |
Family
ID=54699482
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/288,506 Active 2036-01-02 US11080244B2 (en) | 2014-05-28 | 2014-05-28 | Inter-version mapping of distributed file systems |
Country Status (4)
Country | Link |
---|---|
US (1) | US11080244B2 (en) |
EP (1) | EP3149574A4 (en) |
CN (1) | CN106796500B (en) |
WO (1) | WO2015183344A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210173714A1 (en) * | 2019-12-06 | 2021-06-10 | Microstrategy Incorporated | High-throughput parallel data transmission |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11386085B2 (en) | 2014-01-27 | 2022-07-12 | Microstrategy Incorporated | Deriving metrics from queries |
US10635669B1 (en) | 2014-01-27 | 2020-04-28 | Microstrategy Incorporated | Data engine integration and data refinement |
US10255320B1 (en) | 2014-01-27 | 2019-04-09 | Microstrategy Incorporated | Search integration |
US11921715B2 (en) | 2014-01-27 | 2024-03-05 | Microstrategy Incorporated | Search integration |
US11080244B2 (en) | 2014-05-28 | 2021-08-03 | Hewlett Packard Enterprise Development Lp | Inter-version mapping of distributed file systems |
CN111078628B (en) * | 2018-10-18 | 2024-02-23 | 深信服科技股份有限公司 | Multi-disk concurrent data migration method, system, device and readable storage medium |
US11580105B2 (en) | 2018-10-31 | 2023-02-14 | Anaplan, Inc. | Method and system for implementing subscription barriers in a distributed computation system |
US11354324B1 (en) | 2018-10-31 | 2022-06-07 | Anaplan, Inc. | Method and system for servicing query requests using revisions maps |
US11281683B1 (en) | 2018-10-31 | 2022-03-22 | Anaplan, Inc. | Distributed computation system for servicing queries using revisions maps |
US11481378B1 (en) | 2018-10-31 | 2022-10-25 | Anaplan, Inc. | Method and system for servicing query requests using document-based metadata |
US11475003B1 (en) * | 2018-10-31 | 2022-10-18 | Anaplan, Inc. | Method and system for servicing query requests using dataspaces |
US11573927B1 (en) | 2018-10-31 | 2023-02-07 | Anaplan, Inc. | Method and system for implementing hidden subscriptions in a distributed computation system |
US11567965B2 (en) | 2020-01-23 | 2023-01-31 | Microstrategy Incorporated | Enhanced preparation and integration of data sets |
Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1997038388A2 (en) * | 1996-03-26 | 1997-10-16 | Bull Hn Information Systems Inc. | Method of integrating schemas of distributed heterogeneous databases |
US6195662B1 (en) * | 1997-06-27 | 2001-02-27 | Juxtacomm Technologies Inc. | System for transforming and exchanging data between distributed heterogeneous computer systems |
US20020059307A1 (en) * | 2000-11-01 | 2002-05-16 | Aki Tomita | Method for transforming data formats between different database systems, an apparatus for executing the method and the program of the method |
US20050253739A1 (en) | 2004-05-14 | 2005-11-17 | Oracle International Corporation | Cross platform transportable tablespaces |
US20060248058A1 (en) * | 2005-04-28 | 2006-11-02 | Feng Andrew A | Method, apparatus, and system for unifying heterogeneous data sources for access from online applications |
US20090210426A1 (en) * | 2008-02-14 | 2009-08-20 | Ambercore Software Inc. | System and method for efficient management of distributed spatial data |
US7949692B2 (en) | 2007-08-21 | 2011-05-24 | Emc Corporation | Systems and methods for portals into snapshot data |
CN102195895A (en) * | 2011-03-23 | 2011-09-21 | 中兴通讯股份有限公司 | Message sending and receiving processing method and device |
US20110302583A1 (en) | 2010-06-04 | 2011-12-08 | Yale University | Systems and methods for processing data |
US8266182B2 (en) | 2006-06-30 | 2012-09-11 | Harmonic Inc. | Transcoding for a distributed file system |
US20120278473A1 (en) | 2011-04-27 | 2012-11-01 | Rackspace Us, Inc. | Event Queuing and Distribution System |
US20130124483A1 (en) | 2011-11-10 | 2013-05-16 | Treasure Data, Inc. | System and method for operating a big-data platform |
US20130227558A1 (en) | 2012-02-29 | 2013-08-29 | Vmware, Inc. | Provisioning of distributed computing clusters |
US20130254246A1 (en) | 2012-03-21 | 2013-09-26 | Todd Lipcon | Data processing performance enhancement in a distributed file system |
US20130275363A1 (en) * | 2012-04-17 | 2013-10-17 | Mingxi Wu | Meta-data driven data ingestion using mapreduce framework |
CN103561033A (en) | 2013-11-08 | 2014-02-05 | 西安电子科技大学宁波信息技术研究院 | Device and method for user to have remote access to HDFS cluster |
US20140059163A1 (en) | 2012-08-24 | 2014-02-27 | Facebook Inc. | Distributed request processing |
US20140059310A1 (en) | 2012-08-24 | 2014-02-27 | Vmware, Inc. | Virtualization-Aware Data Locality in Distributed Data Processing |
US20140067864A1 (en) | 2012-09-06 | 2014-03-06 | Ca, Inc. | File access for applications deployed in a cloud environment |
CN103647797A (en) | 2013-11-15 | 2014-03-19 | 北京邮电大学 | Distributed file system and data access method thereof |
WO2015183344A1 (en) | 2014-05-28 | 2015-12-03 | Bluedata Software, Inc. | Inter-version mapping of distributed file systems |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7519964B1 (en) * | 2003-12-03 | 2009-04-14 | Sun Microsystems, Inc. | System and method for application deployment in a domain for a cluster |
CN102130950B (en) * | 2011-03-14 | 2013-10-23 | 中国科学技术大学苏州研究院 | Distributed monitoring method based on Hadoop cluster |
-
2014
- 2014-05-28 US US14/288,506 patent/US11080244B2/en active Active
-
2015
- 2015-01-05 EP EP15800015.8A patent/EP3149574A4/en not_active Ceased
- 2015-01-05 WO PCT/US2015/010200 patent/WO2015183344A1/en active Application Filing
- 2015-01-05 CN CN201580028177.5A patent/CN106796500B/en active Active
Patent Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1997038388A2 (en) * | 1996-03-26 | 1997-10-16 | Bull Hn Information Systems Inc. | Method of integrating schemas of distributed heterogeneous databases |
US6195662B1 (en) * | 1997-06-27 | 2001-02-27 | Juxtacomm Technologies Inc. | System for transforming and exchanging data between distributed heterogeneous computer systems |
US20020059307A1 (en) * | 2000-11-01 | 2002-05-16 | Aki Tomita | Method for transforming data formats between different database systems, an apparatus for executing the method and the program of the method |
US20050253739A1 (en) | 2004-05-14 | 2005-11-17 | Oracle International Corporation | Cross platform transportable tablespaces |
US20060248058A1 (en) * | 2005-04-28 | 2006-11-02 | Feng Andrew A | Method, apparatus, and system for unifying heterogeneous data sources for access from online applications |
US8266182B2 (en) | 2006-06-30 | 2012-09-11 | Harmonic Inc. | Transcoding for a distributed file system |
US7949692B2 (en) | 2007-08-21 | 2011-05-24 | Emc Corporation | Systems and methods for portals into snapshot data |
US20090210426A1 (en) * | 2008-02-14 | 2009-08-20 | Ambercore Software Inc. | System and method for efficient management of distributed spatial data |
US20110302583A1 (en) | 2010-06-04 | 2011-12-08 | Yale University | Systems and methods for processing data |
CN102195895A (en) * | 2011-03-23 | 2011-09-21 | 中兴通讯股份有限公司 | Message sending and receiving processing method and device |
US20120278473A1 (en) | 2011-04-27 | 2012-11-01 | Rackspace Us, Inc. | Event Queuing and Distribution System |
US20130124483A1 (en) | 2011-11-10 | 2013-05-16 | Treasure Data, Inc. | System and method for operating a big-data platform |
US20130227558A1 (en) | 2012-02-29 | 2013-08-29 | Vmware, Inc. | Provisioning of distributed computing clusters |
US20130254246A1 (en) | 2012-03-21 | 2013-09-26 | Todd Lipcon | Data processing performance enhancement in a distributed file system |
US20130275363A1 (en) * | 2012-04-17 | 2013-10-17 | Mingxi Wu | Meta-data driven data ingestion using mapreduce framework |
US20140059163A1 (en) | 2012-08-24 | 2014-02-27 | Facebook Inc. | Distributed request processing |
US20140059310A1 (en) | 2012-08-24 | 2014-02-27 | Vmware, Inc. | Virtualization-Aware Data Locality in Distributed Data Processing |
US20140067864A1 (en) | 2012-09-06 | 2014-03-06 | Ca, Inc. | File access for applications deployed in a cloud environment |
CN103561033A (en) | 2013-11-08 | 2014-02-05 | 西安电子科技大学宁波信息技术研究院 | Device and method for user to have remote access to HDFS cluster |
CN103647797A (en) | 2013-11-15 | 2014-03-19 | 北京邮电大学 | Distributed file system and data access method thereof |
WO2015183344A1 (en) | 2014-05-28 | 2015-12-03 | Bluedata Software, Inc. | Inter-version mapping of distributed file systems |
EP3149574A1 (en) | 2014-05-28 | 2017-04-05 | Bluedata Software Inc. | Inter-version mapping of distributed file systems |
Non-Patent Citations (7)
Title |
---|
Extended European Search Report Received for EP Patent Application No. 15800015.8, dated Mar. 27, 2018, 8 pages. |
Google, "Easier, faster and lower cost Big Data processing with the Google Cloud Storage Connector for Hadoop," available online at <https://cloudplafform.googleblog.com/2014/01/easier-faster-lower-cost-big-data-processing-with-the-google-cloud-storage-connector-for-hadoop.html>, Jan. 14, 2014, 3 pages. |
Google, "Performance advantages of the new Google Cloud Storage Connector for Hadoop", available online at <https://cloudplafform.googleblog.com/2014/01/performance-advantages-of-the-new-google-cloud-storage-connector-or-hadoop.html>, Jan. 15, 2014, 8 pages. |
Kolakowski, Nick, "Google Releases Cloud Storage Connector for Hadoop", Dice, available online at <https://insights.dice.com/2014/01/16/google-releases-cloud-storage-connector-for-hadoop/>, Jan. 16, 2014, 4 pages. |
Pinkerton et al., "A Heterogeneous Distributed File System", IEEE, 1990, pp. 424-431. |
Search Report and Written Opinion received for PCT Application No. PCT/US2015/010200, dated Apr. 29, 2015, 13 pages. |
Shvachko et al., "The Hadoop Distributed File System", 2010, 978-1-4244-7153-9/10 IEEE, 10 pages printed (Year: 2010). * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210173714A1 (en) * | 2019-12-06 | 2021-06-10 | Microstrategy Incorporated | High-throughput parallel data transmission |
US11614970B2 (en) * | 2019-12-06 | 2023-03-28 | Microstrategy Incorporated | High-throughput parallel data transmission |
Also Published As
Publication number | Publication date |
---|---|
WO2015183344A1 (en) | 2015-12-03 |
EP3149574A4 (en) | 2018-04-25 |
US20150347450A1 (en) | 2015-12-03 |
CN106796500A (en) | 2017-05-31 |
CN106796500B (en) | 2020-12-25 |
EP3149574A1 (en) | 2017-04-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11080244B2 (en) | Inter-version mapping of distributed file systems | |
US10915449B2 (en) | Prioritizing data requests based on quality of service | |
US10362101B2 (en) | Mechanism for providing load balancing to an external node utilizing a clustered environment for storage management | |
US11243707B2 (en) | Method and system for implementing virtual machine images | |
US10585806B2 (en) | Associating cache memory with a work process | |
US10564999B2 (en) | Employing application containers in a large scale processing environment | |
US9952782B1 (en) | Method and system for accessing data between different virtual disk formats in a virtualization environment | |
US10838754B2 (en) | Virtualized systems having hardware interface services for controlling hardware | |
US10474373B2 (en) | Provisioning applications in a heterogeneous virtual environment | |
US9378039B2 (en) | Virtual machine storage replication schemes | |
US20170147497A1 (en) | Data caching in a large-scale processing environment | |
US10922300B2 (en) | Updating schema of a database | |
US9971785B1 (en) | System and methods for performing distributed data replication in a networked virtualization environment | |
US9588792B2 (en) | Method and system for sorting and bucketizing alerts in a virtualization environment | |
US20170185444A1 (en) | System for batch processing of workflows for virtualized infrastructure | |
US11086874B2 (en) | Management of a virtual infrastructure via an object query language | |
US10826795B2 (en) | Architecture for implementing service level management for a virtualization environment | |
US9916148B2 (en) | System for managing software package configuration | |
Bonthu et al. | Study on database virtualization for database as a service (DBaaS) | |
Manjunath et al. | NEBULA: Deployment of Cloud Using Open Source | |
Virtualization | Linux* Containers Streamline Virtualization and Complement Hypervisor-Based Virtual Machines |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BLUEDATA SOFTWARE, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PHELAN, THOMAS A.;LAKSHMINARAYANAN, GUNASEELAN;MORETTI, MICHAEL;AND OTHERS;SIGNING DATES FROM 20140516 TO 20140527;REEL/FRAME:032972/0797 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STCV | Information on status: appeal procedure |
Free format text: APPEAL BRIEF (OR SUPPLEMENTAL BRIEF) ENTERED AND FORWARDED TO EXAMINER |
|
AS | Assignment |
Owner name: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP, TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BLUEDATA SOFTWARE, INC.;REEL/FRAME:050037/0071 Effective date: 20190430 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCV | Information on status: appeal procedure |
Free format text: EXAMINER'S ANSWER TO APPEAL BRIEF MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |