WO2018089344A1 - Formation and manipulation of test data in a database system - Google Patents

Formation and manipulation of test data in a database system Download PDF

Info

Publication number
WO2018089344A1
WO2018089344A1 PCT/US2017/060352 US2017060352W WO2018089344A1 WO 2018089344 A1 WO2018089344 A1 WO 2018089344A1 US 2017060352 W US2017060352 W US 2017060352W WO 2018089344 A1 WO2018089344 A1 WO 2018089344A1
Authority
WO
WIPO (PCT)
Prior art keywords
tenant
data
sandbox
original
key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2017/060352
Other languages
English (en)
French (fr)
Inventor
Jameison Bear Martin
Subho Sanjay CHATTERJEE
Patrick James Helland
Nathaniel WYATT
Thomas Fanghaenel
Terry Chong
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Salesforce Inc
Original Assignee
Salesforce com Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Salesforce com Inc filed Critical Salesforce com Inc
Priority to AU2017356901A priority Critical patent/AU2017356901B2/en
Priority to JP2019523038A priority patent/JP7090606B2/ja
Priority to CA3043001A priority patent/CA3043001C/en
Priority to EP17804736.1A priority patent/EP3539021B1/en
Priority to CN201780069069.1A priority patent/CN109952564B/zh
Publication of WO2018089344A1 publication Critical patent/WO2018089344A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Prevention of errors by analysis, debugging or testing of software
    • G06F11/3698Environments for analysis, debugging or testing of software
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668Testing of software
    • G06F11/3696Methods or tools to render software testable
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing

Definitions

  • FIG. 3 A shows an example system and method of storing and retrieving data using a virtualization layer according to an implementation of the disclosed subject matter.
  • applications can be run on servers for production systems, and test systems may be separate from the production systems.
  • the test systems may be used to test new features and/or versions of the applications.
  • the production system applications may be run on a on a company's or an organization's servers (i.e., running an application "on premises"), or by running an application in a cloud-based computing environment (e.g., where the servers are remote from the company's or organization's servers).
  • An extent reference may be used as a logical reference to a physical extent that is stored in physical storage (e.g., a storage device), and may be used to virtualize access to the physical storage.
  • an authorized user of the database system that is associated with the source tenant may request that a new extent reference may be added for each extent that contains keys for the source tenant, with instructions to translate the tenant IDs in a key range from the source (i.e., an original tenant) to the target (i.e., a new sandbox tenant).
  • the translation can map certain read operations issued by requests generated from users associated with the new sandbox tenant onto original tenant data as of a point in time.
  • a method for an authorized user of a database system that is associated with an original tenant to create a sandbox for the original tenant at a point in time, the original tenant having original tenant data stored in an immutable storage associated with an original tenant identifier, the original tenant data as of the sandbox creation point in time being a virtual snapshot of the original tenant data accessible by a sandbox tenant, where the sandbox tenant data may be changed without changing the original tenant data, and the original tenant data may be changed without changing the sandbox tenant data.
  • the method may include creating a sandbox tenant identifier.
  • the original tenant may access original tenant data created both before and after the sandbox creation point in time that is stored in the immutable storage.
  • the subsequent immutable original tenant data is written as new original tenant data that does not alter older original tenant data (e.g., the original and the sandbox tenant data) stored in the immutable storage and the original tenant cannot access the data that is created or augmented by the sandbox tenant.
  • the method 20 may be used in connection with the method 10 described above and shown in FIG. 1.
  • the database system may receive a query including a key indicating the sandbox tenant.
  • the database system may be a server, cloud server, and/or database, or the like.
  • the database system that receives a query may be system 100 shown in FIGS. 3A-3B, server 502 and/or server 504 shown in FIG. 4A, central component 700 and/or a second computer 800 shown in FIG. 5, and/or database systems 1200a-1200d shown in FIG. 6.
  • the query may be generated, by for example, computer 600 shown in FIG. 5.
  • the database system may perform an operation on the translated data at operation 28, and may store data resulting from the performed operation in the immutable storage that is associated with the key of the sandbox tenant at operation 30.
  • the database system may, in some implementations, store the data in the immutable storage using a LSM tree data structure.
  • Operations on the data associated with the sandbox may include, for example, the manipulation of the data based on one or more functions of the application being tested. That is, the data may be manipulated based on the version of the application being tested and/or a new function that has been added to the application to be tested. However, modifications to that data being tested are not accessible to users associated only with the original tenant.
  • the extent 511 associated with the server 502 may have a key range of key ([first key], [last key]) of key (10, 98), and the extent 512 associated with the second server 504 may have a key range of key (33, 66).
  • the one or more servers of the systems shown in FIGS. 5-6 may operate on data so as to retrieve data from the immutable storage based on the mapped key.
  • the servers may translate the data from the immutable storage indicated by the key of the original tenant to data of the key indicating the sandbox tenant and perform an operation on the translated data.
  • the data resulting from the performed operation may be in the immutable storage of a device that is associated with the key of the sandbox tenant, such as the storage device associated with central component 700, second computer 800, and/or one or more of the database systems 1200a-1200d.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Debugging And Monitoring (AREA)
PCT/US2017/060352 2016-11-08 2017-11-07 Formation and manipulation of test data in a database system Ceased WO2018089344A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
AU2017356901A AU2017356901B2 (en) 2016-11-08 2017-11-07 Formation and manipulation of test data in a database system
JP2019523038A JP7090606B2 (ja) 2016-11-08 2017-11-07 データベース・システムにおけるテスト・データの形成及び動作
CA3043001A CA3043001C (en) 2016-11-08 2017-11-07 TEST DATA FORMATION AND MANIPULATION IN A DATABASE SYSTEM
EP17804736.1A EP3539021B1 (en) 2016-11-08 2017-11-07 Formation and manipulation of test data in a database system
CN201780069069.1A CN109952564B (zh) 2016-11-08 2017-11-07 数据库系统中测试数据的形成与操纵

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/345,914 2016-11-08
US15/345,914 US10241896B2 (en) 2016-11-08 2016-11-08 Formation and manipulation of test data in a database system

Publications (1)

Publication Number Publication Date
WO2018089344A1 true WO2018089344A1 (en) 2018-05-17

Family

ID=60473640

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/060352 Ceased WO2018089344A1 (en) 2016-11-08 2017-11-07 Formation and manipulation of test data in a database system

Country Status (7)

Country Link
US (2) US10241896B2 (enExample)
EP (1) EP3539021B1 (enExample)
JP (1) JP7090606B2 (enExample)
CN (1) CN109952564B (enExample)
AU (1) AU2017356901B2 (enExample)
CA (1) CA3043001C (enExample)
WO (1) WO2018089344A1 (enExample)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022500724A (ja) * 2018-09-24 2022-01-04 セールスフォース ドット コム インコーポレイティッド テナント識別子の置換

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10855794B2 (en) 2018-04-12 2020-12-01 Pearson Management Services Limited Systems and method for automated package-data asset generation
US10963454B2 (en) 2018-09-24 2021-03-30 Salesforce.Com, Inc. System and method for bulk removal of records in a database
US11151110B2 (en) * 2018-09-24 2021-10-19 Salesforce.Com, Inc. Identification of records for post-cloning tenant identifier translation
US11238174B2 (en) 2019-01-31 2022-02-01 Salesforce.Com, Inc. Systems and methods of database encryption in a multitenant database management system
US11409904B2 (en) * 2019-04-18 2022-08-09 Microsoft Technology Licensing, Llc User interface for building a data privacy pipeline and contractual agreement to share data
US11321294B2 (en) 2019-09-09 2022-05-03 Salesforce.Com, Inc. Database index repair
US11442960B2 (en) * 2019-12-17 2022-09-13 Verizon Patent And Licensing Inc. Edge key value store for a distributed platform
US11526465B2 (en) * 2019-12-18 2022-12-13 Salesforce.Com, Inc. Generating hash trees for database schemas
US11483150B2 (en) 2020-06-01 2022-10-25 Salesforce.Com, Inc. Private key cache in secure enclave
US11361106B2 (en) 2020-09-01 2022-06-14 Microsoft Technology Licensing, Llc Chaining, triggering, and enforcing entitlements
CN112434284B (zh) * 2020-10-29 2022-05-17 格物钛(上海)智能科技有限公司 一种基于沙箱环境的机器学习训练平台实现
KR102411806B1 (ko) * 2020-11-17 2022-06-23 쿠팡 주식회사 데이터베이스 쿼리 효율 향상을 위한 시스템 및 방법
US10963438B1 (en) 2020-11-17 2021-03-30 Coupang Corp. Systems and methods for database query efficiency improvement
US11741050B2 (en) 2021-01-29 2023-08-29 Salesforce, Inc. Cloud storage class-based variable cache availability
US11625386B2 (en) 2021-01-29 2023-04-11 Salesforce.Com, Inc. Fast skip list purge
US11216322B1 (en) * 2021-06-07 2022-01-04 Snowflake Inc. Stored procedures in a network based database system
CN114238451A (zh) * 2021-11-08 2022-03-25 普强时代(珠海横琴)信息技术有限公司 数据横向扩展分析方法及装置
US12282470B2 (en) 2022-01-31 2025-04-22 Salesforce, Inc. Skip-list checkpoint creation
US11922145B2 (en) 2022-03-01 2024-03-05 Microsoft Technology Licensing, Llc Initiating data privacy pipelines using reusable templates
US12182004B2 (en) 2022-03-01 2024-12-31 Microsoft Technology Licensing, Llc Debugging data privacy pipelines using sample data
CN114595159B (zh) * 2022-05-10 2022-09-02 太平金融科技服务(上海)有限公司 测试数据生成方法、装置、设备及存储介质
US12360759B2 (en) 2023-01-31 2025-07-15 Salesforce, Inc. Downgrading database software via a downgradability test
US12455884B2 (en) 2023-01-31 2025-10-28 Salesforce, Inc. Execution tracing for node cluster
US12450126B2 (en) 2023-02-13 2025-10-21 Pure Storage, Inc. Automated backups in a distributed storage system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140379677A1 (en) * 2013-06-24 2014-12-25 Sap Ag Test sandbox in production systems during productive use
US20160048844A1 (en) * 2014-08-15 2016-02-18 Salesforce.Com, Inc. Techniques, architectures and mechanisms for management of electronic licensure

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05224992A (ja) * 1992-02-13 1993-09-03 Hitachi Ltd プログラムテスト装置
JP3497053B2 (ja) * 1996-11-13 2004-02-16 株式会社日立製作所 オンラインデータベース管理システムにおける処理方法及びオンラインデータベース管理システム
US6038639A (en) * 1997-09-09 2000-03-14 Storage Technology Corporation Data file storage management system for snapshot copy operations
US7101172B2 (en) * 2002-08-30 2006-09-05 Emerson Electric Co. Apparatus and methods for variable furnace control
KR100439675B1 (ko) * 2002-10-24 2004-07-14 한국전자통신연구원 대용량 공유 저장장치를 위한 효율적인 스냅샷 수행방법
US20050091658A1 (en) * 2003-10-24 2005-04-28 Microsoft Corporation Operating system resource protection
US9208188B2 (en) * 2009-02-17 2015-12-08 Microsoft Technology Licensing, Llc Tenant management of a hosted multi-tenant application
US8161077B2 (en) * 2009-10-21 2012-04-17 Delphix Corp. Datacenter workflow automation scenarios using virtual databases
US8150808B2 (en) * 2009-10-21 2012-04-03 Delphix Corp. Virtual database system
US8479286B2 (en) * 2009-12-15 2013-07-02 Mcafee, Inc. Systems and methods for behavioral sandboxing
US8793291B2 (en) * 2010-03-15 2014-07-29 Salesforce.Com, Inc. System, method and computer program product for deploying an update between environments of a multi-tenant on-demand database system
US8706772B2 (en) * 2010-12-30 2014-04-22 Sap Ag Strict tenant isolation in multi-tenant enabled systems
WO2012142854A1 (zh) * 2011-04-18 2012-10-26 北京新媒传信科技有限公司 一种应用服务平台系统及其实现方法
US9367431B2 (en) * 2011-07-01 2016-06-14 Salesforce.Com, Inc. Testing data silo
US10860237B2 (en) * 2014-06-24 2020-12-08 Oracle International Corporation Storage integrated snapshot cloning for database
WO2016117007A1 (ja) * 2015-01-19 2016-07-28 株式会社日立製作所 データベースシステム及びデータベース管理方法
KR102378367B1 (ko) * 2015-03-20 2022-03-24 한국전자통신연구원 분산 파일 시스템
US9904538B2 (en) * 2015-08-24 2018-02-27 International Business Machines Corporation Maintenance of multi-tenant software programs
CN106027545B (zh) * 2016-06-27 2019-02-19 西安空间无线电技术研究所 一种基于虚拟数据的鲁棒信息隐藏方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140379677A1 (en) * 2013-06-24 2014-12-25 Sap Ag Test sandbox in production systems during productive use
US20160048844A1 (en) * 2014-08-15 2016-02-18 Salesforce.Com, Inc. Techniques, architectures and mechanisms for management of electronic licensure

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022500724A (ja) * 2018-09-24 2022-01-04 セールスフォース ドット コム インコーポレイティッド テナント識別子の置換
JP7408626B2 (ja) 2018-09-24 2024-01-05 セールスフォース インコーポレイテッド テナント識別子の置換

Also Published As

Publication number Publication date
JP7090606B2 (ja) 2022-06-24
CN109952564B (zh) 2022-06-21
CA3043001C (en) 2024-10-01
CN109952564A (zh) 2019-06-28
US20180129585A1 (en) 2018-05-10
US10241896B2 (en) 2019-03-26
JP2020502626A (ja) 2020-01-23
EP3539021A1 (en) 2019-09-18
US20190163613A1 (en) 2019-05-30
CA3043001A1 (en) 2018-05-17
AU2017356901B2 (en) 2020-12-24
EP3539021B1 (en) 2022-06-01
AU2017356901A1 (en) 2019-05-23

Similar Documents

Publication Publication Date Title
CA3043001C (en) TEST DATA FORMATION AND MANIPULATION IN A DATABASE SYSTEM
US10621071B2 (en) Formation and manipulation of test data in a database system
US11797498B2 (en) Systems and methods of database tenant migration
US20200201745A1 (en) Formation and manipulation of test data in a database system
US10592353B2 (en) Systems and methods of restoring a dataset of a database for a point in time
US11714785B2 (en) Deduplicating extents across systems
CN112889054A (zh) 多租户数据库管理系统中数据库加密的系统和方法
US10083196B2 (en) Creating secure virtual databases storing masked data
US8386445B2 (en) Reorganizing database tables
CA3066254A1 (en) Systems and methods of creation and deletion of tenants within a database
US11650967B2 (en) Managing a deduplicated data index
US10732840B2 (en) Efficient space accounting mechanisms for tracking unshared pages between a snapshot volume and its parent volume

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17804736

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019523038

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 3043001

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017356901

Country of ref document: AU

Date of ref document: 20171107

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2017804736

Country of ref document: EP

Effective date: 20190611