WO2018089344A1 - Formation and manipulation of test data in a database system - Google Patents
Formation and manipulation of test data in a database system Download PDFInfo
- Publication number
- WO2018089344A1 WO2018089344A1 PCT/US2017/060352 US2017060352W WO2018089344A1 WO 2018089344 A1 WO2018089344 A1 WO 2018089344A1 US 2017060352 W US2017060352 W US 2017060352W WO 2018089344 A1 WO2018089344 A1 WO 2018089344A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- tenant
- data
- sandbox
- original
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3698—Environments for analysis, debugging or testing of software
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3696—Methods or tools to render software testable
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
Definitions
- FIG. 3 A shows an example system and method of storing and retrieving data using a virtualization layer according to an implementation of the disclosed subject matter.
- applications can be run on servers for production systems, and test systems may be separate from the production systems.
- the test systems may be used to test new features and/or versions of the applications.
- the production system applications may be run on a on a company's or an organization's servers (i.e., running an application "on premises"), or by running an application in a cloud-based computing environment (e.g., where the servers are remote from the company's or organization's servers).
- An extent reference may be used as a logical reference to a physical extent that is stored in physical storage (e.g., a storage device), and may be used to virtualize access to the physical storage.
- an authorized user of the database system that is associated with the source tenant may request that a new extent reference may be added for each extent that contains keys for the source tenant, with instructions to translate the tenant IDs in a key range from the source (i.e., an original tenant) to the target (i.e., a new sandbox tenant).
- the translation can map certain read operations issued by requests generated from users associated with the new sandbox tenant onto original tenant data as of a point in time.
- a method for an authorized user of a database system that is associated with an original tenant to create a sandbox for the original tenant at a point in time, the original tenant having original tenant data stored in an immutable storage associated with an original tenant identifier, the original tenant data as of the sandbox creation point in time being a virtual snapshot of the original tenant data accessible by a sandbox tenant, where the sandbox tenant data may be changed without changing the original tenant data, and the original tenant data may be changed without changing the sandbox tenant data.
- the method may include creating a sandbox tenant identifier.
- the original tenant may access original tenant data created both before and after the sandbox creation point in time that is stored in the immutable storage.
- the subsequent immutable original tenant data is written as new original tenant data that does not alter older original tenant data (e.g., the original and the sandbox tenant data) stored in the immutable storage and the original tenant cannot access the data that is created or augmented by the sandbox tenant.
- the method 20 may be used in connection with the method 10 described above and shown in FIG. 1.
- the database system may receive a query including a key indicating the sandbox tenant.
- the database system may be a server, cloud server, and/or database, or the like.
- the database system that receives a query may be system 100 shown in FIGS. 3A-3B, server 502 and/or server 504 shown in FIG. 4A, central component 700 and/or a second computer 800 shown in FIG. 5, and/or database systems 1200a-1200d shown in FIG. 6.
- the query may be generated, by for example, computer 600 shown in FIG. 5.
- the database system may perform an operation on the translated data at operation 28, and may store data resulting from the performed operation in the immutable storage that is associated with the key of the sandbox tenant at operation 30.
- the database system may, in some implementations, store the data in the immutable storage using a LSM tree data structure.
- Operations on the data associated with the sandbox may include, for example, the manipulation of the data based on one or more functions of the application being tested. That is, the data may be manipulated based on the version of the application being tested and/or a new function that has been added to the application to be tested. However, modifications to that data being tested are not accessible to users associated only with the original tenant.
- the extent 511 associated with the server 502 may have a key range of key ([first key], [last key]) of key (10, 98), and the extent 512 associated with the second server 504 may have a key range of key (33, 66).
- the one or more servers of the systems shown in FIGS. 5-6 may operate on data so as to retrieve data from the immutable storage based on the mapped key.
- the servers may translate the data from the immutable storage indicated by the key of the original tenant to data of the key indicating the sandbox tenant and perform an operation on the translated data.
- the data resulting from the performed operation may be in the immutable storage of a device that is associated with the key of the sandbox tenant, such as the storage device associated with central component 700, second computer 800, and/or one or more of the database systems 1200a-1200d.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Priority Applications (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| AU2017356901A AU2017356901B2 (en) | 2016-11-08 | 2017-11-07 | Formation and manipulation of test data in a database system |
| JP2019523038A JP7090606B2 (ja) | 2016-11-08 | 2017-11-07 | データベース・システムにおけるテスト・データの形成及び動作 |
| CA3043001A CA3043001C (en) | 2016-11-08 | 2017-11-07 | TEST DATA FORMATION AND MANIPULATION IN A DATABASE SYSTEM |
| EP17804736.1A EP3539021B1 (en) | 2016-11-08 | 2017-11-07 | Formation and manipulation of test data in a database system |
| CN201780069069.1A CN109952564B (zh) | 2016-11-08 | 2017-11-07 | 数据库系统中测试数据的形成与操纵 |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/345,914 | 2016-11-08 | ||
| US15/345,914 US10241896B2 (en) | 2016-11-08 | 2016-11-08 | Formation and manipulation of test data in a database system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018089344A1 true WO2018089344A1 (en) | 2018-05-17 |
Family
ID=60473640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2017/060352 Ceased WO2018089344A1 (en) | 2016-11-08 | 2017-11-07 | Formation and manipulation of test data in a database system |
Country Status (7)
| Country | Link |
|---|---|
| US (2) | US10241896B2 (enExample) |
| EP (1) | EP3539021B1 (enExample) |
| JP (1) | JP7090606B2 (enExample) |
| CN (1) | CN109952564B (enExample) |
| AU (1) | AU2017356901B2 (enExample) |
| CA (1) | CA3043001C (enExample) |
| WO (1) | WO2018089344A1 (enExample) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2022500724A (ja) * | 2018-09-24 | 2022-01-04 | セールスフォース ドット コム インコーポレイティッド | テナント識別子の置換 |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10855794B2 (en) | 2018-04-12 | 2020-12-01 | Pearson Management Services Limited | Systems and method for automated package-data asset generation |
| US10963454B2 (en) | 2018-09-24 | 2021-03-30 | Salesforce.Com, Inc. | System and method for bulk removal of records in a database |
| US11151110B2 (en) * | 2018-09-24 | 2021-10-19 | Salesforce.Com, Inc. | Identification of records for post-cloning tenant identifier translation |
| US11238174B2 (en) | 2019-01-31 | 2022-02-01 | Salesforce.Com, Inc. | Systems and methods of database encryption in a multitenant database management system |
| US11409904B2 (en) * | 2019-04-18 | 2022-08-09 | Microsoft Technology Licensing, Llc | User interface for building a data privacy pipeline and contractual agreement to share data |
| US11321294B2 (en) | 2019-09-09 | 2022-05-03 | Salesforce.Com, Inc. | Database index repair |
| US11442960B2 (en) * | 2019-12-17 | 2022-09-13 | Verizon Patent And Licensing Inc. | Edge key value store for a distributed platform |
| US11526465B2 (en) * | 2019-12-18 | 2022-12-13 | Salesforce.Com, Inc. | Generating hash trees for database schemas |
| US11483150B2 (en) | 2020-06-01 | 2022-10-25 | Salesforce.Com, Inc. | Private key cache in secure enclave |
| US11361106B2 (en) | 2020-09-01 | 2022-06-14 | Microsoft Technology Licensing, Llc | Chaining, triggering, and enforcing entitlements |
| CN112434284B (zh) * | 2020-10-29 | 2022-05-17 | 格物钛(上海)智能科技有限公司 | 一种基于沙箱环境的机器学习训练平台实现 |
| KR102411806B1 (ko) * | 2020-11-17 | 2022-06-23 | 쿠팡 주식회사 | 데이터베이스 쿼리 효율 향상을 위한 시스템 및 방법 |
| US10963438B1 (en) | 2020-11-17 | 2021-03-30 | Coupang Corp. | Systems and methods for database query efficiency improvement |
| US11741050B2 (en) | 2021-01-29 | 2023-08-29 | Salesforce, Inc. | Cloud storage class-based variable cache availability |
| US11625386B2 (en) | 2021-01-29 | 2023-04-11 | Salesforce.Com, Inc. | Fast skip list purge |
| US11216322B1 (en) * | 2021-06-07 | 2022-01-04 | Snowflake Inc. | Stored procedures in a network based database system |
| CN114238451A (zh) * | 2021-11-08 | 2022-03-25 | 普强时代(珠海横琴)信息技术有限公司 | 数据横向扩展分析方法及装置 |
| US12282470B2 (en) | 2022-01-31 | 2025-04-22 | Salesforce, Inc. | Skip-list checkpoint creation |
| US11922145B2 (en) | 2022-03-01 | 2024-03-05 | Microsoft Technology Licensing, Llc | Initiating data privacy pipelines using reusable templates |
| US12182004B2 (en) | 2022-03-01 | 2024-12-31 | Microsoft Technology Licensing, Llc | Debugging data privacy pipelines using sample data |
| CN114595159B (zh) * | 2022-05-10 | 2022-09-02 | 太平金融科技服务(上海)有限公司 | 测试数据生成方法、装置、设备及存储介质 |
| US12360759B2 (en) | 2023-01-31 | 2025-07-15 | Salesforce, Inc. | Downgrading database software via a downgradability test |
| US12455884B2 (en) | 2023-01-31 | 2025-10-28 | Salesforce, Inc. | Execution tracing for node cluster |
| US12450126B2 (en) | 2023-02-13 | 2025-10-21 | Pure Storage, Inc. | Automated backups in a distributed storage system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140379677A1 (en) * | 2013-06-24 | 2014-12-25 | Sap Ag | Test sandbox in production systems during productive use |
| US20160048844A1 (en) * | 2014-08-15 | 2016-02-18 | Salesforce.Com, Inc. | Techniques, architectures and mechanisms for management of electronic licensure |
Family Cites Families (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05224992A (ja) * | 1992-02-13 | 1993-09-03 | Hitachi Ltd | プログラムテスト装置 |
| JP3497053B2 (ja) * | 1996-11-13 | 2004-02-16 | 株式会社日立製作所 | オンラインデータベース管理システムにおける処理方法及びオンラインデータベース管理システム |
| US6038639A (en) * | 1997-09-09 | 2000-03-14 | Storage Technology Corporation | Data file storage management system for snapshot copy operations |
| US7101172B2 (en) * | 2002-08-30 | 2006-09-05 | Emerson Electric Co. | Apparatus and methods for variable furnace control |
| KR100439675B1 (ko) * | 2002-10-24 | 2004-07-14 | 한국전자통신연구원 | 대용량 공유 저장장치를 위한 효율적인 스냅샷 수행방법 |
| US20050091658A1 (en) * | 2003-10-24 | 2005-04-28 | Microsoft Corporation | Operating system resource protection |
| US9208188B2 (en) * | 2009-02-17 | 2015-12-08 | Microsoft Technology Licensing, Llc | Tenant management of a hosted multi-tenant application |
| US8161077B2 (en) * | 2009-10-21 | 2012-04-17 | Delphix Corp. | Datacenter workflow automation scenarios using virtual databases |
| US8150808B2 (en) * | 2009-10-21 | 2012-04-03 | Delphix Corp. | Virtual database system |
| US8479286B2 (en) * | 2009-12-15 | 2013-07-02 | Mcafee, Inc. | Systems and methods for behavioral sandboxing |
| US8793291B2 (en) * | 2010-03-15 | 2014-07-29 | Salesforce.Com, Inc. | System, method and computer program product for deploying an update between environments of a multi-tenant on-demand database system |
| US8706772B2 (en) * | 2010-12-30 | 2014-04-22 | Sap Ag | Strict tenant isolation in multi-tenant enabled systems |
| WO2012142854A1 (zh) * | 2011-04-18 | 2012-10-26 | 北京新媒传信科技有限公司 | 一种应用服务平台系统及其实现方法 |
| US9367431B2 (en) * | 2011-07-01 | 2016-06-14 | Salesforce.Com, Inc. | Testing data silo |
| US10860237B2 (en) * | 2014-06-24 | 2020-12-08 | Oracle International Corporation | Storage integrated snapshot cloning for database |
| WO2016117007A1 (ja) * | 2015-01-19 | 2016-07-28 | 株式会社日立製作所 | データベースシステム及びデータベース管理方法 |
| KR102378367B1 (ko) * | 2015-03-20 | 2022-03-24 | 한국전자통신연구원 | 분산 파일 시스템 |
| US9904538B2 (en) * | 2015-08-24 | 2018-02-27 | International Business Machines Corporation | Maintenance of multi-tenant software programs |
| CN106027545B (zh) * | 2016-06-27 | 2019-02-19 | 西安空间无线电技术研究所 | 一种基于虚拟数据的鲁棒信息隐藏方法 |
-
2016
- 2016-11-08 US US15/345,914 patent/US10241896B2/en active Active
-
2017
- 2017-11-07 WO PCT/US2017/060352 patent/WO2018089344A1/en not_active Ceased
- 2017-11-07 EP EP17804736.1A patent/EP3539021B1/en active Active
- 2017-11-07 AU AU2017356901A patent/AU2017356901B2/en active Active
- 2017-11-07 CA CA3043001A patent/CA3043001C/en active Active
- 2017-11-07 JP JP2019523038A patent/JP7090606B2/ja active Active
- 2017-11-07 CN CN201780069069.1A patent/CN109952564B/zh active Active
-
2019
- 2019-01-31 US US16/263,067 patent/US20190163613A1/en not_active Abandoned
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140379677A1 (en) * | 2013-06-24 | 2014-12-25 | Sap Ag | Test sandbox in production systems during productive use |
| US20160048844A1 (en) * | 2014-08-15 | 2016-02-18 | Salesforce.Com, Inc. | Techniques, architectures and mechanisms for management of electronic licensure |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2022500724A (ja) * | 2018-09-24 | 2022-01-04 | セールスフォース ドット コム インコーポレイティッド | テナント識別子の置換 |
| JP7408626B2 (ja) | 2018-09-24 | 2024-01-05 | セールスフォース インコーポレイテッド | テナント識別子の置換 |
Also Published As
| Publication number | Publication date |
|---|---|
| JP7090606B2 (ja) | 2022-06-24 |
| CN109952564B (zh) | 2022-06-21 |
| CA3043001C (en) | 2024-10-01 |
| CN109952564A (zh) | 2019-06-28 |
| US20180129585A1 (en) | 2018-05-10 |
| US10241896B2 (en) | 2019-03-26 |
| JP2020502626A (ja) | 2020-01-23 |
| EP3539021A1 (en) | 2019-09-18 |
| US20190163613A1 (en) | 2019-05-30 |
| CA3043001A1 (en) | 2018-05-17 |
| AU2017356901B2 (en) | 2020-12-24 |
| EP3539021B1 (en) | 2022-06-01 |
| AU2017356901A1 (en) | 2019-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3043001C (en) | TEST DATA FORMATION AND MANIPULATION IN A DATABASE SYSTEM | |
| US10621071B2 (en) | Formation and manipulation of test data in a database system | |
| US11797498B2 (en) | Systems and methods of database tenant migration | |
| US20200201745A1 (en) | Formation and manipulation of test data in a database system | |
| US10592353B2 (en) | Systems and methods of restoring a dataset of a database for a point in time | |
| US11714785B2 (en) | Deduplicating extents across systems | |
| CN112889054A (zh) | 多租户数据库管理系统中数据库加密的系统和方法 | |
| US10083196B2 (en) | Creating secure virtual databases storing masked data | |
| US8386445B2 (en) | Reorganizing database tables | |
| CA3066254A1 (en) | Systems and methods of creation and deletion of tenants within a database | |
| US11650967B2 (en) | Managing a deduplicated data index | |
| US10732840B2 (en) | Efficient space accounting mechanisms for tracking unshared pages between a snapshot volume and its parent volume |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17804736 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 2019523038 Country of ref document: JP Kind code of ref document: A |
|
| ENP | Entry into the national phase |
Ref document number: 3043001 Country of ref document: CA |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: 2017356901 Country of ref document: AU Date of ref document: 20171107 Kind code of ref document: A |
|
| ENP | Entry into the national phase |
Ref document number: 2017804736 Country of ref document: EP Effective date: 20190611 |