AU2010100276B4 - A method and apparatus for minimising disk fragmentation of Lotus Notes Databases - Google Patents

A method and apparatus for minimising disk fragmentation of Lotus Notes Databases Download PDF

Info

Publication number
AU2010100276B4
AU2010100276B4 AU2010100276A AU2010100276A AU2010100276B4 AU 2010100276 B4 AU2010100276 B4 AU 2010100276B4 AU 2010100276 A AU2010100276 A AU 2010100276A AU 2010100276 A AU2010100276 A AU 2010100276A AU 2010100276 B4 AU2010100276 B4 AU 2010100276B4
Authority
AU
Australia
Prior art keywords
database
disk
lotus notes
file
minimising
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
AU2010100276A
Other versions
AU2010100276A4 (en
Inventor
Adam Osborne
Grant Osborne
Mark Vallis
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
PREEMPTIVE CONSULTING Pty Ltd
Original Assignee
PREEMPTIVE CONSULTING Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by PREEMPTIVE CONSULTING Pty Ltd filed Critical PREEMPTIVE CONSULTING Pty Ltd
Priority to AU2010100276A priority Critical patent/AU2010100276B4/en
Publication of AU2010100276A4 publication Critical patent/AU2010100276A4/en
Application granted granted Critical
Publication of AU2010100276B4 publication Critical patent/AU2010100276B4/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0674—Disk device
    • G06F3/0676—Magnetic disk device

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Description

Amended Description BACKGROUND Field of the Invention This invention relates to the field of computer systems and disk file operations. More specifically, this invention relates to a method and an apparatus for managing the disk storage usage of Lotus Notes databases in such a way that minimises ongoing disk file fragmentation and the need to de fragment these database files in order to maintain optimal system performance. Lotus Notes and Domino are products and trademarks of IBM Corporation. Related Art Lotus Notes databases are a facility for storage or structured, semi-structured and unstructured data and associated actions upon that data. A Lotus Notes database is typically stored in a discrete operating system file and given a file suffix of either .NSF or .NTF to denote the file type. As a Lotus Notes database is used (either for reading or writing), or maintained by the database management system (Domino), the database will at various times need to grow in disk file size. The additional disk space used by the database is located on the disk drive at the discretion of the underlying operating system, and is usually allocated in such a way that it is in a non-contiguous location of the disk drive. Over time, the frequent allocations of additional disk will often result in the Lotus Notes database's disk file being greatly fragmented over the entire disk drive, leading to degraded performance. De-fragmenting the allocated disk storage for a database is a relatively expensive operation that often only provides temporary relief from the issues of fragmentation, as continued use of the database will result in new non-contiguous disk allocations being used by the Lotus Notes database. What is needed is a mechanism to pre-emptively preallocate disk space to a Lotus Notes database such that the storage growth needs of the database can be accommodated by the preallocated space, rather than by extending the file size of the database in an ad-hoc manner. SUMMARY This invention describes a method and an apparatus for pre-emptively allocating and maintaining the disk storage used by a Lotus Notes database in such a way that minimises the fragmentation of the database over the underlying file system. BRIEF DESCRIPTION OF THE FIGURES Figures 1 and 2 illustrate that when a Lotus Notes database expands to accommodate extra data, the underlying disk file may become fragmented. Page 2 Figures 3 to 7 illustrate the operation of the invention as a series of steps, where a database is first expanded in size, then de-fragmented, then extra data is added within the pre-allocated disk space. DETAILED DESCRIPTION The following description and specifications have been presented for the purposes of description and illustration and to permit any person skilled in the art to make and use the invention. Further, the following description and specifications are provided in the context of a particular embodiment of the invention. They are not intended to limit the scope of the invention in application or form. Many variations and modifications will be apparent to practitioners skilled in the art. The invention is to be accorded the widest scope consistent with the principles and features disclosed below and defined by the appended claims. The situation prior to the invention is displayed in figures 1 and 2. Figure 1 represents a Lotus Notes database that is full of data and resides on a contiguous range of storage on the underlying disk subsystem. Figure 2 shows what would typically happened when additional data is added to the Lotus Notes database. The underlying operating system is free to utilise any disk space to hold the increased data, and usually ends up using disk storage that is not contiguous with the initially used storage. The has a negative effect on the subsequent performance of operation that use the Lotus Notes database as the disk subsystem has to hunt around the disk to locate each fragment. A simple de-fragmentation of the disk storage at this point would bring the separate disk allocations together into a contiguous block, however the next piece of data to be added to the Lotus Notes database would result in a fresh, non-contiguous allocation of disk storage. Figures 3 to 7 show the operation of the invention. Figure 3 is the same as figure 1 and shows a full Lotus Notes database located on a contiguous range of disk storage. The invention operates by first detecting that the amount of allocated but unused space in the Lotus Notes database is less than the pre-configured threshold value. In the case of figure 3, there is no space that is unused. The invention then adds temporary data to the Lotus Notes database as shown in figure 4. This temporary data will frequently result in non-contiguous disk allocations being assigned to the database. It is important to note that the amount of storage to use in the temporary data is configurable, for example (but not limited to) by percentage of size, ratio or by a fixed amount. in the case of the diagram, it is a 50% size increase over the initial database size. The invention then deletes the temporary data from the Lotus Notes database- Lotus Notes databases do not automatically shrink to fit the reduced amount of data, so the file holding the database is left containing unused space, as shown in figure 5. Page 3 The invention proceeds to de-fragment the file holding the Lotus Notes database as shown in figure 6. Note that there may not be sufficient contiguous space on the disk to hold the entire database file, in which case invention attempts to move the various database file allocations into available spaces on the disk drive such that the total number of contiguous blocks of disk used to hold the file is reduced. At this point the Lotus Notes database has been expanded into contiguous space (or at least more contiguous space) on disk. If new data is added to the Lotus Notes database (figure 7), the database management system (Domino) will use the allocated but unused space to store the new data. The disk file is not fragmented, preventing the need to perform a relatively expensive de-fragmentation to optimise the storage. As figure 7 shows, it may be possible to avoid fragmentation of the Lotus Notes database file even when several addition pieces or data are added to the database. A point worth noting is that if the initial state of the Lotus Notes database (this is, before the operation of the invention upon it) is such that the existing disk file is fragmented, then the invention will achieve free space expansion of the database as well as de-fragmentation of the underlying disk file. Page 4
AU2010100276A 2010-03-26 2010-03-26 A method and apparatus for minimising disk fragmentation of Lotus Notes Databases Ceased AU2010100276B4 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2010100276A AU2010100276B4 (en) 2010-03-26 2010-03-26 A method and apparatus for minimising disk fragmentation of Lotus Notes Databases

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
AU2010100276A AU2010100276B4 (en) 2010-03-26 2010-03-26 A method and apparatus for minimising disk fragmentation of Lotus Notes Databases

Publications (2)

Publication Number Publication Date
AU2010100276A4 AU2010100276A4 (en) 2010-04-29
AU2010100276B4 true AU2010100276B4 (en) 2010-09-23

Family

ID=42139185

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2010100276A Ceased AU2010100276B4 (en) 2010-03-26 2010-03-26 A method and apparatus for minimising disk fragmentation of Lotus Notes Databases

Country Status (1)

Country Link
AU (1) AU2010100276B4 (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6745311B2 (en) * 2001-01-24 2004-06-01 Networks Associates Technology, Inc. Method of allocating clusters of computer readable medium to a file while minimizing fragmentation of the computer readable medium

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6745311B2 (en) * 2001-01-24 2004-06-01 Networks Associates Technology, Inc. Method of allocating clusters of computer readable medium to a file while minimizing fragmentation of the computer readable medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"How to purge document deletion stubs immediately", IBM, 24 July 2009. *
"Purging documents in Lotus Notes", IBM, 5 March 2008. *

Also Published As

Publication number Publication date
AU2010100276A4 (en) 2010-04-29

Similar Documents

Publication Publication Date Title
US9778881B2 (en) Techniques for automatically freeing space in a log-structured storage system based on segment fragmentation
US8838541B2 (en) Data processing apparatus and method of processing data
US9851917B2 (en) Method for de-duplicating data and apparatus therefor
US8782368B2 (en) Storing chunks in containers
US8452739B2 (en) Highly scalable and distributed data de-duplication
EP2921963A1 (en) Memory recycling method and device
US11126591B2 (en) Filter file system with inode number as primary database key
US20110145207A1 (en) Scalable de-duplication for storage systems
JP2006195960A5 (en)
KR20010082032A (en) System and method for persistent and robust storage allocation
EP3196781A1 (en) Method and apparatus for deleting duplicate data
US20160004598A1 (en) Grouping chunks of data into a compression region
CN101727503A (en) Method for establishing disk file system
US8190655B2 (en) Method for reliable and efficient filesystem metadata conversion
US8909897B2 (en) Method for generating a delta for compressed data
US8745350B2 (en) Temporary VTOC locking during defragmentation
US20140340778A1 (en) De-Duplication
AU2010100276B4 (en) A method and apparatus for minimising disk fragmentation of Lotus Notes Databases
US9798793B1 (en) Method for recovering an index on a deduplicated storage system
US7779207B2 (en) Accelerated write performance
US12001397B2 (en) Method, apparatus and device for deleting distributed system file, and storage medium
US10204002B1 (en) Method for maintaining a cache index on a deduplicated storage system
CN106991021B (en) Method and system for constructing new data file from existing data file
US20160103623A1 (en) Method for controlled collision of hash algorithm based on nand flash memory
AU2010100571A4 (en) A method and apparatus for integrating a disk defragmentation process within Lotus Notes.

Legal Events

Date Code Title Description
FF Certified innovation patent
MK22 Patent ceased section 143a(d), or expired - non payment of renewal fee or expiry