WO2017131789A1 - Memory management with versioning of objects - Google Patents

Memory management with versioning of objects Download PDF

Info

Publication number
WO2017131789A1
WO2017131789A1 PCT/US2016/015839 US2016015839W WO2017131789A1 WO 2017131789 A1 WO2017131789 A1 WO 2017131789A1 US 2016015839 W US2016015839 W US 2016015839W WO 2017131789 A1 WO2017131789 A1 WO 2017131789A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
write
version
log
write log
Prior art date
Application number
PCT/US2016/015839
Other languages
French (fr)
Inventor
Izzat EL HAJJ
Alexander MERRITT
Gerd ZELLWEGER
Dejan S. Milojicic
Reto ACHERMANN
Original Assignee
Hewlett Packard Enterprise Development Lp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett Packard Enterprise Development Lp filed Critical Hewlett Packard Enterprise Development Lp
Priority to PCT/US2016/015839 priority Critical patent/WO2017131789A1/en
Publication of WO2017131789A1 publication Critical patent/WO2017131789A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/0223User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/0292User address space allocation, e.g. contiguous or non contiguous base addressing using tables or multilevel address translation means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/0223User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023Free address space management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10Providing a specific technical effect
    • G06F2212/1012Design facilitation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10Providing a specific technical effect
    • G06F2212/1041Resource optimization
    • G06F2212/1044Space efficiency improvement

Definitions

  • Figure 1 is a block diagram illustrating a computing environment suitable for implementing aspects of a system for versioning in-memory objects according to one example.
  • FIG. 2 is a diagram illustrating an object descriptor table (ODT) according to one example.
  • ODT object descriptor table
  • Figure 3 is a diagram illustrating a page table structure according to one example.
  • Figure 4 is a diagram illustrating a write log according to one example.
  • Figure 5 is a diagram illustrating a log tracking table (LTT) according to one example.
  • Figure 6 is control flow diagram illustrating object version creation as software modifies the object according to one example.
  • Figure 7 is a flow diagram illustrating a method for performing a register object operation according to one example.
  • Figure 8 is a flow diagram illustrating a method for performing a create version operation according to one example.
  • Figure 9 is a flow diagram illustrating a method for performing a log a write operation according to one example.
  • Figure 10 is a flow diagram illustrating a method for performing a clean write log operation according to one example.
  • Figure 1 1 is a flow diagram illustrating a method for performing a copy object version operation according to one example.
  • Figure 12 is a flow diagram illustrating a method for performing a restore object version operation according to one example.
  • Examples disclosed herein address the lack of fast versioning capabilities for memory objects by leveraging efficient hardware mechanisms tied to the memory system.
  • Some methods for tracking changes to memory may involve an understanding of the architectural descriptions of memory, which may be done via address translation mechanisms in the CPU, such as page tables.
  • CPUs may provide support for system software to track changes via access and dirty flags in the translation structures directly. These structures are complex and time-consuming to scan. With large memory spaces, scanning incurs high overheads to locate these flags within the structures.
  • Alternative CPU architectures may provide binary incompatible representations of these structures, complicating system software.
  • a page-to-object mapping table would have to be consulted to associate the set of dirty pages with the pages backing a specific memory object or objects.
  • Some examples disclosed herein are directed to a memory management system that uses memory write logging for fast versioning of in-memory objects.
  • the system uses abstractions that support the notion of memory objects and object versions, and includes mechanisms to quickly identify change sets of memory objects. Examples disclosed herein have the ability to efficiently track and identify change sets (e.g., definitions) of memory objects using hardware features, and provide software simplification in identifying memory changes through hardware features that abstract
  • Some examples provide the ability to track memory object modifications efficiently, leveraging hardware-provided dirty memory tracking via write modification logging and object descriptor tables. Examples disclosed herein quickly identify change sets of memory objects, and apply change sets of objects in support of object versioning.
  • a "memory object" is a (virtually) contiguous buffer of memory, and is a system abstraction for associating memory resources with application-level interpretations of objects. These buffers may be defined in lengths of multiples of physical page sizes, as defined by the CPU architecture's memory management support.
  • a "memory object version” is a representation of the object state from a given moment in time.
  • a version may carry a unique identifier or key, attributes such as immutability, and a reference to the predecessor from which it is derived, etc. Versions may exist in a logical "timeline" reflecting the changes made to an object over time.
  • a "versionable memory object” is a memory object that is capable of being versioned over its lifetime. Methods for creating versions, instantiating prior versions, etc., may be made available for use.
  • a "version Identifier” is an identifier (typically a unique number) by which the system can uniquely refer to a memory object version within the system.
  • Figure 1 is a diagram illustrating a computing environment 10 suitable for implementing aspects of a system for versioning in-memory objects according to one example.
  • the computing system or computing device 10 includes processing units 12 and system memory 14.
  • memory 14 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two.
  • Computing device 10 may also have additional features/functionality.
  • computing device 10 may also include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape.
  • additional storage is illustrated in Figure 1 by removable storage 16 and non-removable storage 18.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any suitable method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Memory 14, removable storage 16 and non-removable storage 18 are all examples of non- transitory computer storage media (e.g., non-transitory computer-readable storage media storing computer-executable instructions that when executed by at least one processor cause the at least one processor to perform a method).
  • Non-transitory computer storage media includes RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, and magnetic disk storage or other magnetic storage devices. Any such non-transitory computer storage media may be part of computing device 10.
  • computing device 10 The various elements of computing device 10 are communicatively coupled together via communication links 15.
  • Computing device 10 also includes communication connections 24 that allow computing device 10 to communicate with other computers/applications 26.
  • Computing device 10 may also include input device(s) 22, such as keyboard, pointing device (e.g., mouse), pen, voice input device, touch input device, scanner, etc.
  • Computing device 10 may also include output device(s) 20, such as a display, speakers, printer, etc.
  • module 17 is stored in system memory 14.
  • Module 17 facilitates object versioning techniques described herein.
  • module 17 may include program instructions executable by a processor.
  • Features of the versioning of in-memory objects according to examples disclosed herein may also be implemented in hardware.
  • Various combinations of software and hardware, including firmware, may be utilized to implement specific examples.
  • Figure 1 and the above discussion are intended to provide a brief general description of a suitable computing environment in which examples may be implemented. It should be understood, however, that handheld, portable, and other computing devices of all kinds are contemplated for use.
  • Figure 1 thus illustrates an example of a suitable computing system environment 10 in which the examples may be implemented, although as made clear above, the computing system environment 10 is one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the examples. Neither should the computing environment 10 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the example operating environment 10.
  • Some examples disclosed herein may use at least one of an object descriptor table, a page table, a write log, and a log tracking table, which are described in further detail below with reference to Figures 2-5, respectively.
  • FIG. 2 is a diagram illustrating an object descriptor table (ODT) 200 according to one example.
  • the ODT 200 according to one example is implemented in hardware and describes to the hardware a set of software- identified memory objects to track.
  • the ODT 200 may be referenced by a separate register in hardware that stores an ODT root pointer 201 containing the physical address to one instance of an ODT.
  • ODT 200 includes a plurality of entries 202, with each entry 202 corresponding to a particular object.
  • Each entry 202 may contain a user-defined object identifier (ID) 204 that uniquely identifies the object, the base virtual address 206 to the object, the object's length 208, and a log field 210 to indicate a write to the object has been recorded.
  • ID user-defined object identifier
  • FIG. 3 is a diagram illustrating a page table structure 300 according to one example.
  • the structure 300 includes a plurality of page tables 301 , with each page table 301 including a plurality of page table entries 302.
  • Each page table entry 302 corresponds to a particular page of main memory 306 and includes a page log bit 304, which indicates whether version tracking is enabled for the corresponding page. If at least one page log bit 304 is enabled in any entry 302 within a given page table 301 , this indicates to hardware the presence of an ODT 200 immediately following the page descriptor table of that page table 301 .
  • the page descriptor table is the last level of a page table 301 , and the ODT 200 is physically located after the page descriptor table. In this example, the ODT 200 is not referenced by a separate register in hardware.
  • Each entry 202 in the ODT 200 includes an object ID field 204 that indicates to which objects the associated pages belong.
  • FIG. 4 is a diagram illustrating a write log 400 according to one example.
  • the write log 400 is a location in memory provided to the CPU to record (append) write operations as indicated by the ODT 200 ( Figure 2) or other structures.
  • the write log 400 includes a plurality of entries 402 created by the CPU.
  • the entries 402 are organized into slabs 403. Each slab 403 contains a specific quantity of log entries 402.
  • Each entry 402 contains information recording a write operation, such as a user-defined object identifier (ID) 404 that uniquely identifies an object that was subject to the write operation, a (machine) address 406 associated with the object, and a valid bit 408. If the valid bit 408 is zero, this indicates it has been consumed by software.
  • ID user-defined object identifier
  • FIG. 5 is a diagram illustrating a log tracking table (LTT) 500 according to one example.
  • the LTT 500 includes a range 502, a granularity 504, and a contiguous in-memory bitmask field 506.
  • Each bit in field 506 represents one chunk within contiguous memory at a given granularity specified by the granularity 504.
  • the granularity 504 specifies the granularity at which memory writes are added to the write log 400 ( Figure 4).
  • Each bit in the field 506 informs the CPU whether to record a write to the given address into the write log 400.
  • a value of zero in field 506 permits the hardware to record a write issued by software to the write log 400.
  • the LTT 500 may be used in combination with the ODT 200 extensions to the page tables 301 .
  • Figure 6 is control flow diagram illustrating object version creation as software modifies the object according to one example.
  • the object includes a first version 602(1 ), which includes pages 604(1 ), 604(2), and 604(3).
  • a write operation is then performed on page 604(2) of the first version 602(1 ), and a second version 602(2) is created.
  • the second version 602(2) includes a page reference 606(1 ) that refers to the unmodified page 604(1 ) of the first version 602(1 ); a page 606(2) that includes the data from the write operation; and a page reference 606(3) that refers to the unmodified page 604(3) of the first version 602(1 ).
  • a third version 602(3) is then created, which includes a page reference 608(1 ) that refers to the unmodified page 604(1 ) of the first version 602(1 ); a page reference 608(2) that refers to the page 606(2) of the second version 602(2); and a page reference 608(3) that refers to the unmodified page 604(3) of the first version 602(1 ).
  • Write operations are then performed on pages 604(1 ), 606(2), and 604(3), and a fourth version 602(4) is created that includes updated pages 610(1 ), 610(2), and 610(3).
  • Register object (see Figure 7). Software may identify a virtually contiguous memory buffer for versioning, and if hardware state exists, software will additionally initialize this state for this object.
  • Software-backed or hardware-backed metadata instruct this operation's behavior in when and where to create log entries.
  • FIG. 7 is a flow diagram illustrating a method 700 for performing a register object operation according to one example.
  • a request to register an object is received.
  • the LTT 500 ( Figure 5) is initialized. If it is determined at 710 that the LTT 500 is initialized, the method 700 moves to 712. If it is determined at 710 that the LTT 500 is not initialized, the LTT 500 is initialized at 708, and the method 700 moves to 712.
  • software identifies the memory object.
  • software creates meta-data describing the object.
  • a new entry 202 in the ODT 200 is added for the object.
  • FIG 8 is a flow diagram illustrating a method 800 for performing a create version operation according to one example.
  • software creates new object version metadata.
  • references are added to all regions associated with the object to the version.
  • a version identifier is set to an initial value.
  • all associated log bits in either ODT 200 or LTT 500 are set to zero.
  • the method 800 returns to client.
  • the method 800 moves to 814.
  • the referenced memory region is copied to a new location.
  • a new reference is added to the new location to object version.
  • the log bit in either ODT 200 or LTT 500 is set to zero.
  • the entry in the write log 400 is
  • the method 800 returns to 814. [0044] If it is determined at 814 that the write log 400 does not have valid entries with object ID 404, the method 800 moves to 824. At 824, for all regions not yet references in the version, references are copied from the prior version. At 826, the object version is incremented. At 828, the method 800 returns to client.
  • FIG. 9 is a flow diagram illustrating a method 900 for performing a log a write operation according to one example.
  • software issues a store (or write) instruction.
  • hardware appends a record of the write to the write log 400.
  • hardware increments the write log index.
  • hardware sets the log bit to a value of one.
  • dirty bit support in the page table 301 ( Figure 3) enables software to scan the page table 301 for these bits for those pages backing the object. Once identified, each dirty page is copied to a new set of pages and linked with the next version identifier value. The version identifier is incremented for that object.
  • write logging is not provided by the hardware and not explicitly performed by software.
  • Another example implementation may leverage hardware providing general page-granularity write logging for an address space.
  • Another example implementation may use ODT 200. Registering an object creates a new entry 202 into the ODT 200. Should software request to track objects at granularities equal to their length, a log field 210 may be supported in each ODT entry 202 to indicate a write to the object has been recorded - a log bit - used as follows.
  • the hardware When the hardware encounters a write to an address, it consults the ODT 200, and if an object is found to contain the address, and if the entry's log bit 210 is zero, it sets the log bit 210 to one and records the write to the write log 400 ( Figure 4), storing the user-defined value provided in the entry, and the physical base address.
  • software is responsible for clearing the log bits.
  • Another example implementation may leverage hardware extensions to the page table 300 to track large objects using a granularity independent of the virtual memory translations system. Such is provided by the LTT 500.
  • a page log bit 304 indicating whether tracking is enabled for this page. If at least one such bit is enabled in a given entry 302 within a page table 301 , hardware then expects to locate an ODT 200 for the set of translations associated with the page table.
  • hardware Upon a write by software, hardware will examine the page log bit 304, and if set, examine the associated bit in the LTT 500 using the address from the write and the granularity configured for the LTT 500. If the associated bit in the LTT 500 is zero, hardware records this write into the write log 400 with the physical address and object identifier. The LTT bit is then monotonically incremented.
  • Some examples perform management operations on the write log 400, e.g., to optimize it for scanning operations. Examples disclosed herein support the following operation:
  • [0050] 1 Clean log (see Figure 10).
  • the write log 400 may become fragmented due to the request stream by software to version objects, whereby some entries 402 become invalid. Subsequent version requests may incur costs of scanning the entire log 400, based on the location of the first valid log entry.
  • An example implementation of log cleaning manages the write log 400 in the granularity of memory slabs 403.
  • Each slab 403 contains a specific quantity of log entries 402, with fields 404, 406, and 408.
  • software When software performs object versioning, it will scan the write log 400 linearly to locate entries 402 associated with the object to compose the state for the next version, invalidating each entry it consumes.
  • Cleaning the write log 400 is a method for compressing the state of the log 400, discarding invalid entries and releasing memory backing each slab 403 to the system.
  • all valid entries 402 are counted and copied (in a stable manner, preserving their order) to another set of empty slabs 403.
  • the scan direction may begin with the last entry 402 in the slab 403 immediately preceding the current slab 403, until the first valid entry 402 is reached, skipping invalid entries 402.
  • the compressed slabs 403 become the new head of the write log 400 used for scanning when new object versions are created.
  • FIG 10 is a flow diagram illustrating a method 1000 for performing a clean write log operation according to one example.
  • valid entries in write log slabs 403 between the head and the current index of the write log 400 are counted.
  • new slabs sufficient to contain the number of entries identified at 1002 are allocated.
  • the method 1000 moves to 1012 to return an error to the client. If it is determined at 1010 that there is available memory, the method 1000 moves to 1014. At 1014, all valid entries are copied to new slabs 403, and the original order of the entries is preserved. At 1016, the write log head is updated. At 1018, the method frees the old slabs 403.
  • Some examples perform operations on object versions, e.g., to access or delete older versions. Examples disclosed herein support the following four operations:
  • Copy object version (see Figure 1 1 ).
  • a previously created object version may be provided to software for use. This instance does not have write tracking enabled, as it is provided initially as a non-versionable memory object.
  • An example implementation of the copy object version operation scans the object version state to locate the most recent instance of each page of the object (where the granularity of a page is defined by the logging granularity), and copy that the content to a provided buffer, beginning with the version of the object requested.
  • Restore object version (see Figure 12). Similar to operation 1 (copy object version), except the system discards all versions newer than the provided version in the object's history and resumes tracking.
  • implementation of the restore object version operation performs a version copy operation, with the added effect that newer versions are removed from the history, and all buffers backing tracked writes are released. The buffer holding the object is overwritten with the state from the associated version.
  • a version may be removed from the recorded history, which involves merging of adjacent version states. This operation may be used to remove a range of versions, e.g., versions older than a specific time stamp. A configurable behavior of this may, instead of merging a set of prior versions, remove those versions entirely, altering what is seen as the original derived version in the history.
  • An example implementation of the delete object version operation moves the change set held by a version and merges it into the prior version, updating any references from the prior version to point to the subsequent version (or current copy of the object).
  • Branch object version A prior object version is provided as a new derived instance of the object to software, with write tracking enabled. New versions derived from this instance are recorded to a separate "branch" in the version tree.
  • Figure 1 1 is a flow diagram illustrating a method 1 100 for performing a copy object version operation according to one example.
  • the client presents a buffer to software to receive an object version copy.
  • Figure 12 is a flow diagram illustrating a method 1200 for performing a restore object version operation according to one example.
  • the region referenced in the specified version is copied into the region of the object.
  • the latest version ID is updated to the specified version ID.
  • resources for all newer versions of the object are released.
  • all associated write log tracking bits in ODT 200 or LTT 500 are reset.
  • One example of the present disclosure is directed to a memory
  • the system includes a processor, and a write log to allow the processor to record write log entries, wherein each write log entry
  • the system represents a write to main memory that has been performed.
  • the system includes at least one table to facilitate versioning of in-memory objects stored in the main memory, and system software to track changes to the in-memory objects and create new versions of the in-memory objects using the write log and the at least one table.
  • the at least one table may include an object descriptor table (ODT) including a plurality of ODT entries, wherein each ODT entry corresponds to one of the in-memory objects.
  • ODT entry may include an object ID, an object address, and an object length.
  • Each ODT entry may further include a log bit to indicate when a write to the object corresponding to the entry has been performed.
  • the at least one table may include a page table including a plurality of page table entries, wherein each page table entry includes a page log bit that indicates whether version tracking is enabled. If the page log bit indicates that version tracking is enabled, an object descriptor table (ODT) may immediately follow the page table and include a plurality of ODT entries, wherein each ODT entry corresponds to one of the in-memory objects.
  • ODT object descriptor table
  • Each write log entry may include an object identifier that identifies one of the in-memory objects that was subject to a write, and an address associated with the identified one of the in-memory objects. Each write log entry may further include a valid bit indicating whether the write log entry has been consumed by the system software.
  • the at least one table may include a log tracking table that includes a granularity field and a bitmask field storing a plurality of bits, wherein each bit in the bitmask field corresponds to one contiguous chunk within the main memory at a granularity specified in the granularity field. Each bit in the bitmask field may indicate whether the processor is to record a write in the write log.
  • the memory management system may support at least one of: a register object operation to allow the system software to register a new in-memory object; a create version operation to allow the system software to create a new version of one of the in-memory objects; a clean log operation to cause invalid entries in the write log to be discarded; a copy object version operation to copy a version of one of the in-memory objects to a buffer specified by the system software; a restore object version operation to discard all versions newer than a specified version; and a delete object version operation to remove an object version or a range of object versions.
  • a register object operation to allow the system software to register a new in-memory object
  • a create version operation to allow the system software to create a new version of one of the in-memory objects
  • a clean log operation to cause invalid entries in the write log to be discarded
  • a copy object version operation to copy a version of one of the in-memory objects to a buffer specified by the system software
  • the method includes recording, with a processor, write log entries into a write log, wherein each write log entry represents a write to main memory that has been performed.
  • the method includes tracking, with system software, changes to in-memory objects stored in the main memory based on the write log and at least one table.
  • the method includes creating, with the system software, new versions of the in-memory objects based on the tracking.
  • Each write log entry may identify one of the in-memory objects, and includes a valid bit indicating whether the write log entry has been consumed by the system software.
  • Yet another example of the present disclosure is directed to a memory management system, which includes a processor, and a write log to allow the processor to record write log entries, wherein each write log entry represents a write to main memory that has been performed, and identifies an in-memory object that was affected by the write.
  • the system includes at least one table to provide information for versioning the in-memory object, and system software to track changes to the in-memory object and create a new version of the in- memory object using the write log and the at least one table.
  • Each write log entry may include a valid indicator indicating whether the write log entry has been consumed by the system software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

Examples disclosed herein include a memory management system, which includes a processor, and a write log to allow the processor to record write log entries, wherein each write log entry represents a write to main memory that has been performed. The system includes at least one table to facilitate versioning of in-memory objects stored in the main memory, and system software to track changes to the in-memory objects and create new versions of the in-memory objects using the write log and the at least one table.

Description

MEMORY MANAGEMENT WITH VERSIONING OF OBJECTS
Background
[0001] The ability to freeze the state of a data set at consistent moments in time is a useful feature for applications, databases, etc. Traditionally, data sets have been kept on disks or other storage systems managed by file systems. With advancements in file system capabilities (such as Z File System (ZFS)) merging functionality of a file system with a volume manager, efficient capabilities to version data have been made possible, at fine granularities and across large volumes of data.
Brief Description of the Drawings
[0002] Figure 1 is a block diagram illustrating a computing environment suitable for implementing aspects of a system for versioning in-memory objects according to one example.
[0003] Figure 2 is a diagram illustrating an object descriptor table (ODT) according to one example.
[0004] Figure 3 is a diagram illustrating a page table structure according to one example.
[0005] Figure 4 is a diagram illustrating a write log according to one example.
[0006] Figure 5 is a diagram illustrating a log tracking table (LTT) according to one example. [0007] Figure 6 is control flow diagram illustrating object version creation as software modifies the object according to one example.
[0008] Figure 7 is a flow diagram illustrating a method for performing a register object operation according to one example.
[0009] Figure 8 is a flow diagram illustrating a method for performing a create version operation according to one example.
[0010] Figure 9 is a flow diagram illustrating a method for performing a log a write operation according to one example.
[0011] Figure 10 is a flow diagram illustrating a method for performing a clean write log operation according to one example.
[0012] Figure 1 1 is a flow diagram illustrating a method for performing a copy object version operation according to one example.
[0013] Figure 12 is a flow diagram illustrating a method for performing a restore object version operation according to one example.
Detailed Description
[0014] In the following detailed description, reference is made to the
accompanying drawings which form a part hereof, and in which is shown by way of illustration specific examples in which the disclosure may be practiced. It is to be understood that other examples may be utilized and structural or logical changes may be made without departing from the scope of the present disclosure. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims. It is to be understood that features of the various examples described herein may be combined, in part or whole, with each other, unless specifically noted otherwise.
[0015] The ability to freeze the state of a data set at consistent moments in time is a useful feature for applications, databases, etc. Traditionally, data sets have been kept on disks or other storage systems managed by file systems. With advancements in file system capabilities (such as Z File System (ZFS)) merging functionality of a file system with a volume manager, efficient capabilities to version data have been made possible, at fine granularities and across large volumes of data.
[0016] With a growth of main memory capacities, and the introduction of byte- addressable persistent memories, applications are being redesigned to leverage the performance of such memories. As a specific example, the use of file system APIs for memory-backed data is no longer useful nor characteristic of high performance, given the mismatch in byte-addressability and the file-system read/write API. Instead, the concept of files is being replaced with memory centric models, such as memory objects. Memory objects, like files or inodes in file systems, describe data more naturally for the medium in which they exist. With file systems becoming less relevant to manage main memory systems, the desire for more capable and efficient memory management solutions is growing.
[0017] Examples disclosed herein address the lack of fast versioning capabilities for memory objects by leveraging efficient hardware mechanisms tied to the memory system. Some methods for tracking changes to memory may involve an understanding of the architectural descriptions of memory, which may be done via address translation mechanisms in the CPU, such as page tables. As store operations are inefficient to trap and log, CPUs may provide support for system software to track changes via access and dirty flags in the translation structures directly. These structures are complex and time-consuming to scan. With large memory spaces, scanning incurs high overheads to locate these flags within the structures. Alternative CPU architectures may provide binary incompatible representations of these structures, complicating system software. Furthermore, a page-to-object mapping table would have to be consulted to associate the set of dirty pages with the pages backing a specific memory object or objects.
[0018] Architectural capabilities that make state available to system software can accelerate procedures for memory object versioning by reducing the time to locate the set of dirty pages. Upon a write, the CPU may add the page referenced by the write to an in-memory log, removing the process of scanning the page tables to locate dirty pages. [0019] Some examples disclosed herein are directed to a memory management system that uses memory write logging for fast versioning of in-memory objects. In some examples, the system uses abstractions that support the notion of memory objects and object versions, and includes mechanisms to quickly identify change sets of memory objects. Examples disclosed herein have the ability to efficiently track and identify change sets (e.g., definitions) of memory objects using hardware features, and provide software simplification in identifying memory changes through hardware features that abstract
architectural representations of memory. Some examples provide the ability to track memory object modifications efficiently, leveraging hardware-provided dirty memory tracking via write modification logging and object descriptor tables. Examples disclosed herein quickly identify change sets of memory objects, and apply change sets of objects in support of object versioning.
[0020] A "memory object" according to some examples disclosed herein is a (virtually) contiguous buffer of memory, and is a system abstraction for associating memory resources with application-level interpretations of objects. These buffers may be defined in lengths of multiples of physical page sizes, as defined by the CPU architecture's memory management support.
[0021] A "memory object version" according to some examples disclosed herein is a representation of the object state from a given moment in time. A version may carry a unique identifier or key, attributes such as immutability, and a reference to the predecessor from which it is derived, etc. Versions may exist in a logical "timeline" reflecting the changes made to an object over time.
[0022] A "versionable memory object" according to some examples disclosed herein is a memory object that is capable of being versioned over its lifetime. Methods for creating versions, instantiating prior versions, etc., may be made available for use.
[0023] A "version Identifier" according to some examples disclosed herein is an identifier (typically a unique number) by which the system can uniquely refer to a memory object version within the system.
[0024] Figure 1 is a diagram illustrating a computing environment 10 suitable for implementing aspects of a system for versioning in-memory objects according to one example. In the illustrated example, the computing system or computing device 10 includes processing units 12 and system memory 14. Depending on the exact configuration and type of computing device, memory 14 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two.
[0025] Computing device 10 may also have additional features/functionality. For example, computing device 10 may also include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape. Such additional storage is illustrated in Figure 1 by removable storage 16 and non-removable storage 18. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any suitable method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Memory 14, removable storage 16 and non-removable storage 18 are all examples of non- transitory computer storage media (e.g., non-transitory computer-readable storage media storing computer-executable instructions that when executed by at least one processor cause the at least one processor to perform a method). Non-transitory computer storage media includes RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, and magnetic disk storage or other magnetic storage devices. Any such non-transitory computer storage media may be part of computing device 10.
[0026] The various elements of computing device 10 are communicatively coupled together via communication links 15. Computing device 10 also includes communication connections 24 that allow computing device 10 to communicate with other computers/applications 26. Computing device 10 may also include input device(s) 22, such as keyboard, pointing device (e.g., mouse), pen, voice input device, touch input device, scanner, etc. Computing device 10 may also include output device(s) 20, such as a display, speakers, printer, etc.
[0027] As shown in Figure 1 , a versioning of in-memory objects module 17 is stored in system memory 14. Module 17 facilitates object versioning techniques described herein. In one example, module 17 may include program instructions executable by a processor. Features of the versioning of in-memory objects according to examples disclosed herein may also be implemented in hardware. Various combinations of software and hardware, including firmware, may be utilized to implement specific examples.
[0028] Figure 1 and the above discussion are intended to provide a brief general description of a suitable computing environment in which examples may be implemented. It should be understood, however, that handheld, portable, and other computing devices of all kinds are contemplated for use. Figure 1 thus illustrates an example of a suitable computing system environment 10 in which the examples may be implemented, although as made clear above, the computing system environment 10 is one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the examples. Neither should the computing environment 10 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the example operating environment 10.
[0029] Some examples disclosed herein may use at least one of an object descriptor table, a page table, a write log, and a log tracking table, which are described in further detail below with reference to Figures 2-5, respectively.
[0030] Figure 2 is a diagram illustrating an object descriptor table (ODT) 200 according to one example. The ODT 200 according to one example is implemented in hardware and describes to the hardware a set of software- identified memory objects to track. The ODT 200 may be referenced by a separate register in hardware that stores an ODT root pointer 201 containing the physical address to one instance of an ODT. ODT 200 includes a plurality of entries 202, with each entry 202 corresponding to a particular object. Each entry 202 may contain a user-defined object identifier (ID) 204 that uniquely identifies the object, the base virtual address 206 to the object, the object's length 208, and a log field 210 to indicate a write to the object has been recorded.
[0031] Figure 3 is a diagram illustrating a page table structure 300 according to one example. The structure 300 includes a plurality of page tables 301 , with each page table 301 including a plurality of page table entries 302. Each page table entry 302 corresponds to a particular page of main memory 306 and includes a page log bit 304, which indicates whether version tracking is enabled for the corresponding page. If at least one page log bit 304 is enabled in any entry 302 within a given page table 301 , this indicates to hardware the presence of an ODT 200 immediately following the page descriptor table of that page table 301 . The page descriptor table is the last level of a page table 301 , and the ODT 200 is physically located after the page descriptor table. In this example, the ODT 200 is not referenced by a separate register in hardware. Each entry 202 in the ODT 200 includes an object ID field 204 that indicates to which objects the associated pages belong.
[0032] Figure 4 is a diagram illustrating a write log 400 according to one example. The write log 400 is a location in memory provided to the CPU to record (append) write operations as indicated by the ODT 200 (Figure 2) or other structures. The write log 400 includes a plurality of entries 402 created by the CPU. The entries 402 are organized into slabs 403. Each slab 403 contains a specific quantity of log entries 402. Each entry 402 contains information recording a write operation, such as a user-defined object identifier (ID) 404 that uniquely identifies an object that was subject to the write operation, a (machine) address 406 associated with the object, and a valid bit 408. If the valid bit 408 is zero, this indicates it has been consumed by software.
[0033] Figure 5 is a diagram illustrating a log tracking table (LTT) 500 according to one example. The LTT 500 includes a range 502, a granularity 504, and a contiguous in-memory bitmask field 506. Each bit in field 506 represents one chunk within contiguous memory at a given granularity specified by the granularity 504. The granularity 504 specifies the granularity at which memory writes are added to the write log 400 (Figure 4). Each bit in the field 506 informs the CPU whether to record a write to the given address into the write log 400. In one example, a value of zero in field 506 permits the hardware to record a write issued by software to the write log 400. The LTT 500 may be used in combination with the ODT 200 extensions to the page tables 301 .
[0034] Figure 6 is control flow diagram illustrating object version creation as software modifies the object according to one example. As shown in Figure 6, the object includes a first version 602(1 ), which includes pages 604(1 ), 604(2), and 604(3). A write operation is then performed on page 604(2) of the first version 602(1 ), and a second version 602(2) is created. The second version 602(2) includes a page reference 606(1 ) that refers to the unmodified page 604(1 ) of the first version 602(1 ); a page 606(2) that includes the data from the write operation; and a page reference 606(3) that refers to the unmodified page 604(3) of the first version 602(1 ). A third version 602(3) is then created, which includes a page reference 608(1 ) that refers to the unmodified page 604(1 ) of the first version 602(1 ); a page reference 608(2) that refers to the page 606(2) of the second version 602(2); and a page reference 608(3) that refers to the unmodified page 604(3) of the first version 602(1 ). Write operations are then performed on pages 604(1 ), 606(2), and 604(3), and a fourth version 602(4) is created that includes updated pages 610(1 ), 610(2), and 610(3).
[0035] Examples disclosed herein support the following three operations:
[0036] 1 . Register object (see Figure 7). Software may identify a virtually contiguous memory buffer for versioning, and if hardware state exists, software will additionally initialize this state for this object.
[0037] 2. Create version (see Figure 8). Software, with assistance from hardware extensions, identifies a set of dirty pages backing a memory object, and then creates a new version, e.g., by copying a subset of the object, containing the set of modified state.
[0038] 3. Log a write (see Figure 9). Hardware creates an entry to an in- memory table describing a write operation it encounters by the software.
Software-backed or hardware-backed metadata instruct this operation's behavior in when and where to create log entries.
[0039] These three operations are described in further detail below with reference to Figures 7-9.
[0040] An example implementation may register an object with software which creates metadata associating the object identifier and the set of physical backing pages for that object. Figure 7 is a flow diagram illustrating a method 700 for performing a register object operation according to one example. At 702 in method 700, a request to register an object is received. At 706, it is determined whether the ODT 200 (Figure 2) is initialized. If it is determined at 706 that the ODT 200 is initialized, the method 700 moves to 710. If it is determined at 706 that the ODT 200 is not initialized, the ODT 200 is initialized at 704, and the method 700 moves to 710.
[0041] At 710, it is determined whether the LTT 500 (Figure 5) is initialized. If it is determined at 710 that the LTT 500 is initialized, the method 700 moves to 712. If it is determined at 710 that the LTT 500 is not initialized, the LTT 500 is initialized at 708, and the method 700 moves to 712. At 712, software identifies the memory object. At 714, software creates meta-data describing the object. At 716, a new entry 202 in the ODT 200 is added for the object.
[0042] Figure 8 is a flow diagram illustrating a method 800 for performing a create version operation according to one example. At 802 in method 800, software creates new object version metadata. At 804, it is determined whether this is the first version of the object. If it is determined at 804 that this is the first version of the object, the method 800 moves to 806. At 806, references are added to all regions associated with the object to the version. At 808, a version identifier is set to an initial value. At 810, all associated log bits in either ODT 200 or LTT 500 are set to zero. At 812, the method 800 returns to client.
[0043] If it is determined at 804 that this is not the first version of the object, the method 800 moves to 814. At 814, it is determined whether the write log 400 has valid entries with object ID 404. If it is determined at 814 that the write log 400 has valid entries with object ID 404, the method 800 moves to 816. At 816, an attempt is made to allocate a buffer for a copy of the region. At 818, a determination is made whether the system is out of memory. If it is determined at 818 that the system is out of memory, the method 800 moves to 830 to free resources, and return an error to the client. If it is determined at 818 that the system is not out of memory, the method 800 moves to 820. At 820, the referenced memory region is copied to a new location. At 832, a new reference is added to the new location to object version. At 834, the log bit in either ODT 200 or LTT 500 is set to zero. At 822, the entry in the write log 400 is
invalidated, and the method 800 returns to 814. [0044] If it is determined at 814 that the write log 400 does not have valid entries with object ID 404, the method 800 moves to 824. At 824, for all regions not yet references in the version, references are copied from the prior version. At 826, the object version is incremented. At 828, the method 800 returns to client.
[0045] Figure 9 is a flow diagram illustrating a method 900 for performing a log a write operation according to one example. At 902 in method 900, software issues a store (or write) instruction. At 904, it is determined whether logging is enabled. If it is determined at 904 that logging is not enabled, the method 900 moves to 906 to resume. If it is determined at 904 that logging is enabled, the method 900 moves to 908. At 908, it is determined whether the log bit is zero. If it is determined at 908 that the log bit is not zero, the method 900 moves to 910 to resume. If it is determined at 908 that the log bit is zero, the method 900 moves to 912. At 912, hardware appends a record of the write to the write log 400. At 914, hardware increments the write log index. At 916, hardware sets the log bit to a value of one.
[0046] In one implementation, upon a request to create a new object version, dirty bit support in the page table 301 (Figure 3) enables software to scan the page table 301 for these bits for those pages backing the object. Once identified, each dirty page is copied to a new set of pages and linked with the next version identifier value. The version identifier is incremented for that object. In this implementation, write logging is not provided by the hardware and not explicitly performed by software.
[0047] Another example implementation may leverage hardware providing general page-granularity write logging for an address space. Another example implementation may use ODT 200. Registering an object creates a new entry 202 into the ODT 200. Should software request to track objects at granularities equal to their length, a log field 210 may be supported in each ODT entry 202 to indicate a write to the object has been recorded - a log bit - used as follows. When the hardware encounters a write to an address, it consults the ODT 200, and if an object is found to contain the address, and if the entry's log bit 210 is zero, it sets the log bit 210 to one and records the write to the write log 400 (Figure 4), storing the user-defined value provided in the entry, and the physical base address. In some examples, software is responsible for clearing the log bits.
[0048] Another example implementation may leverage hardware extensions to the page table 300 to track large objects using a granularity independent of the virtual memory translations system. Such is provided by the LTT 500. In each page table entry 302 is a page log bit 304 indicating whether tracking is enabled for this page. If at least one such bit is enabled in a given entry 302 within a page table 301 , hardware then expects to locate an ODT 200 for the set of translations associated with the page table. Upon a write by software, hardware will examine the page log bit 304, and if set, examine the associated bit in the LTT 500 using the address from the write and the granularity configured for the LTT 500. If the associated bit in the LTT 500 is zero, hardware records this write into the write log 400 with the physical address and object identifier. The LTT bit is then monotonically incremented.
[0049] Some examples perform management operations on the write log 400, e.g., to optimize it for scanning operations. Examples disclosed herein support the following operation:
[0050] 1 . Clean log (see Figure 10). The write log 400 may become fragmented due to the request stream by software to version objects, whereby some entries 402 become invalid. Subsequent version requests may incur costs of scanning the entire log 400, based on the location of the first valid log entry.
[0051] An example implementation of log cleaning manages the write log 400 in the granularity of memory slabs 403. Each slab 403 contains a specific quantity of log entries 402, with fields 404, 406, and 408. When software performs object versioning, it will scan the write log 400 linearly to locate entries 402 associated with the object to compose the state for the next version, invalidating each entry it consumes.
[0052] Cleaning the write log 400 is a method for compressing the state of the log 400, discarding invalid entries and releasing memory backing each slab 403 to the system. For slabs 403 prior to the current append location in the log 400, all valid entries 402 are counted and copied (in a stable manner, preserving their order) to another set of empty slabs 403. The scan direction may begin with the last entry 402 in the slab 403 immediately preceding the current slab 403, until the first valid entry 402 is reached, skipping invalid entries 402. The compressed slabs 403 become the new head of the write log 400 used for scanning when new object versions are created.
[0053] Figure 10 is a flow diagram illustrating a method 1000 for performing a clean write log operation according to one example. At 1002 in method 1000, valid entries in write log slabs 403 between the head and the current index of the write log 400 are counted. At 1004, it is determined whether the number of entries identified at 1002 is greater than the slab capacity. If it is determined at 1004 that the number of entries is not greater than the slab capacity, the method 1000 moves to 1006 to return to client. If it is determined at 1004 that the number of entries is greater than the slab capacity, the method 1000 moves to 1008. At 1008, new slabs sufficient to contain the number of entries identified at 1002 are allocated. At 1010, it is determined whether there is available memory. If it is determined at 1010 that there is no available memory, the method 1000 moves to 1012 to return an error to the client. If it is determined at 1010 that there is available memory, the method 1000 moves to 1014. At 1014, all valid entries are copied to new slabs 403, and the original order of the entries is preserved. At 1016, the write log head is updated. At 1018, the method frees the old slabs 403.
[0054] Some examples perform operations on object versions, e.g., to access or delete older versions. Examples disclosed herein support the following four operations:
[0055] 1 . Copy object version (see Figure 1 1 ). A previously created object version may be provided to software for use. This instance does not have write tracking enabled, as it is provided initially as a non-versionable memory object. An example implementation of the copy object version operation scans the object version state to locate the most recent instance of each page of the object (where the granularity of a page is defined by the logging granularity), and copy that the content to a provided buffer, beginning with the version of the object requested. [0056] 2. Restore object version (see Figure 12). Similar to operation 1 (copy object version), except the system discards all versions newer than the provided version in the object's history and resumes tracking. An example
implementation of the restore object version operation performs a version copy operation, with the added effect that newer versions are removed from the history, and all buffers backing tracked writes are released. The buffer holding the object is overwritten with the state from the associated version.
[0057] 3. Delete object version. A version may be removed from the recorded history, which involves merging of adjacent version states. This operation may be used to remove a range of versions, e.g., versions older than a specific time stamp. A configurable behavior of this may, instead of merging a set of prior versions, remove those versions entirely, altering what is seen as the original derived version in the history. An example implementation of the delete object version operation moves the change set held by a version and merges it into the prior version, updating any references from the prior version to point to the subsequent version (or current copy of the object).
[0058] 4. Branch object version. A prior object version is provided as a new derived instance of the object to software, with write tracking enabled. New versions derived from this instance are recorded to a separate "branch" in the version tree.
[0059] Figure 1 1 is a flow diagram illustrating a method 1 100 for performing a copy object version operation according to one example. At 1 102 in method 1 100, the client presents a buffer to software to receive an object version copy. At 1 104, it is determined whether the buffer is too small. If it is determined at 1 104 that the buffer is too small, the method 1 100 moves to 1 106 to return an error to the client. If it is determined at 1 104 that the buffer is not too small, the method 1 100 moves to 1 108. At 1 108, it is determined whether regions remain to be copied. If it is determined at 1 108 that regions do not remain to be copied, the method 1 100 moves to 1 1 10 to return to client. If it is determined at 1 108 that regions do remain to be copied, the method 1 100 moves to 1 1 12. At 1 1 12, the region referenced by the version is copied to the region in the buffer, and the method 1 100 returns to 1 108. [0060] Figure 12 is a flow diagram illustrating a method 1200 for performing a restore object version operation according to one example. At 1202 in method 1200, for each region in the object, the region referenced in the specified version is copied into the region of the object. At 1204, the latest version ID is updated to the specified version ID. At 1206, resources for all newer versions of the object are released. At 1208, all associated write log tracking bits in ODT 200 or LTT 500 are reset.
[0061] One example of the present disclosure is directed to a memory
management system. The system includes a processor, and a write log to allow the processor to record write log entries, wherein each write log entry
represents a write to main memory that has been performed. The system includes at least one table to facilitate versioning of in-memory objects stored in the main memory, and system software to track changes to the in-memory objects and create new versions of the in-memory objects using the write log and the at least one table.
[0062] The at least one table may include an object descriptor table (ODT) including a plurality of ODT entries, wherein each ODT entry corresponds to one of the in-memory objects. Each ODT entry may include an object ID, an object address, and an object length. Each ODT entry may further include a log bit to indicate when a write to the object corresponding to the entry has been performed.
[0063] The at least one table may include a page table including a plurality of page table entries, wherein each page table entry includes a page log bit that indicates whether version tracking is enabled. If the page log bit indicates that version tracking is enabled, an object descriptor table (ODT) may immediately follow the page table and include a plurality of ODT entries, wherein each ODT entry corresponds to one of the in-memory objects.
[0064] Each write log entry may include an object identifier that identifies one of the in-memory objects that was subject to a write, and an address associated with the identified one of the in-memory objects. Each write log entry may further include a valid bit indicating whether the write log entry has been consumed by the system software. [0065] The at least one table may include a log tracking table that includes a granularity field and a bitmask field storing a plurality of bits, wherein each bit in the bitmask field corresponds to one contiguous chunk within the main memory at a granularity specified in the granularity field. Each bit in the bitmask field may indicate whether the processor is to record a write in the write log.
[0066] The memory management system may support at least one of: a register object operation to allow the system software to register a new in-memory object; a create version operation to allow the system software to create a new version of one of the in-memory objects; a clean log operation to cause invalid entries in the write log to be discarded; a copy object version operation to copy a version of one of the in-memory objects to a buffer specified by the system software; a restore object version operation to discard all versions newer than a specified version; and a delete object version operation to remove an object version or a range of object versions.
[0067] Another example of the present disclosure is directed to a method for memory management. The method includes recording, with a processor, write log entries into a write log, wherein each write log entry represents a write to main memory that has been performed. The method includes tracking, with system software, changes to in-memory objects stored in the main memory based on the write log and at least one table. The method includes creating, with the system software, new versions of the in-memory objects based on the tracking. Each write log entry may identify one of the in-memory objects, and includes a valid bit indicating whether the write log entry has been consumed by the system software.
[0068] Yet another example of the present disclosure is directed to a memory management system, which includes a processor, and a write log to allow the processor to record write log entries, wherein each write log entry represents a write to main memory that has been performed, and identifies an in-memory object that was affected by the write. The system includes at least one table to provide information for versioning the in-memory object, and system software to track changes to the in-memory object and create a new version of the in- memory object using the write log and the at least one table. Each write log entry may include a valid indicator indicating whether the write log entry has been consumed by the system software.
[0069] Although specific examples have been illustrated and described herein, a variety of alternate and/or equivalent implementations may be substituted for the specific examples shown and described without departing from the scope of the present disclosure. This application is intended to cover any adaptations or variations of the specific examples discussed herein. Therefore, it is intended that this disclosure be limited only by the claims and the equivalents thereof.

Claims

1 . A memory management system, comprising:
a processor;
a write log to allow the processor to record write log entries, wherein each write log entry represents a write to main memory that has been performed;
at least one table to facilitate versioning of in-memory objects stored in the main memory; and
system software to track changes to the in-memory objects and create new versions of the in-memory objects using the write log and the at least one table.
2. The memory management system of claim 1 , wherein the at least one table includes an object descriptor table (ODT) including a plurality of ODT entries, wherein each ODT entry corresponds to one of the in-memory objects.
3. The memory management system of claim 2, wherein each ODT entry includes an object ID, an object address, and an object length.
4. The memory management system of claim 3, wherein each ODT entry further includes a log bit to indicate when a write to the object corresponding to the entry has been performed.
5. The memory management system of claim 1 , wherein the at least one table includes a page table including a plurality of page table entries, wherein each page table entry includes a page log bit that indicates whether version tracking is enabled.
6. The memory management system of claim 5, wherein, if the page log bit indicates that version tracking is enabled, an object descriptor table (ODT) immediately follows the page table and includes a plurality of ODT entries, wherein each ODT entry corresponds to one of the in-memory objects.
7. The memory management system of claim 1 , wherein each write log entry includes an object identifier that identifies one of the in-memory objects that was subject to a write, and an address associated with the identified one of the in-memory objects.
8. The memory management system of claim 7, wherein each write log entry further includes a valid bit indicating whether the write log entry has been consumed by the system software.
9. The memory management system of claim 1 , wherein the at least one table includes a log tracking table that includes a granularity field and a bitmask field storing a plurality of bits, wherein each bit in the bitmask field corresponds to one contiguous chunk within the main memory at a granularity specified in the granularity field.
10. The memory management system of claim 9, wherein each bit in the bitmask field indicates whether the processor is to record a write in the write log.
1 1 . The memory management system of claim 1 , wherein the memory management system supports at least one of: a register object operation to allow the system software to register a new in-memory object; a create version operation to allow the system software to create a new version of one of the in- memory objects; a clean log operation to cause invalid entries in the write log to be discarded; a copy object version operation to copy a version of one of the in- memory objects to a buffer specified by the system software; a restore object version operation to discard all versions newer than a specified version; and a delete object version operation to remove an object version or a range of object versions.
12. A method for memory management, comprising:
recording, with a processor, write log entries into a write log, wherein each write log entry represents a write to main memory that has been performed;
tracking, with system software, changes to in-memory objects stored in the main memory based on the write log and at least one table; and
creating, with the system software, new versions of the in-memory objects based on the tracking.
13. The method of claim 12, wherein each write log entry identifies one of the in-memory objects, and includes a valid bit indicating whether the write log entry has been consumed by the system software.
14. A memory management system, comprising:
a processor;
a write log to allow the processor to record write log entries, wherein each write log entry represents a write to main memory that has been performed, and identifies an in-memory object that was affected by the write; at least one table to provide information for versioning the in-memory object; and
system software to track changes to the in-memory object and create a new version of the in-memory object using the write log and the at least one table.
15. The memory management system of claim 14, wherein each write log entry includes a valid indicator indicating whether the write log entry has been consumed by the system software.
PCT/US2016/015839 2016-01-29 2016-01-29 Memory management with versioning of objects WO2017131789A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2016/015839 WO2017131789A1 (en) 2016-01-29 2016-01-29 Memory management with versioning of objects

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2016/015839 WO2017131789A1 (en) 2016-01-29 2016-01-29 Memory management with versioning of objects

Publications (1)

Publication Number Publication Date
WO2017131789A1 true WO2017131789A1 (en) 2017-08-03

Family

ID=59398370

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/015839 WO2017131789A1 (en) 2016-01-29 2016-01-29 Memory management with versioning of objects

Country Status (1)

Country Link
WO (1) WO2017131789A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111783735A (en) * 2020-07-22 2020-10-16 欧冶云商股份有限公司 Steel document analytic system based on artificial intelligence

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040054644A1 (en) * 2002-09-16 2004-03-18 Oracle Corporation Method and mechanism for implementing in-memory transaction logging records
US20120078855A1 (en) * 2010-09-29 2012-03-29 Louis Beatty Method and system of performing a granular restore of a database from a differential backup
US20120323971A1 (en) * 2011-06-14 2012-12-20 Sybase, Inc. Optimizing data storage and access of an in-memory database
WO2013095635A1 (en) * 2011-12-23 2013-06-27 Intel Corporation Instruction for merging mask patterns
US20140164828A1 (en) * 2012-12-07 2014-06-12 International Business Machines Corporation Consistency of data in persistent memory

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040054644A1 (en) * 2002-09-16 2004-03-18 Oracle Corporation Method and mechanism for implementing in-memory transaction logging records
US20120078855A1 (en) * 2010-09-29 2012-03-29 Louis Beatty Method and system of performing a granular restore of a database from a differential backup
US20120323971A1 (en) * 2011-06-14 2012-12-20 Sybase, Inc. Optimizing data storage and access of an in-memory database
WO2013095635A1 (en) * 2011-12-23 2013-06-27 Intel Corporation Instruction for merging mask patterns
US20140164828A1 (en) * 2012-12-07 2014-06-12 International Business Machines Corporation Consistency of data in persistent memory

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111783735A (en) * 2020-07-22 2020-10-16 欧冶云商股份有限公司 Steel document analytic system based on artificial intelligence
CN111783735B (en) * 2020-07-22 2021-01-22 欧冶云商股份有限公司 Steel document analytic system based on artificial intelligence

Similar Documents

Publication Publication Date Title
US10649910B2 (en) Persistent memory for key-value storage
US10296219B2 (en) Data deduplication in a block-based storage system
US10776315B2 (en) Efficient and flexible organization and management of file metadata
US9519575B2 (en) Conditional iteration for a non-volatile device
US11461027B2 (en) Deduplication-aware load balancing in distributed storage systems
US8868624B2 (en) Blob manipulation in an integrated structured storage system
US10318495B2 (en) Snapshots for a non-volatile device
US8620884B2 (en) Scalable blob storage integrated with scalable structured storage
US10176190B2 (en) Data integrity and loss resistance in high performance and high capacity storage deduplication
US10956388B2 (en) Eventual consistency in a deduplicated cloud storage system
US20140089265A1 (en) Time Sequence Data Management
US20120158674A1 (en) Indexing for deduplication
JP2011513853A (en) Associative storage system and method
US20220382760A1 (en) High-performance key-value store
US11494334B2 (en) Embedded reference counts for file clones
US20210397345A1 (en) Managing i/o amplification in log-structured merge trees
US11741073B2 (en) Granularly timestamped concurrency control for key-value store
US11016676B2 (en) Spot coalescing of distributed data concurrent with storage I/O operations
Koo et al. Modernizing file system through in-storage indexing
US20230083104A1 (en) Efficiently Deleting Snapshots in a Log-Structured File System (LFS)-Based Storage System
US11960450B2 (en) Enhancing efficiency of segment cleaning for a log-structured file system
WO2017131789A1 (en) Memory management with versioning of objects
US11620270B2 (en) Representing and managing sampled data in storage systems
US20230079621A1 (en) Garbage collection from archival of storage snapshots
TW201710930A (en) SWAT command and API for atomic swap and trim of logical pages

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16888517

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16888517

Country of ref document: EP

Kind code of ref document: A1