US20050234986A1 - Systems and methods for fragment-based serialization - Google Patents
Systems and methods for fragment-based serialization Download PDFInfo
- Publication number
- US20050234986A1 US20050234986A1 US10/821,687 US82168704A US2005234986A1 US 20050234986 A1 US20050234986 A1 US 20050234986A1 US 82168704 A US82168704 A US 82168704A US 2005234986 A1 US2005234986 A1 US 2005234986A1
- Authority
- US
- United States
- Prior art keywords
- data
- fragment
- members
- type
- byte
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4488—Object-oriented
- G06F9/4493—Object persistence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
- G06F16/90348—Query processing by searching ordered data, e.g. alpha-numerically ordered data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
Definitions
- This invention relates to computing, and more particularly to storage and transmission of data objects.
- Serialization can be defined as the process of storing the state of an object instance to a storage medium. During this process, the public and private fields of an object and the name of the class, are converted to a stream of bytes, which is then written to a data stream. When an object is subsequently deserialized, an exact clone of the original object may be created.
- an object in active computer memory for example, an object with data describing a person.
- the person object has a number of subcomponent members, such as name, address, social security number, phone numbers, spouse, height and weight. While the person's name may be important for a particular application, the height and weight may not be. Thus, the name may remain in active memory where it may be modified, while other fields such as height and weight are evicted from active memory to make room for other data.
- the person object may no longer be needed by the application, and it may be persisted or transmitted to another computer.
- the object To persist or transmit an object, the object must be serialized, which refers to formatting an object in a useful, retrievable way.
- the members of an object are generally uniform for all objects of the same class.
- Each person object for example, has the name, address, social security number, phone numbers, spouse, height and weight members.
- the information changes from person to person, and for some people the information may be unavailable (“null”), but the existence of the same member fields is generally present for all person objects of the person class.
- a person class may be thought of as the generic person object.
- a person object is one instance of a person class. This concept of a class and an instance of a class exists in many programming languages. Regardless of the programming language involved, serialization is typically performed on instances of a class, generating serialized objects.
- Objects may comprise members with various types of data.
- the members may be primitive or complex.
- primitive members are “string” such as the name member from the person object, which is a string of letters; and “integer,” such as the social security number from the person object, which is an integer.
- complex members are “collection,” such as the phone numbers member, which comprises more than one primitive—in this case, more than one integer; “nested,” which is a member that has some structure beyond a simple primitive member, e.g., the collection of phone numbers, or the spouse member, which refers to another person object; and “subtype,” such as a hypothetical “United States address” type that would be a subtype of an address type, and therefore presumably declares additional members such as a U.S. region or U.S. Post Office Box.
- Members may be described in many different ways, and relate to each other in any number of patterns. Therefore serializing objects such as the person object involves effectively dealing with the various members and the relationships of those members that may be included in the object.
- Serialization of objects presents a number of challenges in the industry. Serialized objects should consume as little storage space as possible. If the size of an object is greatly increased when it is serialized, then the storage cost of the object may be too high. Therefore, compact representation is an important aspect of a serialization format.
- Serialized objects should also be efficiently instantiated into active memory. If the processing cost of finding and assimilating the various members of a serialized object is high, it will drain valuable processor resources. Likewise, serialization should allow for instantiation and updating of members of an object without the need to instantiate the entire object. Instantiating the entire person object, for example, only to read or update the person's social security number is a waste of active memory resources needed to store the name, phone number, address, etc. when those members are not involved in the operation.
- Serialization formats should also support all data types that may be contained in an object.
- a very basic serialization format might only support primitives, but more sophisticated formats should support complex members such as the nested members, collection members, and subtype members described above. While a serialization format should be optimal for objects with few levels of nesting and inheritance, because most objects have this characteristic, it should also support many levels of nesting and inheritance to ensure that the serialization can be flexibly used for a broad range of classes.
- a serialization format should also be flexible in handling very large members. Some members may be, for example, a music file, a photograph, or a movie, and such large members pose a challenge in serialization that will be explained in greater detail below.
- XML serialization provides a token for each member.
- the token comprises metadata that identifies a member, usually a member immediately following the token. Therefore, XML serialization may be visualized as follows:
- serialization format The problems with such a serialization format are, first, verbosity: the storage of metadata tokens with each and every member consumes a large amount of disk space. Second, retrieval is impaired in such a format, because in order to find a desired member, the tokens must be searched. This may involve a high active memory cost, because the most effective way to read or update an object that is serialized in this manner may be to instantiate the entire object.
- serialization format is in the “Storage Engine record” format, also referred to as the “SE record,” or simply “record” format.
- SE record Storage Engine record
- record This is an a typical database system record format.
- members for objects of a given class are stored in uniformly formatted records. Instead of providing metadata that describes each and every member, there is metadata that describes the contents of all the records for objects of a particular class. This can be visualized as provided in FIG. 10 .
- the SE record serialization format does not require metadata with each individual member, so it is a more compact serialization technique. Instead, it requires access to metadata describing the layout of the members on disk, such as the Metadata for Person Objects table of FIG. 10 .
- a weakness of the SE record format is that it is inflexible in handling members of variable length, such as many of the music files, movies, and images that are stored with objects today. More accurately, flexibility in the SE record serialization comes at a high processing cost.
- Members of variable length can be stored in such a format, if an offset table is used to identify the locations of variable length data in the record. The consequence of storing an offset table is that whenever a variable length member is updated, the positions of all variable length data that follows it must be adjusted. This can be compared to inserting bytes in the middle of an array—everything to the right of an insert point must be shifted right to make space for inserted new bytes.
- a user can define his own serialization format, use Common Language Runtime (“CLR”) serialization (provided by C# language itself), or let the SQL server store an object in its own format.
- CLR Common Language Runtime
- a method and system for fragment-based serialization places one or more members in fragments.
- Fragments may comprise a header and a payload.
- a header can provide useful information about the fragment, such as an indication of fragment type and an indication of fragment length.
- a payload may comprise one or more members of an object.
- Primitive members may be stored in a fragment with a record format payload. This configuration allows for fast location and updating of primitives.
- Large Object (“LOB”) members may be stored in fragments that have a field for setting forth location types for locations of LOB and FS members.
- Collections may be stored in a series of fragments, a first fragment to indicate a start of a collection, one or more second fragments to serialize collection elements, and a terminator fragment to indicate the end of a collection. These and other fragment types may be organized according to rules that govern generating fragments, placing members in fragments, and sequencing fragments in a manner that provides additional functionality to the serialization format.
- FIG. 1 is a conceptual illustration of the various fragments which may be used to serialize object members. It shows a Binary Fragment with a payload comprising primitive members in record format, a fragment with a non-record format payload, and a fragment with no payload.
- FIG. 2 presents a fragment with a detailed view of the fragment header.
- the header shows a selection of potential fields for use in fragment headers, and many fragment headers may omit some of the fields shown.
- FIG. 3 presents several exemplary object classes for which fragment sequences are provided in the description, in accordance with various embodiments of the invention.
- FIG. 4 is a flowchart demonstrating steps for generating fragments for the primitive members of an object when there are no nested members in the object.
- FIG. 5 is a flowchart demonstrating steps for generating fragments for the primitive members of an object when there are nested members in the object.
- FIG. 6 is a flowchart demonstrating steps for generating fragments for collection members of an object.
- FIG. 7 is a flowchart demonstrating steps for generating fragments for the LOB and FS members of an object.
- FIG. 8 is a flowchart demonstrating steps for the process of placing an entire object, with members various types, into fragments.
- FIG. 9 illustrates objects that have been serialized in accordance with various embodiments of the invention as they may be stored in a single column of a database.
- FIG. 10 illustrates a prior art record serialization format in which metadata is provided for all records, and corresponding data conforms to the format specified in the metadata.
- FIGS. 11 illustrate various fragment types for use in serializing data in accordance with preferred embodiments of the invention.
- FIG. 12 is a top-level diagram of a fragment sequence for the tPartTimeEmployee object displayed in FIG. 3 .
- This fragment sequence may contain additional fragments for each level of nesting.
- a serialization that provides compact representation is provided.
- Objects serialized in the provided format can be efficiently instantiated into active memory, thereby reducing processing costs of finding and assimilating the various members of a serialized object.
- members of objects may be instantiated and updated without the need to instantiate an entire object.
- support for a wide range of data types, including user-defined data types (“UDTs”) is provided.
- the serialization format can be optimized for objects with few levels of nesting and inheritance but also supports many levels of nesting and inheritance. It is flexible in handling very large members.
- the invention can provide a serialization format that is suitable for storage of various types in a single column—for example, an instance of an “employee” object that is a subtype of a “person” object can be stored in a column that is provided to store only “person” objects.
- the serialization format allows for efficient addition of new members to a type, also called efficient type evolution.
- Fragment-based serialization in accordance with various embodiments of the invention can be conceptualized as a hybrid format that employs some of the elements of XML-style serialization, as described in the background, and some of the elements of SE record serialization, also described in the background, in addition to numerous aspects and advantages that are unique to fragment-based serialization itself.
- the members of an object can be placed into fragments. Fragments are displayed in FIG. 1 .
- a fragment may comprise a header and in some cases a payload.
- the header can provide useful information about the fragment, such as an indication of a fragment type, and an indication of a fragment length.
- This header is somewhat similar to the tokens provided by XML serialization, because a new header is given for each fragment, just as a token is provided for each member in the XML serialization context.
- XML tokens are provided for each member
- the fragments of FIG. 1 can include more than one member. This is illustrated in fragment 1, which shows a fragment with a payload comprising several data members.
- Fragment-based methods can serialize and deserialize a variety of data structures including but not limited to: objects with simple primitive fields (integers, strings, etc), entire graphs of connected objects, and collections.
- the fragment payload can comprise a member or members of a serialized object, as well as any other data.
- This payload can employ SE record format-style serialization for the members inside, which allows for speedy retrieval of members in the payload.
- Such a record format payload is a characteristic of Fragment 1 in FIG. 1 .
- the fragment-based serialization has features of SE record serialization. Metadata can be provided in a header or elsewhere that describes the fields included in a payload. The corresponding advantages of compact representation and retrieval of individual members without instantiating an entire object can therefore be achieved. Note that while a payload can be in record format, it need not be, as is illustrated in Fragments 2 and 3.
- the header portion of a fragment may contain various fields, as illustrated in FIG. 2 .
- FIG. 2 shows a fragment with an expanded header section so the various possible fields may be illustrated. Note that the various fields provided in FIG. 2 need not be included in every fragment. Instead, the fields that provide useful information for the payload of the fragment may be included in the header. The various fields of FIG. 2 will be explained in greater detail below, in connection with the description of various suggested types of fragments.
- Various embodiments of the invention utilize multiple types of fragments for additional versatility in serializing objects.
- a variety of suggested fragment types is set forth below.
- One motivating factor is to allow for better serialization of the various types of members that may make up an object. Recall from the background section that objects frequently comprise multiple members of different types. These members may be, for example:
- the invention can be used with several fragment types.
- One or more fragment types may be useful for only one type of member, while other fragment types are useful for multiple member types.
- fragment types can have different formats that are tailored to the contents of the fragment.
- a fragment type is first set forth, followed by a visual depiction of a suggested fragment format for that type. Fragment attributes in italics are optional and depend on the value of the Type column.
- the invention is not limited to the fragment types set forth below.
- new fragment types may be developed for use in accordance with the general principles of fragment-based serialization provided herein.
- FIG. 11 displays various potential embodiments of a Binary Fragment.
- This fragment can contain a Type, Length, and Payload field.
- the type field may be just one byte, or it can be any number of bytes. Additional bytes in the Type field will require additional memory overhead when using the serialization format. Therefore bytes in header fields should be used sparingly.
- a one-byte Type field can include a number of bits for use in indicating various properties of a fragment. One bit may be used to indicate that a fragment is a Binary Fragment. Another bit may be used to indicate a type of member or members contained in the fragment. For example, if all members are primitive, a bit may be set to indicate such information.
- a bit may be set to so indicate. If the Binary Fragment is the first, or the only, fragment for a serialized object, A bit in the Type field may so indicate.
- the type field may also indicate an object type contained in a fragment or fragments, as well as any additional useful information such as the number and types of fragments in the entire object.
- Objects represented with in a single binary fragment may be flagged in the Type field as “self-terminating” fragments, eliminating any need to include a Terminator Fragment at the end of a serialized object.
- This “self-terminator” flag may be in the form of a self-terminator bit in the Type field of a fragment.
- Such a self-terminator bit could also be located in any other field of a fragment header, or in a fragment payload.
- Objects that are represented by multiple fragments need not set the self-terminator bit, because a Terminator Fragment can be generated to mark the end of the serialized object.
- the Length field is optimally 2 bytes, although the length may vary as described above.
- the Length field may be used to indicate a length of a payload.
- the payload in a Binary Fragment may contain any data.
- the payload contains all primitive members of an object.
- the payload in such a fragment can be an SE record, to allow for efficient cracking and updating of the primitives or other members stored therein.
- FIG. 11 (B) displays various potential embodiments of a LOB Fragment.
- This fragment may have Type, Value Type, and Length fields in the header, and a payload containing a LOB or location information for a LOB.
- the Type field as in each of the fragments, need only be one byte, indicating in this case that the fragment is a LOB Fragment.
- the Value Type field may provide additional means to describe the contents of the LOB Fragment.
- Such a Value Type field may be added to contain information about LOB attributes in implementations where it is not desirable to use up bits in the Type field for LOB attributes. This way, only LOB Fragments have the overhead (here, an additional one byte per fragment).
- Information stored in the Value Type field can describe a type of location in which a LOB is stored. Allowing additional description of the location of LOB members provides flexibility in handling large values.
- LOB data in contrast to a LOB reference
- an application or user of a computer
- the LOB can initiate the generation of a LOB Inlined type fragment, use an 8 byte length, and put the LOB inline.
- the LOB can be placed in the payload of the LOB Fragment.
- the Value Type field indicates a LOB Inlined type
- the length field can be, for example, 8 bytes and the payload can contain a LOB value.
- the Value Type field can indicate a LOB Pointer type, meaning that the payload of the fragment contains a pointer to a LOB location.
- the Length field can be, for example, 2 bytes, and the payload can contain a LOB reference.
- the Value Type field can also indicate a LOB Delayed type which can mean that the fragment payload contains a LOB reference to a cell in a database that presumably contains a LOB.
- the fragment length can be, for example, 2 bytes and the payload can contain a cell reference.
- a ‘cell reference’ is a combination of a table identifier, a row identifier, and a column identifier.
- a cell reference When combined with a ‘path’ (as described below) for a LOB fragment, a cell reference gives enough information to locate actual LOB data. Any other location type information can be contained in the Value Type field of a LOB or FS fragment. Providing such an additional location type field for LOB and FS Fragments gives the serialization format additional flexibility while keeping overhead low.
- any of the fragments discussed herein may be null if a particular object does not have a member that is provided for in a serialization for a particular class. If a fragment is null, this information can be set in a bit in the type field for the fragment. In this regard, the Length field and the payload may be removed from the LOB fragment of FIG. 11 (B) to form a Null LOB Fragment with any location specified in the Value Type field.
- FIG. 11 (C) displays various possible embodiments for an FS Fragment.
- the FS Fragment can have a Type field that indicates a fragment type—here, an FS Fragment.
- the FS Fragment can include a Value Type field. Once again, this field can indicate various location types for an FS.
- the FS could be serialized with the rest of the object, or Inline type (again, this can correlate to a larger length field, e.g., 8 bytes).
- FS Pointer type which in the context of an FS, may indicate a length field of, for example, 2 bytes and a payload that contains a Global Unique Identifier (“GUID”) for the appropriate FS file.
- GUID Global Unique Identifier
- the Value Type field could also indicate an FS Delayed location type, which can correlate to a length of, for example, 2 bytes, and a payload that contains a cell reference.
- FIG. 11 (D) shows various possible embodiments for a Terminator Fragment.
- the Type byte is relevant for the Terminator Fragment. This is because the function of the Terminator Fragment is to mark the end of a serialized object, or to mark the end of a collection or other set of related fragments within a serialized object.
- the Terminator Fragment can perform this function with a Type filed indicating that it is a Terminator Fragment, and need not include additional information. However, it may be useful to include some additional information with the Terminator Fragment, and such embodiments are certainly within the scope of the invention described herein.
- FIG. 11 (E) displays various possible embodiments for a Collection Start Fragment.
- This fragment may contain a Type field and a Bit field of, for example, 2 bytes.
- the Type filed may indicate that the fragment is a Collection Start Fragment.
- the Bit field may indicate properties of a collection. For example, the bit field may indicate an “unordered” collection, which can correspond to a collection that is not in any particular order. It could also indicate and “ordered” collection, indicating that a collection is in a particular order.
- This fragment when used only for the purpose of describing a collection, can omit the length field, because the Collection Start Field marks the start of a collection, and as such need not contain a payload.
- the Collection Start Fragment is used to mark and describe a collection, it does not have a payload of its own and therefore need not have a length field either.
- a Null Collection Start Fragment will look very much like the Collection Start Fragment in FIG. 11 (E). The only difference in the situation where the Collection Start Fragment is null is the bit set in the Type field, as described above in connection with Null LOB Fragments.
- FIG. 11 (F) displays various possible embodiments for a Collection Element Fragment.
- the Type field in such fragment can indicate that it is a Collection Element Fragment.
- the Length field can indicate a length of the payload of the Collection Element Fragment.
- FIG. 11 (F) shows an exemplary Length field size of 2 bytes, which should be sufficient to indicate the length of a payload containing a collection element.
- a Locator field may also be included with Collection Element Fragments.
- the Locator field like the Value Type field from the LOB and FS fragments, can be used to indicate additional properties of a Collection Element Fragment.
- Collection Element Fragment may have a payload in SE record format, such as that of the Binary Fragment.
- the Type field may indicate whether the Collection Element Fragment is a Self-Terminator, by using a bit to indicate whether a fragment terminates itself. If the self-terminator bit is not set, the system can anticipate a Terminator Fragment for the fragment.
- the Locator field can be used to address a particular element of a collection very much like the GUID of a FS fragment. In the case of a Collection Element Fragment, however the locator field indicates a unique location within a collection and not necessarily a globally unique location.
- Locator field it may also be preferable to allow for some anticipation of a locator field in a collection element.
- a the bit in the Bit field of the Collection Start Fragment may be set to indicate an upcoming Collection Element Fragment with a Locator field.
- the system can be configured to deduce that Locator field is present in the Collection Element Fragment.
- Null Collection Element Fragment FIG. 11 (H) displays various possible embodiments for a Null Collection Element Fragment.
- Null representation for Collection Element Fragments may contain a Type field indicating that the Fragment is a Null Collection Element Fragment. They may also contain a Locator field, but need not contain the Length or payload fields, because the existence of the Null Collection Element Fragment indicates that the particular serialized object does not have data corresponding to a particular aspect of a class that is otherwise designed to contain such data. Again, when no members or other member information are included in a payload, there may be no need for a Length field to describe the length of the payload.
- FIG. 11 (G) displays various possible embodiments for a Null Fragment.
- the Null Fragment like the Terminator Fragment, can be represented by a single Type field. Again, this is practice may be restricted to representation of non-collection element null fragments. See below for a description of Collection Element Null Fragments.
- Metadata and Annotation Fragments may be used to describe one or more fragments to a receiver of a serialized object. Such fragments are useful in a variety of situations even though they may not be necessary to deserialize an object.
- a annotation fragment may allow a client to check information about a particular member or object, or to insert notes or information about a serialized object.
- representation overhead refers to an amount of additional information stored with an object to allow the object to be effectively retrieved.
- the fragment-based serialization techniques do contain representation overhead, but the overhead is minimized for the corresponding flexibility and functionality of the format.
- the first field in a fragment header is the Type field.
- the Type field consumes 1 byte. This minimizes the associated overhead.
- the Locator field associated with the Collection Element Fragment amounts to overhead. Most small unordered collections can be adequately represented in a Locator field that consumes no more than 4 bytes. Larger, ordered collections may consume more than 4 bytes, however, and in this case the Locator field may be replaced with a variable binary (“varbinary”) field that may require more representation overhead.
- Varbinary variable binary
- Length fields associated with several other fragment types are representation overhead.
- the Length field in preferred embodiments can be either 2 bytes or 8 bytes long, depending on the fragment type.
- the invention is not limited to the exact number of bytes for such fields, and the parameters set forth herein should be considered useful tips from an experienced practitioner and not hard and fast requirements for the invention itself.
- the various fragment types may serve a variety of fragment members.
- An object containing, for example, the primitive, nested, collection, and subtype members can be decomposed into fragments according to a set of rules. While the invention is not limited to a particular correlation of fragment types and member types, a useful set of rules has been developed and is explained in this section. Note that the rules are for clarity of explanation, not to indicate that they must be carried out in a particular order. In practice, the operations corresponding to the rules below may be carried out simultaneously, with the generation, population, and sequencing of fragments conducted as a processor steps through the members of an object. For an exemplary application of these rules to the serialization of an object, refer to FIG. 3 and corresponding text, below. The rules are as follows.
- Embodiments of the invention can be said to engage in type-based container relative fragment generation.
- Additional fragments may be generated to suit the needs of a particular class. Likewise, the above fragments may not be required for serialization of some classes.
- an object may be translated into a fragment-based serialization fragment using a top-to-bottom technique.
- any base type members in an object may be serialized, followed by subtypes.
- a scan may be made of the contained members to determine if there are any nested types, subtypes or LOB/FS type members.
- Some or all primitive members may be placed into one or more Binary Fragments.
- a preferred embodiment handles objects with no nested members differently from objects with nested members. These two scenarios are depicted in FIG. 4 and FIG. 5 .
- the non-nested primitives may be placed inside a single Binary Fragment, and serialized therein using an SE record format.
- a Binary Fragment may be generated and an indication that there are no nested members can be placed in the Type field of the fragment.
- the practical difference between objects containing nested members and objects not containing nested members may be that objects with nested members can be serialized into multiple fragments, while objects without nested members may be serialized into a single binary fragment. Therefore, if there are no nested members, a self terminator bit can be set in a field of a binary fragment.
- the Length field in the Binary Fragment may be set to correspond to the length of the combined primitive members. The fragment can then be emitted.
- the process for no nested members with respect to the Binary Fragment can altered somewhat to allow the nested members to be recursively serialized. In this situation, the self-terminator bit need not be set.
- the nested type members may be recursively processed into fragments of their own.
- a Terminator Fragment may be generated. The Terminator Fragment may then be also be emitted.
- a flowchart of the process for generating collection fragments is provided in FIG. 6 .
- a Collection Start Fragment can be generated. If the collection is not ordered, the bit in the bit-field may be set to “unordered.” Each element of the collection may then be recursively serialized by generating Collection Element Fragments, described below. After all the elements have been serialized, a Terminator Fragment can be generated to indicate the end of the collection.
- a collection element can be serialized into of one or more fragments. If a collection element is represented using more than one fragment the representation may have its own Terminator Fragment.
- the first fragment of a collection element may include a Locator field.
- One purpose of such a filed can be to keep track of a number of elements processed when serializing a collection. By incrementing a counter in a Locator field to properly indicate a current element for serialization, the serialization process can return to the proper location for serializing the next element of a collection.
- both the LOB and FS Fragments may be configured to indicate more than one location type for the corresponding payload. This indication may be made in a Value Type field.
- the Value Type field may describe a payload as comprising pointer, inlined, or delayed location types.
- the appropriate value type can be determined from a member. For example, if a LOB is to be serialized with an object, i.e.
- the value type for a LOB Inline can be selected and the LOB can be serialized accordingly. If instead the LOB is stored in a cell of a database an not with the serialized object, a LOB reference can be serialized with the object, and the appropriate location type can be stored in the Value Type field. The fragment can then be emitted.
- Subtype Fragments may be generated in the same manner as the fragments for any other non-primitive member.
- a Collection Start Fragment along with any Collection Element Fragments and a Terminator Fragment may be generated to mark the end of the serialization of the subtype member.
- the subtype member is a nested LOB member, a LOB Fragment may be generated to contain the subtype member.
- a binary fragment is generated for the subtype, which includes all small primitive members in the subtype, such as int, float etc, in a similar manner as binary fragment is generated for the base type.
- Fragments are stored in a sequence that makes up a serialization for any particular class. If a class of objects contains only primitive members, the objects can be serialized into a self-terminating single fragment. Classes with complex or other non-primitive members, can be serialized into more than one fragment. If a serialization for a class comprises more than one fragment, then one or more Terminator Fragments can also be generated. The set of fragments from the starting fragment to the final Terminator Fragment comprises the sequence of fragments as that term is used in this description.
- Fragments corresponding to subtypes in an instance may be nested under a fragment for the base type.
- an instance of tPartTimeEmployee can be visually depicted in the diagram provided by FIG. 12 . Note that fragments for tEmployee and tPartTimeEmployee in FIG. 12 can be at the same level of nesting.
- FIG. 3 provides several sample schema. Each schema represents a class of objects, each with one or more members of various types. For the purpose of this discussion, FIG. 3 provides a class of “person” objects, each with a name string (a primitive member), an age integer (also a primitive member), and a locations collection (this is a nested complex member). FIG. 3 also provides a class defining “address” objects, each such object having three primitive members: street, city, and zip. FIG.
- the part-time employee class inherits the employee class, and therefore includes all the members of the employee class (including those inherited from the person class), in addition to one primitive, hours per week.
- An instance of tPerson with a non-empty collection of addresses can have the following fragments in the following order:
- An instance of tEmployee with a non-empty collection of addresses can have the following fragments in the following order:
- the task of locating a member involves locating the fragment or fragments in which the member has been placed. If there is more than one such fragment, the first fragment may be located first.
- the position of a fragment relative to the first fragment of a serialization can be determined from metadata. Note that while this technique does not provide for direct addressability of fragments, the task of comparing tokens for each member to identify which aspect of the class an associated member represents is eliminated. Instead, the serialization metadata, when scanned, can quickly direct the processor to an appropriate fragment.
- serialization of an object can be accomplished in one pass.
- a serializer process may proceed in a top-down manner, from base type to subtype and containing type to nested type. At each level of nesting, the serializer process may produce one or more fragments. Such a serializer is never required to update a previously generated fragment, though it may be configured to do so if desired.
- serialization format set forth herein can be configured to support fixed, variable length and bit type primitive members; objects nested within other objects; inheritance; inline and out-of-line LOBs (including file streams); ordered collections (providing via the locator function a serialization of collection elements in the appropriate order); unordered collections; and null values. It may also support composable serializations.
- a nested object may be extracted from an existing serialization without a trace of the fragment container in which the object was serialized, i.e., without any state corresponding to its container. This property of embodiments of the invention allows insertion or updating of entire nested objects.
- the proposed serialization format supports adding members to a class without updating any other existing data.
- the addition of fields to a class is very common in the use of XML schemas, where schemas are frequently updated by adding fields.
- a fragment of any type can be altered to add new members (primitive, collections, or nested objects) as long as they have a default value of NULL and are added to the end of an existing type.
- the addition of large members to an object need not be of concern as with record format serialization, because objects stored in the proposed fragment-based format can be stored as a “bag of bytes,” in a single column of a database, and can be arbitrarily large.
- a path identifies a particular fragment in a serialization. Path can be, for example a set of numbers at each fragment identifying a fragment at each nesting level and subtype level. Since fields of an object reside in a pre-determined fragment, they can be located using the path. In a sense, paths are like addresses for fragments.
- a path can be stored along with a fragment or can be computed by navigating from a first serialized fragment. In order to support embedding and nesting of types a path may account for nesting levels and subtype levels.
- the fragment-based serialization format also allows for supporting access to members without instantiating the object.
- the a member location process or navigator, can navigate to any desired fragment. Once positioned at a fragment, such a process can allow access to the fragment itself or to an entire sequence of fragments rooted at the located fragment.
- locating a fragment can be accomplished even faster.
- Such a directory can store fragments in a table organized as a BTree. In such an implementation, one fragment may be stored per row, allowing the path to the fragment to be used as part of a key for each row.
- a navigator can also efficiently skip fragments that are not of interest for a particular operation.
- Navigation of the fragments serialized according to the invention may comprise tracking either a number of nesting levels that are open, or a subtype number. Once the navigator arrives at a desired nesting level or subtype, the navigator may count the number of fragments at that level or subtype. Such a fragment could itself start a new level of nesting.
- access to primitive members can comprise the following simple operation.
- a navigator can locate a Binary Fragment.
- Primitive members, as described above, are advantageously stored in such a fragment.
- the required member can be extracted using standard, optimized record cracking code. It can also be updated using standard, optimized record building code.
- This simple operation provides for high-performance location of primitive members in a serialized object that can conveniently be stored in a single database column.
- An advantage of allowing for member location in this fashion is that, as mentioned above, members may be updated without the need to instantiate an entire host object. This allows for the replacement of a fragment or sequence of fragments without instantiating an entire object.
- Each fragment can be self-contained, the invention can be configured so the identifier of a fragment sequence is the presence of a begin-nesting fragment and Terminator Fragment. This allows for performance of updates without fixing up lengths anywhere else, and avoids the offset tables of standard record-format serialization.
- Storage of a stream of fragments as generated according to the techniques set forth herein may be conducted by storing a stream of fragments as a LOB.
- a LOB may have a tree-structured storage format whose key is an offset location.
- This technique for storing the stream of fragments provides for a predictable insert and delete time depending on the size of the LOB. It also allow for updating only portions of the LOB.
- the on-wire format for objects serialized in fragments is the same as the on-disk format with respect to the shape of the fragment headers. In the case of fragments other than the LOB and FS fragments, there need be no change at all for fragments that are on-wire versus on-disk.
- This aspect of the invention allows objects to be fast-copied onto a wire, providing significant gains in the speed of transporting objects from location to location. Note, however, that fragment contents may vary for LOB and FS fragments to provide for additional flexibility. This aspect of the invention is explained above with the summary of potential fragment types.
- FIG. 9 shows an assortment of objects in a single column of a database.
- the objects are serialized with a first fragment that contains primitive members in record format (this is the gray fragment with a subdivided payload), and subsequent fragments that could be any of the fragments set forth in this description.
- FIG. 9 refers to FIG. 9 as a reference for the scenario in which embodiments of the invention are stored in a column of a database.
- UDT User-Defined Type
- Basic algorithms for operations that may be performed on a UDT object stored as fragment stream are provided.
- Operation 1 Uniquely identifying an object using its path.
- a UDT is stored as a sequence of fragments, every fragment in the sequence can be uniquely identified by a ‘path’.
- a path is a sequence of steps, where each step can be exactly one of:
- a UDT is stored as a fragment stream.
- the fragment stream can be implemented on top of a byte stream.
- Operation 3 Locating a fragment from path information. Consider a fragment stream representing a UDT. This section explains how to locate a fragment in it given its path.
- This method compares two steps according to the following rules. First, a ‘nesting step’ is less than another ‘nesting step’ if fragment ID in the former is less than that in the later. Second, an ‘inheritance step’ is less than another ‘inheritance step’ if depth in the former is less than that in the latter. Third, a ‘locator step’ is less than another ‘locator step’ if the locator in the former is less than that in the latter. Fourth, a ‘nesting step’ is always less than ‘inheritance step’ (as we store all nested fields before the fields from subtypes).
- a ‘locator step’ is incomparable with ‘nesting step’ or an ‘inheritance step’ and it is an error to attempt such a comparison.
- Pre-condition First i-1 steps of current path already match // those from target // Path and ith step is as not larger than that in targetPath.
- Post-condition First I steps of current path match those from target path. // FragmentStream::AdvanceTillStep(Step# I, Path targetPath) ⁇ While (currentPath[i] ⁇ targetPath[i]) ⁇ // This updates currentPath appropriately // GetNextFragment( ); // see section 0 ⁇ ⁇
- Operation 4 Selecting a primitive field. Selection of a primitive field involves first locating the fragment containing the primitive field using its path. The payload of this fragment is in record format and has the primitive field. Next, one can use standard optimized record manipulation code to extract the primitive field from the payload.
- Operation 5 Updating a primitive field. Updating a primitive field can be accomplished in three steps: First, locate the fragment containing the primitive field using its path in the same way as selecting a primitive field. Next, make a copy of the payload of the fragment and use standard optimized record manipulation code to replace the old value of the primitive field that needs to be updated with the new value. This gives a new payload, which could be longer or shorter than the original payload. Third, update the fragment by replacing old payload with the new payload. Note that this may increase or decrease the length of fragment and its length will have to adjusted accordingly.
- Copying an entire embedded UDT involves first using locateFragment( ), set forth above, to locate the fragment that marks the beginning of the embedded UDT from a path of the embedded UDT. Second, one can use a CopyOutFragmentSequence( ) function explained below to copy out the fragments belonging to the embedded UDT.
- Terminated TRUE; // Note: If this the first fragment being copied and is collection member // fragment, then strip off the locator and copy as a binary fragment // Copy current fragment to destination If (Terminated) Break; // Note that this will change current path, current fragment etc // GetNextFragment( ); // CanContinueNavigation determines whether to copy this fragment. //It may also set terminated to TRUE. Explained below. // If (CanContinueNavigation(pathToThisUDT, Terminated)) ⁇ Continue; ⁇ ⁇ ⁇
- a continue navigation function “FragmentStream::CanContinue Navigation( )” can follow this basic algorithm: // Given the path to nested UDT that we are in the process of copying, // and the current path (available in the state of FragmentStream), // this method determines whether to copy next fragment. Also, it sets // lastFragment to TRUE if the next fragment is the last fragment to copy.
- Operation 7 Removing all fragments of a embedded UDT. Similar to CopyOutFragmentSequence( ), a method DeleteFragmentSequence( ) is provided to delete all the fragments belonging to an embedded UDT. For each fragment, the underlying ByteStream class can be used to remove the bytes for that fragment. Special processing is needed if the fragment is filestream fragment to actually delete the underlying file.
- Operation 8 Updating an embedded UDT with a new UDT.
- An algorithm referred to by the inventors as “FragmentStream::ReplaceFragmentSequence( )” can first locate an embedded UDT using its path, next use DeleteFragmentSequence( ) to delete all the fragments that belong to the embedded UDT, and finally read the fragments for the new UDT and insert them at the current location.
- the underlying ByteStream class can be used to put the bytes for the new fragment. Special processing is needed to insert filestream fragment as the file needs to be created with the proper data, and pointer to that file needs to be put in the fragment.
- Operation 9 Inserting a collection member.
- a UDT object A supplied in the form a FragmentStream object, as a member of a collection in a FragmentStream object representing object B.
- the following algorithm is referred to by the inventors as FragmentStream::InsertCollectionElement( ):first, locate the collection in the B using path to the collection. Next, find a locator and position of insert. As mentioned in earlier, if a collection member was deleted earlier, it results in a ‘gap’ in the locators. If any such gap exists, the unused locator is assigned to the new member and it is inserted such that all members are laid out in increasing order of locators. Otherwise, the new member is assigned a locator that is 1 more than last member and is inserted after the last member. Note that the first fragment of A will have to be modified to put the locator in it.
- Operation 10 Deleting a collection member.
- An algorithm referred to by the inventors as “FragmentStream::DeleteCollectionElement( )” can specify a member to delete is using its locator. Deletion involves first locating the collection using its path, and next locating the member to delete within the collection. Note that this is a simpler problem that locating a fragment given its path, because we are just dealing with locator step here. So similar logic as in LocateFragment method can be used for this. Next, call DeleteFragmentSequence once we are positioned on the first fragment belonging to the member to delete.
- Operation 11 Updating an entire collection or a single collection member. Replacement of an entire collection with a different collection is done in the same way as replacing an embedded UDT with another UDT. Similarly, once a collection member has been located, updating it can be done in the same way as updating an embedded UDT.
- Operation 12 Selecting or updating multiple fields of a UDT. When selecting or updating multiple fields, the following optimizations are performed: First, ordering the selects/updates. Comparison and ordering of two steps has been explained above. Path can be considered as a string of steps and a ‘lexicographical’ ordering can be defined on paths.
- Fields are selected or updated in the increasing order of the paths to the fragments containing them. Note that this ordering is the same order in which the fragments appear in the fragment stream. Multiple primitive fields could be located in the same fragment. In such a case, once the fragment containing them is visited, the standard record manipulation code is used to select or update all desired fields efficiently, without the need to visit the fragment again. By doing the above, even in a complex UDT, at most one pass is required over its fragment stream to select or update all the desired fields of that UDT.
- Operation 13 Enhancing LocateFragment to make use of current position.
- LocateFragment needs to start from the beginning only for locating the fragment containing the first field that needs to be selected or updated. For locating the fragment containing subsequent fields that need to be selected or updated, locateFragment can start from the current position. Enhancements needed to LocateFragment are briefly mentioned below. There are two basic cases here:
- Operation 14 Lazy Materialization.
- sending a serialization of a UDT from server to client sending LOB data and file data from filestream fields tend to be the most time consuming factor. So the fragment stream manager can provide a ‘lazy materialization’ option.
- Schema evolution refers to changing the UDT by adding, removing, or modifying fields (such as changing data type of a field) of a UDT, or changing the inheritance hierarchy by defining new UDTs etc. Such changes may impact instances of UDT that have already been persisted.
- a simple solution is to ensure that any field IDs and fragment IDs assigned to existing fields continue to remain the same when new fields are added. Then, addition of new fields to a UDT can be supported without modifying any pre-existing persisted instances of the UDT.
- the various techniques described herein may be implemented in connection with hardware or software or, where appropriate, with a combination of both.
- the methods and apparatus of the present invention may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
- the computing device In the case of program code execution on programmable computers, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.
- One or more programs that may implement or utilize the user interface techniques of the present invention are preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system.
- the program(s) can be implemented in assembly or machine language, if desired.
- the language may be a compiled or interpreted language, and combined with hardware implementations.
- exemplary embodiments refer to utilizing the present invention in the context of one or more stand-alone computer systems, the invention is not so limited, but rather may be implemented in connection with any computing environment, such as a network or distributed computing environment. Still further, the present invention may be implemented in or across a plurality of processing chips or devices, and storage may similarly be effected across a plurality of devices. Such devices might include personal computers, network servers, handheld devices, supercomputers, or computers integrated into other systems such as automobiles and airplanes. Therefore, the present invention should not be limited to any single embodiment, but rather should be construed in breadth and scope in accordance with the appended claims.
- a symbol in quotes “ ” is a terminal.
- self-terminating bin frag represents a terminal.
- the various terminal symbols are defined in the previous section.
- a symbol in angle brackets ⁇ > represents a non-terminal.
- ⁇ containment frag> is a non-terminal.
- a symbol in curly braces ⁇ ⁇ indicates that symbol may be repeated zero or more times.
- ⁇ containment frag> ⁇ indicates there may be zero or more instances of ⁇ containment frag>.
- Annotation fragments can exist anywhere in the stream.
- ⁇ udt —serialization>:: “self-terminating bin frag”
- ⁇ explicitly terminated bin frag> ⁇ explicitly terminated bin frag>:: ”open-ended bin frag” ⁇ nested frag> “terminator frag” ⁇ nested frag> ⁇ containment frag> ⁇ ⁇ containment frag> ⁇
- ⁇ containment frag> ⁇ containment frag> ⁇ containment frag> ⁇ containment frag> ⁇ ⁇ subtype frag> ⁇ subtype frag>:: ⁇ LOB/FS frag>
- ⁇ collection > ⁇ subtype frag>:: “self-terminating subtype bin frag”
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/821,687 US20050234986A1 (en) | 2004-04-09 | 2004-04-09 | Systems and methods for fragment-based serialization |
KR1020057010619A KR20070053083A (ko) | 2004-04-09 | 2004-07-29 | 단편-기반 직렬화를 위한 시스템 및 방법 |
PCT/US2004/024539 WO2005103937A1 (fr) | 2004-04-09 | 2004-07-29 | Systemes et procedes de serialisation basee sur des fragments |
JP2007507295A JP2007532998A (ja) | 2004-04-09 | 2004-07-29 | フラグメントベースのシリアライゼーションのシステムおよび方法 |
CNA2004800017149A CN1761956A (zh) | 2004-04-09 | 2004-07-29 | 基于片段的串行化系统与方法 |
EP04779553A EP1618487A4 (fr) | 2004-04-09 | 2004-07-29 | Systemes et procedes de serialisation basee sur des fragments |
US11/154,496 US7702637B2 (en) | 2004-04-09 | 2005-06-15 | Systems and methods for fragment-based serialization |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/821,687 US20050234986A1 (en) | 2004-04-09 | 2004-04-09 | Systems and methods for fragment-based serialization |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/154,496 Division US7702637B2 (en) | 2004-04-09 | 2005-06-15 | Systems and methods for fragment-based serialization |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050234986A1 true US20050234986A1 (en) | 2005-10-20 |
Family
ID=35097508
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/821,687 Abandoned US20050234986A1 (en) | 2004-04-09 | 2004-04-09 | Systems and methods for fragment-based serialization |
US11/154,496 Expired - Fee Related US7702637B2 (en) | 2004-04-09 | 2005-06-15 | Systems and methods for fragment-based serialization |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/154,496 Expired - Fee Related US7702637B2 (en) | 2004-04-09 | 2005-06-15 | Systems and methods for fragment-based serialization |
Country Status (6)
Country | Link |
---|---|
US (2) | US20050234986A1 (fr) |
EP (1) | EP1618487A4 (fr) |
JP (1) | JP2007532998A (fr) |
KR (1) | KR20070053083A (fr) |
CN (1) | CN1761956A (fr) |
WO (1) | WO2005103937A1 (fr) |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7702637B2 (en) | 2004-04-09 | 2010-04-20 | Microsoft Corporation | Systems and methods for fragment-based serialization |
US20110196890A1 (en) * | 2010-02-08 | 2011-08-11 | Navteq North America, Llc | Virtual table generator for analyzing geographic databases |
US20120066394A1 (en) * | 2010-09-15 | 2012-03-15 | Oracle International Corporation | System and method for supporting lazy deserialization of session information in a server cluster |
US20150142854A1 (en) * | 2011-11-10 | 2015-05-21 | Microsoft Technology Licensing, Llc | Deep cloning of objects using binary format |
US20150178292A1 (en) * | 2013-05-07 | 2015-06-25 | Tencent Technology (Shenzhen) Company Limited | Methods and systems for data serialization and deserialization |
US20170034275A1 (en) * | 2014-08-07 | 2017-02-02 | Sap Se | High speed communication protocol |
US20190050378A1 (en) * | 2017-08-11 | 2019-02-14 | Microsoft Technology Licensing, Llc | Serializable and serialized interaction representations |
US10671245B2 (en) | 2017-03-29 | 2020-06-02 | Microsoft Technology Licensing, Llc | Collection and control of user activity set data and activity set user interface |
US10693748B2 (en) | 2017-04-12 | 2020-06-23 | Microsoft Technology Licensing, Llc | Activity feed service |
US10732796B2 (en) | 2017-03-29 | 2020-08-04 | Microsoft Technology Licensing, Llc | Control of displayed activity information using navigational mnemonics |
US10853220B2 (en) | 2017-04-12 | 2020-12-01 | Microsoft Technology Licensing, Llc | Determining user engagement with software applications |
US11516160B1 (en) * | 2017-08-30 | 2022-11-29 | Meta Platforms, Inc. | Techniques for efficient messaging client communication by updating user subscription stores based on subscription type and commands |
US20220398235A1 (en) * | 2021-06-11 | 2022-12-15 | Actian Corporation | Method and apparatus for storing object tokens in a database |
US11580088B2 (en) | 2017-08-11 | 2023-02-14 | Microsoft Technology Licensing, Llc | Creation, management, and transfer of interaction representation sets |
Families Citing this family (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7571153B2 (en) * | 2005-03-28 | 2009-08-04 | Microsoft Corporation | Systems and methods for performing streaming checks on data format for UDTs |
US20080040498A1 (en) * | 2006-08-10 | 2008-02-14 | Nokia Corporation | System and method of XML based content fragmentation for rich media streaming |
US7801886B1 (en) * | 2006-10-10 | 2010-09-21 | Intuit Inc. | Method and apparatus for performing database operations involving custom fields |
CN101192148B (zh) * | 2006-12-01 | 2012-02-01 | 深圳迈瑞生物医疗电子股份有限公司 | 兼容新旧应用程序的数据处理方法及其数据存储方法 |
US7761411B2 (en) * | 2007-07-20 | 2010-07-20 | Oracle International Corporation | Delta operations on a large object in a database |
US8626720B2 (en) | 2008-02-11 | 2014-01-07 | International Business Machines Corporation | System and method of reconstructing complex custom objects |
US20100332529A1 (en) * | 2009-06-24 | 2010-12-30 | Microsoft Corporation | Mapping Of Metadata Between A Web Service And A Line-Of-Business System |
US8417714B2 (en) * | 2010-01-22 | 2013-04-09 | Oracle International Corporation | Techniques for fast and scalable XML generation and aggregation over binary XML |
US9898551B2 (en) | 2014-11-25 | 2018-02-20 | Sap Se | Fast row to page lookup of data table using capacity index |
US9891831B2 (en) | 2014-11-25 | 2018-02-13 | Sap Se | Dual data storage using an in-memory array and an on-disk page structure |
US9965504B2 (en) | 2014-11-25 | 2018-05-08 | Sap Se | Transient and persistent representation of a unified table metadata graph |
US9875024B2 (en) | 2014-11-25 | 2018-01-23 | Sap Se | Efficient block-level space allocation for multi-version concurrency control data |
US9792318B2 (en) | 2014-11-25 | 2017-10-17 | Sap Se | Supporting cursor snapshot semantics |
US10042552B2 (en) | 2014-11-25 | 2018-08-07 | Sap Se | N-bit compressed versioned column data array for in-memory columnar stores |
US9824134B2 (en) | 2014-11-25 | 2017-11-21 | Sap Se | Database system with transaction control block index |
US10558495B2 (en) | 2014-11-25 | 2020-02-11 | Sap Se | Variable sized database dictionary block encoding |
US10474648B2 (en) | 2014-11-25 | 2019-11-12 | Sap Se | Migration of unified table metadata graph nodes |
US10127260B2 (en) | 2014-11-25 | 2018-11-13 | Sap Se | In-memory database system providing lockless read and write operations for OLAP and OLTP transactions |
US9798759B2 (en) | 2014-11-25 | 2017-10-24 | Sap Se | Delegation of database post-commit processing |
US10552402B2 (en) | 2014-11-25 | 2020-02-04 | Amarnadh Sai Eluri | Database lockless index for accessing multi-version concurrency control data |
US10255309B2 (en) | 2014-11-25 | 2019-04-09 | Sap Se | Versioned insert only hash table for in-memory columnar stores |
US10296611B2 (en) * | 2014-11-25 | 2019-05-21 | David Wein | Optimized rollover processes to accommodate a change in value identifier bit size and related system reload processes |
US9779104B2 (en) | 2014-11-25 | 2017-10-03 | Sap Se | Efficient database undo / redo logging |
US9513811B2 (en) * | 2014-11-25 | 2016-12-06 | Sap Se | Materializing data from an in-memory array to an on-disk page structure |
US10725987B2 (en) | 2014-11-25 | 2020-07-28 | Sap Se | Forced ordering of a dictionary storing row identifier values |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5568639A (en) * | 1993-11-24 | 1996-10-22 | Menai Corporation | Method and apparatus for providing an object-oriented file structuring system on a computer |
US5634123A (en) * | 1993-07-08 | 1997-05-27 | Park City Group, Inc. | Data management using nested records and code points |
US5727203A (en) * | 1995-03-31 | 1998-03-10 | Sun Microsystems, Inc. | Methods and apparatus for managing a database in a distributed object operating environment using persistent and transient cache |
US5768578A (en) * | 1994-02-28 | 1998-06-16 | Lucent Technologies Inc. | User interface for information retrieval system |
US6012067A (en) * | 1998-03-02 | 2000-01-04 | Sarkar; Shyam Sundar | Method and apparatus for storing and manipulating objects in a plurality of relational data managers on the web |
US6266673B1 (en) * | 1997-10-31 | 2001-07-24 | Oracle Corporation | Performing operations on objects in a database system in a response to a request that specifies references that indicate where the objects reside |
US20020143521A1 (en) * | 2000-12-15 | 2002-10-03 | Call Charles G. | Methods and apparatus for storing and manipulating variable length and fixed length data elements as a sequence of fixed length integers |
US20020164149A1 (en) * | 2000-09-06 | 2002-11-07 | Wilkinson James Hedley | Combining video material and data |
US6631130B1 (en) * | 2000-11-21 | 2003-10-07 | Transwitch Corporation | Method and apparatus for switching ATM, TDM, and packet data through a single communications switch while maintaining TDM timing |
US20040078467A1 (en) * | 2000-11-02 | 2004-04-22 | George Grosner | Switching system |
US20040220946A1 (en) * | 2003-05-01 | 2004-11-04 | Oracle International Corporation | Techniques for transferring a serialized image of XML data |
US6904454B2 (en) * | 2001-03-21 | 2005-06-07 | Nokia Corporation | Method and apparatus for content repository with versioning and data modeling |
US7523111B2 (en) * | 2002-12-02 | 2009-04-21 | Silverbrook Research Pty Ltd | Labelling of secret information |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2836568A1 (fr) * | 2002-02-28 | 2003-08-29 | Bull Sa | Procede iteratif de serialisation d'objets logiciels structures |
US6941316B2 (en) | 2003-10-23 | 2005-09-06 | Microsoft Corporation | System and method for object persistence in a database store |
US20050234986A1 (en) | 2004-04-09 | 2005-10-20 | Microsoft Corporation | Systems and methods for fragment-based serialization |
-
2004
- 2004-04-09 US US10/821,687 patent/US20050234986A1/en not_active Abandoned
- 2004-07-29 CN CNA2004800017149A patent/CN1761956A/zh active Pending
- 2004-07-29 EP EP04779553A patent/EP1618487A4/fr not_active Withdrawn
- 2004-07-29 WO PCT/US2004/024539 patent/WO2005103937A1/fr not_active Application Discontinuation
- 2004-07-29 JP JP2007507295A patent/JP2007532998A/ja active Pending
- 2004-07-29 KR KR1020057010619A patent/KR20070053083A/ko not_active Application Discontinuation
-
2005
- 2005-06-15 US US11/154,496 patent/US7702637B2/en not_active Expired - Fee Related
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5634123A (en) * | 1993-07-08 | 1997-05-27 | Park City Group, Inc. | Data management using nested records and code points |
US5568639A (en) * | 1993-11-24 | 1996-10-22 | Menai Corporation | Method and apparatus for providing an object-oriented file structuring system on a computer |
US5768578A (en) * | 1994-02-28 | 1998-06-16 | Lucent Technologies Inc. | User interface for information retrieval system |
US5727203A (en) * | 1995-03-31 | 1998-03-10 | Sun Microsystems, Inc. | Methods and apparatus for managing a database in a distributed object operating environment using persistent and transient cache |
US6266673B1 (en) * | 1997-10-31 | 2001-07-24 | Oracle Corporation | Performing operations on objects in a database system in a response to a request that specifies references that indicate where the objects reside |
US6012067A (en) * | 1998-03-02 | 2000-01-04 | Sarkar; Shyam Sundar | Method and apparatus for storing and manipulating objects in a plurality of relational data managers on the web |
US20020164149A1 (en) * | 2000-09-06 | 2002-11-07 | Wilkinson James Hedley | Combining video material and data |
US20040078467A1 (en) * | 2000-11-02 | 2004-04-22 | George Grosner | Switching system |
US6631130B1 (en) * | 2000-11-21 | 2003-10-07 | Transwitch Corporation | Method and apparatus for switching ATM, TDM, and packet data through a single communications switch while maintaining TDM timing |
US20020143521A1 (en) * | 2000-12-15 | 2002-10-03 | Call Charles G. | Methods and apparatus for storing and manipulating variable length and fixed length data elements as a sequence of fixed length integers |
US6904454B2 (en) * | 2001-03-21 | 2005-06-07 | Nokia Corporation | Method and apparatus for content repository with versioning and data modeling |
US7523111B2 (en) * | 2002-12-02 | 2009-04-21 | Silverbrook Research Pty Ltd | Labelling of secret information |
US20040220946A1 (en) * | 2003-05-01 | 2004-11-04 | Oracle International Corporation | Techniques for transferring a serialized image of XML data |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7702637B2 (en) | 2004-04-09 | 2010-04-20 | Microsoft Corporation | Systems and methods for fragment-based serialization |
US10289636B2 (en) * | 2010-02-08 | 2019-05-14 | Here Global B.V. | Virtual table generator for analyzing geographic databases |
US20110196890A1 (en) * | 2010-02-08 | 2011-08-11 | Navteq North America, Llc | Virtual table generator for analyzing geographic databases |
US20120066394A1 (en) * | 2010-09-15 | 2012-03-15 | Oracle International Corporation | System and method for supporting lazy deserialization of session information in a server cluster |
US9495392B2 (en) | 2010-09-15 | 2016-11-15 | Oracle International Corporation | System and method for parallel multiplexing between servers in a cluster |
US9811541B2 (en) * | 2010-09-15 | 2017-11-07 | Oracle International Corporation | System and method for supporting lazy deserialization of session information in a server cluster |
US9864759B2 (en) | 2010-09-15 | 2018-01-09 | Oracle International Corporation | System and method for providing scatter/gather data processing in a middleware environment |
US20150142854A1 (en) * | 2011-11-10 | 2015-05-21 | Microsoft Technology Licensing, Llc | Deep cloning of objects using binary format |
US9817857B2 (en) * | 2011-11-10 | 2017-11-14 | Microsoft Technology Licensing, Llc | Deep cloning of objects using binary format |
US20150178292A1 (en) * | 2013-05-07 | 2015-06-25 | Tencent Technology (Shenzhen) Company Limited | Methods and systems for data serialization and deserialization |
US20170034275A1 (en) * | 2014-08-07 | 2017-02-02 | Sap Se | High speed communication protocol |
US10218788B2 (en) * | 2014-08-07 | 2019-02-26 | Sap Se | High speed communication protocol |
US10732796B2 (en) | 2017-03-29 | 2020-08-04 | Microsoft Technology Licensing, Llc | Control of displayed activity information using navigational mnemonics |
US10671245B2 (en) | 2017-03-29 | 2020-06-02 | Microsoft Technology Licensing, Llc | Collection and control of user activity set data and activity set user interface |
US10693748B2 (en) | 2017-04-12 | 2020-06-23 | Microsoft Technology Licensing, Llc | Activity feed service |
US10853220B2 (en) | 2017-04-12 | 2020-12-01 | Microsoft Technology Licensing, Llc | Determining user engagement with software applications |
US20190050378A1 (en) * | 2017-08-11 | 2019-02-14 | Microsoft Technology Licensing, Llc | Serializable and serialized interaction representations |
US11580088B2 (en) | 2017-08-11 | 2023-02-14 | Microsoft Technology Licensing, Llc | Creation, management, and transfer of interaction representation sets |
US11516160B1 (en) * | 2017-08-30 | 2022-11-29 | Meta Platforms, Inc. | Techniques for efficient messaging client communication by updating user subscription stores based on subscription type and commands |
US20220398235A1 (en) * | 2021-06-11 | 2022-12-15 | Actian Corporation | Method and apparatus for storing object tokens in a database |
Also Published As
Publication number | Publication date |
---|---|
KR20070053083A (ko) | 2007-05-23 |
WO2005103937A1 (fr) | 2005-11-03 |
JP2007532998A (ja) | 2007-11-15 |
EP1618487A1 (fr) | 2006-01-25 |
EP1618487A4 (fr) | 2010-02-17 |
US7702637B2 (en) | 2010-04-20 |
US20050234868A1 (en) | 2005-10-20 |
CN1761956A (zh) | 2006-04-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7702637B2 (en) | Systems and methods for fragment-based serialization | |
EP0662228B1 (fr) | Appareil de stockage et restitution de donnees | |
US6484181B2 (en) | Method and system for handling foreign key update in an object-oriented database environment | |
US6915304B2 (en) | System and method for converting an XML data structure into a relational database | |
EP1066570B1 (fr) | Recherche et indexation rapides de chaines de caracteres | |
US7710988B1 (en) | Method and system for non-deterministic finite automaton filtering | |
US6594665B1 (en) | Storing hashed values of data in media to allow faster searches and comparison of data | |
US6535869B1 (en) | Increasing efficiency of indexing random-access files composed of fixed-length data blocks by embedding a file index therein | |
US8335889B2 (en) | Content addressable storage systems and methods employing searchable blocks | |
JP5506290B2 (ja) | 検索可能なブロックを用いた連想記憶システムおよびその方法 | |
JP5407043B2 (ja) | バイナリにエンコードされたxmlデータの効率的な区分的アップデート | |
KR101129083B1 (ko) | 표현 그룹화 및 평가 | |
KR101110988B1 (ko) | 구조적 데이터 변환을 위한 장치 | |
US20040111668A1 (en) | Annotation validity using partial checksums | |
US20080040381A1 (en) | Evaluating Queries Against In-Memory Objects Without Serialization | |
US20070016605A1 (en) | Mechanism for computing structural summaries of XML document collections in a database system | |
KR20100061308A (ko) | 정보 처리 장치, 데이터베이스 시스템, 정보 처리 방법 및 프로그램 | |
US8073843B2 (en) | Mechanism for deferred rewrite of multiple XPath evaluations over binary XML | |
EP0520232A2 (fr) | Procédé et dispositif pour stocker et retrouver des résultats fonctionels | |
US7051016B2 (en) | Method for the administration of a data base | |
US20060015483A1 (en) | SQL query enhancement technique | |
US20060106857A1 (en) | Method and system for assured document retention | |
WO2002027486A1 (fr) | Procedes et appareil pour produire des identifiants uniques pour composants logiciels | |
US6092092A (en) | Gap-based style-run array mechanism | |
Haw et al. | Improving the support for XML dynamic updates using a hybridization labeling scheme (ORD-GAP) |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MICROSOFT CORPORATION, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TEREK, F. SONER;KALHAN, AJAY;PONNEKANTI, NAGAVAMSI;AND OTHERS;REEL/FRAME:015707/0053;SIGNING DATES FROM 20040816 TO 20040818 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034766/0001 Effective date: 20141014 |