ZA200600645B - Systems and methods for data modeling in an item-based storage platform - Google Patents

Description

QT

SYSTEMS AND METHODS FOR DATA MOD _ELING IN Ce

AN ITEME-BASED STORAGE PLATFO™M :

CROSS-REFERENCE

[0001] This application is related by subject matter to the inventions disclosed in the following commonly assigned ap»plications: U.S. Patent Applicatiosn No. (not yet assigned) (Atty. Docket No. MSFT-1748), filed on even date herewith, entitl_ed “SYSTEMS AND

METHODS FOR REPRESENTMNG UNITS OF INFORMATION MANAGEABLE BY A

HARDWARE/SOFTWARE INXERFACE SYSTEM BUT INDEE2ENDENT OF PHYSICAL

REPRESENTATION: U.S. Patent Application No. (not yet assig-med) (Atty. Docket No.

MSFT-1749), filed on even date herewith, entitled “SYSTEMS AT™ND METHODS FOR

SEPARATING UNITS OF INFORMATION MANAGEABLE B™Y A

HARDWARE/SOFTWARE INTERFACE SYSTEM FROM THEZIR PHYSICAL

ORGANIZATION"; U.S. Patent Application No. (not yet assigned) (Atty. Docket No.

MSFT-1750), filed on even date: herewith, entitled “SYSTEMS AND METHODS FOR THE

IMPLEMENTATION OF A BASE SCHEMA FOR ORGANIZINIG UNITS OF

INFORMATION MANAGEABLE BY A HARDWARE/SOFTW= ARE INTERFACE

SYSTEM”; U.S. Patent Application No. (not yet assigned) (Atty. Docket No. MSFT-1751), filed on even date herewith, ent tled “SYSTEMS AND METHOD» S FOR THE

IMPLEMENTATION OF A CORE SCHEMA FOR PROVIDING A TOP-LEVEL

STRUCTURE FOR ORGANIZING UNITS OF INFORMATIONT MANAGEABLE BY A

HARDWARE/SOFTWARE IN"TERFACE SYSTEM”; U.S. Paterat Application No. (not yet assigned) (Atty. Docket No. MSSFT-1752), filed on even date here=with, entitled “SYSTEMS

AND METHOD FOR REPRESSENTING RELATIONSHIPS BEX WEEN UNITS OF

. INFORMATION MANAGEABLE BY A HARDWARE/SOFTW./ ARE INTERFACE . SYSTEM”; U.S. Patent Application No. (not yet assigned) (Atty. Docket No. MSFT-2733), "filed on even date herewith, entitled “SYSTEMS AND METHOIDS FOR INTERFACING

APPLICATION PROGRAMS “WITH AN ITEM-BASED STORAGE PLATFORM™; and - U.S. Patent Application No. (not yet assigned) (Atty. Docket No. MSFT-27 34), filed on even . date herewith, entitled “STORAGE PLATFORM FOR ORGANIZING, SEARCHING, AND

SHARING DATA”.

FIELD OF THE IN"VENTION

[0002] The present invention relates generzally to the field of informatio=n storage and retrieval, amd, more particularly, to an active storage platform for organizing , searching, and sharing different types of datain a computerized system.

BACKGROUND OF THEE INVENTION

[0003] Individual disk capacity has been g=rowing at roughly seventy percent (70%) - per year over thae last decade. Moore’s law accurately predicted the tremendous gains in central processi.ng unit (CPU) power that has occurmred over the years. Wired ancl wireless technologies hamve provided tremendous connectivity and bandwidth. Presuming current trends continue-, within several years the average lagptop computer will possess roughly one terabyte (TB) of storage and contain millions of files, and 500 gigabyte (GB) drives will become commonplace.

[0004] Consumers use their computers primarily for communication amnd organizing personal information, whether it is traditional perscenal information manager (PIM) style data or media such =as digital music or photographs. The= amount of digital content, ard the ability to store the raw bytes, has increased tremendously; however the methods availaWble to consumers for =organizing and unifying this data ha=s not kept pace. Knowledge workers _ spend enormoims amounts of time managing and sh=aring information, and some studies "estimate that kmnowledge workers spend 15-25% of their time on non-productive= information related activitiezs. Other studies estimate that a typical knowledge worker spendss about 2.5 : hours per day searching for information.

[0005] Developers and information technology (IT) departments invesst significant ‘amounts of tire and money in building their own lata stores for common storagge : abstractions to represent such things as people, places, times, and events. Not conly does this result in dupliccated work, but it also creates islandss of common data with no me=chanisms for common searching or sharing of that data. Just con_sider how many address books can exist today on a cormrputer running the Microsoft Windo~ws operating system. Many =applications, such as e-mail clients and personal finance prograrms, keep individual address beooks, and there is little sknaring among applications of the adc3ress book data that each such program individually m_aintains. Consequently, a finance program (like Microsoft Mone=y) does not share addresses for payees with the addresses mairmtained in an email contact fo-lder (like the one in Microsoft Outlook). Indeed, many users ha—ve multiple devices and logically should . a

. synchronize their personal data amongst themselves and across a wide variety of additiormal sources, including cell phones to commercial services su_ch as MSN and AOL; nevertheless, collaboration of shared documents is largely achieved by attaching documents to e-mail messages—that is, manually and inefficiently. : [0006] One reason for this lack of collaboration is that traditional approaches to the organization of information in computer systems have ce=ntered on the use of file-folder-zand- directory-based systems (“file systems”) to organize plu ralities of files into directory hierarchies of folders based on an abstraction of the phy~sical organization of the storage medium used to store the files. The Multics operating system, developed during the 196=0s, can be credited with pioneering the use of the files, fold ers, and directories to manage storable units of data at the operating system level. Spe=cifically, Multics used symbolic addresses within a hierarchy of files (thereby introducimmg the idea of a file path) where physical addresses of the files were not transparent to tine user (applications and end-use=rs).

This file sys tem was entirely unconcerned with the file ~format of any individual file, anc the relationships amongst and between files was deemed irmrelevant at the operating system Bevel (that is, othesr than the location of the file within the hierarchy). Since the advent of Mu ltics, storable data has been organized into files, folders, and directories at the operating syste=m level. These files generally include the file hierarchy itself (the “directory”) embodied mn a special file xnaintained by the file system. This directomry, in turn, maintains a list of ent ries carresponding to all of the other files in the directory amd the nodal location of such file=s in the hierarchy (herein referred to as the folders). Such mas been the state of the art for approximately forty years.

[00007] However, while providing a reasonabl-e representation of information residing in the computer’s physical storage system, a fi_le system is nevertheless an abstraction of that physical storage system, and therefoere utilization of the files requires a _ level of indirection (interpretation) between what the u ser manipulates (units having context, features, an d relationships to other units) and what the operating system provides (files folders, and directories). Consequently, users (applications and/or end-users) have no choice but to forces units of information into a file system structure even when doing so is inefficient, inconsisten't, or otherwise undesirable. Moreover, exissting file systems know little abomut the structure of data stored in individual files and, because- of this, most of the information remains locked up in files that may only be accessed (znd comprehensible) to the appliscations that wrote them. Consequently, this lack of schematic description of information, and mechanisms for managing inform ation, leads to the creation of silos of data with little data sharing among the individual silos. For example, many personal computer (PC) users have more than five distinct stores that contain information about the peoples they interact with on some level—for example, Outlook Contacts, online account addresseess, Windows Address

Book, Quicken Payees, and instaxat messaging (IM) buddy lists—becatase organizing files presents a significant challenge to these PC users. Because most existimg file systems utilize a nested folder metaphor for organizing files and folders, as the number of files increases the effort necessary to maintain an organization scheme that is flexible an«d efficient becomes quite daunting. In such situations, it would be very useful to have multiple classifications ofa single file; however, using hard or soft links in existing file systems is cumbersome and difficult to maintain.

[0008] Several unsucce ssful attempts to address the shortcormings of file systems have been made in the past. Sorrie of these previous attempts have inwolved the use of content addressable memory to provide a mechanism whereby data could be accessed by content rather than by physical address. However, these efforts have gproven unsuccessful because, while content addressable memory has proven useful for small-scale use by devices such as caches and memory man agement units, large-scale use for dewices such as physical storage media has not yet been possible for a variety of reasons, and thus such a solution simply does not exist. Other attempts using object-oriented database OODB) systems have been made, but these attempts, while featuring strong database characteristics and good non- file representations, were not effective in handling file representations and could not replicate the speed, efficiency, and simplicity of the file and folder based hieramrchical structure at the hardware/software interface system level. Other efforts, such as those that attempted to use

SmallTalk (and other derivatives), proved to be quite effective at handling file and non-file representations but lacked database features necessary to efficiently organize and utilize the relationships that exist between he various data files, and thus the overall efficiency of such systems was unacceptable. Yet other attempts to use BeOS (and other such operating systems research) proved to be i madequate at handling non-file repres entations—the same core shortcoming of traditional File systems—despite being able to adequately represent files while providing some necessary database features.

[00097] Database technology is another area Of the art in which similar challenges . exits. For exaample, while the relational database modes] has been a great commercial success, in truth indepwendent software vendors (ISV) generally” exercise a small portion of the functionality available in relational database software products (such as Microsoft SQL.

Server). Instead, most of an application’s interaction with such a product is in the formm of simple “gets” and “puts”. While there are a number of readily apparent reasons for thi s— such as being platform or database agnostic—one key reason that often goes unnoticed is that the database does not necessarily provide the exact abstractions that a major business . application wendor really needs. For example, while the real world has the notion of * jtems”, such as “cus-tomers” or “orders” (along with an orders embedded “line iterns” as items in and of themselvess), relational databases only talk in terms of tables and rows.” Consequently, while the application may desire to have aspects of consistency, locking, security, and Jor "triggers at tie item level (to name a few), generally dlatabases provide these features o-nly at the table/rowe level. While this may work fine if each item gets mapped to a single rove in some table i the database, in the case of an order with multiple line items there may Boe reasons why an item actually gets mapped to multipl e tables and, when that is the cases, the simple relatJional database system does not quite provide the right abstractions. * Consequently, an application must build logic on top of the database to provide these basic abstractionss. In other words, the basic relational mo del does not provide a sufficient platform for storage cof data on which higher-level applications can easily be developed becaus-e the basic relational model requires a level of indirection between the application and the =storage system--wh_ere the semantic structure of the data might only be visible in the applicat=ion in "certain instances. While some database vendors are building higher-level functionali~ty into their produccts—such as providing object relational capabilities, new organizational meodels, and the like=—none have yet to provide the kind of comprehensive solution needed, whhere a "truly comprehensive solution is one which provides both useful data model abstractions (such as “Items,” “Extensions,” “Relationships,” and so om) for useful domain abstractions (such as “persons,” “Locations,” “Events,” etc.).

[00710] In view of the foregoing deficiencies in existing data storage and damabase technologie=s, there is a need for a new storage platform that provides an improved ability to organize, sezarch, and share all types of data in a connputer system--a storage platform that extends anc broadens the data platform beyond exis ting file systems and database sy stems,

and that is designed to be the store for all types of data. The= present invention satisfies this : need.

SUMMARY OF THE INVENTION

[0011] The folloswing summary provides an overview of various aspects of the invention. It is not intendlled to provide an exhaustive descri ption of all of the important aspects of the invention, ror to define the scope of the inveration. Rather, this summary is intended to serve as an in—troduction to the detailed descripti on and figures that follow.

[0012] The pres ent invention is directed to a storage platform for organizing, : searching, and sharing da ta. The storage platform of the prezsent invention extends and broadens the concept of lata storage beyond existing file systems and database systems, a_nd is designed to be the store= for all types of data including strmuctured, non-structured, or Sermi- structured data.

[0013] Accordimng to one aspect of the present inveention, the storage platform of the . present invention compri ses a data store implemented on a database engine. In various embodiments of the pressent invention, the database engine comprises 2 relational databasee engine with object relational extensions. The data store implements a data model that . supports organization, searching, sharing, synchronization, and security of data. Specific types of data are described in schemas, and the platform preovides a mechanism to extend the set of schemas to define mew types of data (essentially subtypes of the basic types provides by the schemas). A synchrOnization capability facilitates the sharing of data among users or systems. File-system-lil=e capabilities are provided that allow interoperability of the datam store with existing file ssystems but without the limitation osf such traditional file systems. A change tracking mechanilism provides the ability track chan_ges to the data store. The stor=age platform further compris=es a set of application program interfaces that enable application sto access all of the foregoirg capabilities of the storage platform and to access the data described in the schemas.

[0014] Accordiing to another aspect of the inventi_on, the data model implemented by the data store defines un=its of data storage in terms of item s, elements, and relationships. An item is a unit of data stomrable in a data store and can comprise one or more elements and relationships. An element is an instance of a type comprising one or more fields (also referred to herein as a property). A relationship is a link bestween two items. (As used herein, these and other specific terms may be capitalized in order ®o offset them from other termss used in close proximity; however, there is no intention whatsoever to dist-inguish between a capitalized term, e.g. “Item”, and the same term when not capitalized, e.gz., “item”, and no “such distinction should be presumed or implied.)

[0015] According to another aspect of the invention, a computes system comprises a plurality of Items where each Item constitutes a discrete storable unit of information that can " be manipulated by a hardware/software interface system; a plurality of It=em Folders that constitute an organizational structure for said Items; and a hardware/soft=ware interface system for manipulating a plurality of Items and wherein each Item belomngs to at least one - Item Folder and may belong to more than one Item Folder. : [0016] According to another aspect of the invention, a compute=r system comprises a "plurality of Items, where each Item constitutes a discrete unit of information that can be manipulated by a hardware/software interface system, and the Item or somme of the Item’s property values are computed dynamically as opposed to being derived rom a persistent store. In other words, the hardware/software interface system does not require that the Item be stored, and certain operations are supported such as the ability to enumerate the current set of Items or the ability to retrieve an Item given its identifier (which is m-ore fully described in the sections that describe the application programming interface, or API) of the storage platform -- for example, an Item might be the current location of a cell phone or the temperature reading on a temperature sensor.

[0017] According to another aspect of the invention, a hardwamre/software interface system for a computer system, wherein said hardware/software interface system manipulates a plurality of Items, further comprises Items interconnected by a plurality of Relationships managed by the hardware/software interface system. According to anot-her aspect of the invention, a hardware/software interface system for a computer system “wherein said hardware/software interface system manipulates a plurality of discrete umnits of information having properties understandable by said hardware/software interface system. According to another aspect of the invention, a hardware/software interface system for a computer system comprises a core schema to define a set of core Items which said hardw are/software interface system understands and can directly process in a predetermined and predictable way.

According to another aspect of the invention, a method for manipulating a plurality of discrete units of information (“Items”) in a hardware/software interface system for a computer system, said method comprising interconnecting said Items with a plurality of

L Relationships and managing said Relzationships at the hardware/software iraterface system le=vel, is disclosed.

[0018] According to another feature of the invention, the API of &he storage pl atform provides data classes for each item, item extension, and relationskip defined in the se=t of storage platform schemas. In aaddition, the application programming, interface provides a set of framework classes that defines a common set of behaviors for the d ata classes and that, tomgether with the data classes, providlle the basic programming model for the storage platform

A_PL According to another feature o=f the invention, the storage platform API provides a © simplified query model that enables application programmers to form quexies based on various properties of the items in the= data store, in a manner that insulates the application pxogrammer from the details of the cyuery language of the underlying datatbase engine. + Aaccording to yet another aspect of tine storage platform API of the present invention, the API ceollects changes to an item made by an application program and then organizes them into the c-orrect updates required by the database engine (or any kind of storage erm gine) on which the d_ata store is implemented. This ena bles application programmers to make= changes to an item im memory, while leaving the complexity of data store updates to the API.

[0019] Through its commen storage foundation and schematized data, the storage pelatform of the present invention enables more efficient application devel opment for consumers, knowledge workers and enterprises. It offers a rich and extermsible application perogramming interface that not only» makes available the capabilities inhe=rent in its data mnodel, but also embraces and extenads existing file system and database a<ccess methods. ‘ [0020] Other features and advantages of the invention may becc>me apparent from tthe following detailed description of the invention and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The foregoing sum-nary, as well as the following detaile d description of the : invention, is better understood wher read in conjunction with the appended drawings. For the purpose of illustrating the invention, there is shown in the drawings e xemplary . embodiments of various aspects of she invention; however, the invention is not limited to the specific methods and instrumentalit—es disclosed. In the drawings:

[0022] Fig. 1 is a block di=agram representing a computer system in which aspects of the present invention may be incorporated;

[04023] Fig. 2 is a block diagram illustrating a computer system divided nto three componemt groups: the hardware component, the hhardware/software interface sysstem coimponemt, and the application programs comporment;

[08024] Fig. 2A illustrates the traditional tree-based hierarchical structuree for files grouped &n folders in a directory in a file-based operating system;

[08025] Fig. 3 is a block diagram illustrating a storage platform in accoredance with the presemt invention;

[0026] Fig. 4 illustrates the structural reJationship between Items, Item Folders, and

Categories in various embodiments of the present invention;

[0027] Fig. 5A is a block diagram illustarating the structure of an Item; [€028] Fig. SB is a block diagram illustrating the complex property typees of the

Item of Fig. SA;

[029] Fig. 5C is a block diagram illust-wating the “Location” Item wherein its complex types are further described (explicitly listed); [#0030] Fig. 6A illustrates an Item as a subtype of the Item found in the Base

Schema; [#0031] Fig. 6B is a block diagram illustrating the subtype Item of Fig. «6A wherein its inherited types are explicitly listed (in additiom to its immediate properties);

[0032] Fig. 7 is a block diagram illustrating the Base Schema including its two top- -level cla_ss types, Item and PropertyBase, and thes additional Base Schema types derived therefrom;

[0033] Fig. 8A is a block diagram illus&rating Items in the Core Schenma; [C0034] Fig. 8B is a block diagram illus&rating the property types in the Core

Schemas

[0035] Fig. 9 is a block diagram illustrating an Item Folder, its membe=r Items, and the inter-connecting Relationships between the Item Folder and its member Items;

[0036] Fig. 10 is a block diagram illustrating a Category (which, agair, is an Item itself), ists member Items, and the interconnecting Relationships between the Category and its member Items;

[0037] Fig. 11 is a diagram illustrating a reference type hierarchy of tie data model of the storage platform, in accordance with the prresent invention;

. BVO 2005/029313 PCT/WIS2003/026144

[0038] Fig. 12 is a diagram illustxating how relationships are classified, in . accosrdance with an embodiment of the present invention;

[0039] Fig. 13 is a diagram illustrating a notification mechanism, in a«ccordance witha an embodiment of the present invention;

[0040] Fig. 14 is a diagram illustrating an example in which two transsactions are both inserting a new record into the same B-Tree;

[0041] Fig. 15 illustrates a data change detection process in accordanece with an embodiment of the present invention;

[0042] Fig. 16 illustrates an exexmplary directory tree;

[0043] Fig. 17 shows an exampl € in which an existing folder of a diresctory-based file system is moved into the storage platform data store in accordance with arm aspect of the pressent invention;

[0044] Fig. 18 illustrates the coracept of Containment Folders, in accordance with an aspect of the present invention;

[0045] Fig. 19 illustrates the bassic architecture of the storage platforrm APL

[0046] Fig. 20 schematically represents the various components of tine storage platform API stack;

[0047] Figs. 21A and 21B are a pictorial representation of an exemplary Contacts sch ema (Items and Elements);

[0048] Fig. 22 illustrates the ruratime framework of the storage platfom APL in accordance with an aspect of the present invention;

[0049] Fig. 23 illustrates the execution of a FindAll operation, in accordance with an embodiment of the present invention;

[0050] Fig. 24 illustrates the process by which storage platform API classes are gererated from the storage platform Schema, in accordance with an aspect of the present invention;

[0051] Fig. 25 illustrates a sche-ma on which a File API is based, in accordance with another aspect of the present invention;

[0052] Fig. 26 is a diagram illustrating an access mask format used #or data security pumposes, in accordance with an embodirment of the present invention;

[053] Figs. 27(a), (b), and (c) depict a new identically protected security region being carv ed out of an existing security region, im accordance with an embodirment of one -aspect of te present invention;

[0054] Fig. 28 is a diagram illustrating the concept of an Item search view, in accordance with an embodiment of one aspect off the present invention; and

[0055] Fig. 29 is a diagram illustrating an exemplary Item hierarchy —in accordance with an embodiment of the present invention. [Remainder of Page Intentionally Left Blank]

DETAILXD DESCRIPTION OF THE INVENTION

- TABLE OF CONTENTS

CL INTRODUCTION. cooeoeesesssssesssssssssssssississssossssn. sissssssssssnsssnssnnss = 212

E A. EXEMPLARY COMPUTING ENVIRONMENT .....ccooovivinivmnmmmmnnininees = 21 -

B. TRADITION AL FILE-BASED STORAGE ......cceocnieisnm iemmnansrnneniessisisvsnsaese = 25 -

Io. A NEW STORAGE PLATFORM FOR ORGANIZING, SEAR CHING, AND ]

B. STORAGE PLATFORM OVERVIEW ......oviiiiiriireniree coreinininnninnseniessesiiosass = 27 - 2. TEIN TACTIEFICAON covvvreeeeeeverenscrsessmssansassasssnsasm ssorsermsessssisssssssssssssnses = SOB = a) TEI REFELENCES. coeur eremrserertnsrsanssrsessesssesm sssrscmsmressssssennnmssnssssnsess™ Sb = (1) IemIDREfEreNCe .....coceeeererirenersm arervesssssnnsnssnsisnissssanens™ 34 - (2) TteMPAthREFEIENCE .crvererreearierrrs m vivessrrmeresareneersenssesenssres™ Jd = b) Reference type MErArChY. ..comecrsueisisses msissersssesssssssssscssssersssns™ 300 = 3. Item Folders and CateOTIES. .....ureerereesrecremssercs msisnesresssesssssansssassisassas = Jed = 4. SCRERIIES ....e.eoeeveeressreesresesse ses ssassasssssnasssosssssnsos mussserssssssassressssssssssansass = 3 J = a) BASE SCHEMA co.veeveeiriven orerriinresresnnresssrs mresssasssesssssesssssossasesnens= 3 0° b) COTE SCHEIMA veecneivinriicrerecanrrrerneanssarares moresseseeresassosserssnsissesrans™ Sud = ’ 5. RELATIONSHIPS o...ovvcverrrernersennsesecnsessrssssssnserseseressnss messeansssssssssssssssessscsenes = HE = . a) Relationship Declaration ........cvueesivrreas sussssrsncrsssssasssrassssnrense= $00 - b) Holding Relationship........cceeeuseenereens coremmenresesessneneresssiansocse= 422 = c) Embedding Relationships........ceevevernreeee cnvvenmmrcsnsessesssenscseisanenes= 423 = d) Reference Relationships.....ceceenieerrvens rvimrnvesiiriniserensnsssanean= 4d €) Rules and CONSITAINES. ..veeueereereeecrerissersiann mrersssssasersisssoserossasssasssss= Gah =

2 Ordering of Relationships .....—cvcereriimimnsnsniisnmnsisssstansinne os 45 - 6. EEXIEOSTDIILY «evveecuerasrecasenrieretemnismnnries merrssnisnsransistusresssissssisnssiasasanensas one = 50 - a.) TESTI) EXLETISIONS. cvvsreerssesseresneas merosmoressssessessssmsssssassssssssnssammssenn sens™ IL = >) Extending NestedEIemMent tY[Des. ...cecvuemserisssmmminrasresssessnsssmm anes” 55 - 1. IData Store Implementation Using ULDTS c.cciiinnccsininninnse =e = 58 - 2. J ERTS BE e) 1011 STEER EL 59 - 3. EEXENSION MAPPING cecvrererinernreremrrees messosiosmimrmiinsrassststsnass sus ssnsssnes oeee = 62 - 4. Nested Element Mapping ...eceeeveseeiee maressnssssstssisnseisessninmsmsnssesessisne one 63 - 5. CODBJECE TABNEILY ..oevcevvrenereriersnrenssesssies menrssesstssesisssssassanssmsssesssnesasssssn ns sine = 63 - 6. SQL Object NANG .u.vevueierrrrmrreresess mcoeomesssssssisnsisssismstansenssssasessssas om sors = 63 - 7. SCOIUMN NAMUNE ee vevenenisecressncansrrssarns sessssssasstssssissssrsassirsnmimsresnassessn ese = 64 - 8. Search VIEWS. ....covcvcrenennieenieninianenes eee eerste snes ersten -64 -

Z) HEMeoereeoeoeereeeeessesererensiennann ssssesscsssssnsienseaessessensressasasisases menses 09 = (1) Master Item Search View ........ccooccocceuemurerrssssssssssenss muenns 63 = 2) Typed Item Search Views woven mene” 66 - “b) tem BXIENSIONS 1veverererrrereesm sessersesssesscssensasnascassssesssssssnrannnss waren 06 = (1) Master Extension Search VIEW ..........ccveeriersessnsesnes orenee= 66 = 2) Typed Extension Sea 1ch VIEWS ......ccveiiveerrnrerenns oreenn= 67 = c) NeSted BIEIMENLS -....euveeeeeeere mo ecreesieneeseerasensesasaesessenssnssassesssase aoress= 08 = d REIAHONSNIPS .vevvvereverinssrsessmereneserererensorersssisssssrensesstosnnsnass ssiens 08 = (1) Master Relationship SSearch View ......cccvvverenrnreaee cnnne 68 - (2) Relationship Instance Search Views ......coeeeeerverae weee= 69 - 10. Change Tracking & TOMDSIONES..... —vierererrrinrieiunreneremsinssorseseessase monies = 10 = a) Change TTaACKING c..ceecviinsiier merinriiseeinisesnnrormssessssnesssnsssnsscnse nm assess 10 = eh) Change Tracking in “M=aster” Search Views ............ ...- 71- : 2) Change Tracking in “Taped” Search Views .....ccc..o oom 72 -

B b) TOIIDSLOIIES «.vvereveeemrosrosssnrensanammsrreessissnsssnsnsstosssissssessostaranasns em sees” 72 - (1) Ttem TOMDSLONES ...coccoemmunressrsssissssemsanssissnssssasssssssanas mee 73 - (2) Extension TOMDBSIONES —....couueumcuniiirimsssmanissasussesusnsa mens” 73 - 3) Relationships TOMDSIOMIE .....c.vveaureurermansinanssiscsssses =eee™ 74 - (4) Tombstone ClEANUD..... ecemiosiisimmeriinsessisasessensasass sien” 74 - . 11. Helper APIs and PUNCHONS ..ococtersvsnrsmm somnscimssisneenmsinssinsissenissasasase cone = 75 - a) Function [System.Storage].GEMUEI........cvmimrmmiiimenscsssseni eine” 75- b) Function [System.Storage].GeAEXLENSION ....ccvuvimrieriniersess munis” 75 - cd) Function [System.Storage].GetRelationship .......coueeseeersrese meena 75 -

A FC 75 - a) SChEINa MELAAALA ....ovevevreereree rirsresssssssseesssnesansasssensasssssssssrsmuesa™ 19 = b) INSLANCE MELAAALA. o.eeeereeereeeeree eoorreensesirenssesessssnsssmenceeessssnn meres” 19 = 1. ODVEIVIEW 1rereereecereeeremsreosssiosanssnssesssss sostsssissasssmsassesssesssssssssnssasassans wssss = 10 = 2. Detailed Description of the Security MvIodel......cocuvrimrrareerccessieciens wns = 81 a) Security Descriptor SETUCIITE. — ....c.cuvimieierssisssensssssnsnasessse reens™ 81 = a) Access Mask FOTMAL . — ....c.cvevireeereenrriseerersnssissnssens sennne= 82 = 2) Generic Access RIghtSs ......vvunececeniirinesernersanasss sosen= 82 = (3) Standard Access Righ®s.........oiinnieeiinnenscesmvienen= 83 - b) Item Specific RIGALS c.u..ccervemvmurivernieeireernsnsssnsasescsesssessses —oneen= 84 = (1) File and Directory objeect specific rights ..........cccoocmuiinn- 84 - (2) WinFSItemRead........cuiriiernnicnsiensinnseescesesemeneeni 85 = (3) WinFSItemReadARrilontes.........cocererircrivansrereriesne meen 86 = (4) WinFSItemWrite AttriBbULES .....oovcvivceriririreeecenrnaenees mnienas 86 =

(5) WiINFSHEMWIIE .. ocecesesercescrmsssnsinessssisssscmsastssnnsnns ee” 86 - (6) WinFSHEMAAALAMK...oosccrrrsersrssssssssnresmssssscsssssss eves” 87 = (7) WinFSTtembDeletel ink ......coovnsiommmnmseneniniisniisinisnins snes” 87 - ; (8) Rights to delete an HeM......crmrsussmumsucmmssssssssisssss sonee™ 87 = (9) Rights to COPY AN RAEMLcrueiuniirisnnsanisnascisissnsnnsans mes 88 - (10) Rights tO MOVE AN HEIM..coirieiiiniarencninssisissnsisss menses” 88 - (11) Rights to view the security policy on an tem overs merece 88 = (12) Rights to change the security policy on an item... .....- 88 - (13) Rights that don’t wave a direct equivalent.......coeee cone 89 - 3. IDPIEMENEAtON. .....oviumerrnrsrrensesn om sesssssssamssasssssnssesscasrsssinsonenanssescas sence = 89 - a) Creating a new item in a COntAMNer oi irireniienic iris nee” 90 - b) Adding an explicit ACL t 0 aR Item. ..ovceremniirimmimmnnnssenrnns sm ines 90 - c) Adding a holding Relatiomship to an item. .....coeinnreersees menue” 90 - d) Deleting a holding Relati«onship from an item .......cceevves —uvernen 91 - e) Deleting an explicit ACL- from an item..........covvuereieiesniees cvinen 91 - f) Modifying an ACL assoc ated with an item.......ccooveeeveens ceveenes 91 -

F. NOTIFICATIONS AND CHANGE TRACKING .....coniirennisnaserssssces ovieene = 91 = 1. Storage Change EVENS... ....cc.ccee ceurrerrrssrsmsssssssssssassacsssssssssssssisns risers = 92 = : a) EVENS c.vevveveeereeseeseeereeae cesessessesesensesasmassassnsasasessossssesssresse asosorse™ IR = b) WV ALCHEES ...vvveeeeneeereeer oe cressresessesessasansnsssanesmesanseesassassassnsnme snsvenss™ 33 = 2. Change Tracking and Notification Generation Mechanism... = 94 - a) Change TIACKINE v.cvverres mo eorcercsissirenenssssessssnsssnsssesessasssssmnssans™ 99 = b) Timestamp Management .........eecreeseurreesemessinesesisessassscmeenesesm 90 = c) Data Change Detection - Event Detection ..........cccecoevevneimeneeni= 97 = 1. Storage Platform-to-Storage Pla&form Synchronization .............. —.......- 98 -

a) Syrchronization (Sync) Controllings Applications ..........esere — 99 - ® b) SCHEMA ANMNOLALON «.vveereermerererirmsnes mmerrsnesessissssssasssssnsansssnnssoniass = 99 - c) Sy MC CONTIGUTALON c.vvvvenrionnaerrssss suisrsmssssrmsssssenssssasssansssssss™ 101 - 1)» Community Folder - Mappi DES.....ccommimmmasanesesesess™ 01- 2» PLOLIES wvovvenreerectsersaresssssenm semtessessernsssnssssonssinnssssssaass™ 102 - 3» SCREADIES ee vereeercreesarrsoneess mm ssiresssssererraniaansrossessssssnanasaas™ 103 - d) Conflict HANAINE ....ceccsveressismranr ss momsosssssssssansamsiuenssesssnssensees = 103 - (1D Conflict DEteCtON cuvecsserees =rismessusessscssenissmsmmmanseasssncenees = 103 - (a) Knowledge-based c=onflicts -103- (®) Constraint-based conflicts -104 - 2) Conflict PIOCESSING ..cveioaams crrterenicsasesasiensansinnsoanaesensiaee —= 104 - (a) Automatic Conflict resolution -105 - (b) Conflict Logging -105- (©) Conflict inspection and resolution -107 - (d Convergence of replicas and Propagation of Conflict Resolut-ions -107 - 2. Synchrommizing to non-storage platform damta StOTeS......cceceerrereeerrrnnnen = 108 = a) SYNC SEIVICES w.eovirerniianrsrnsisnsaionse simtriminenimtsisissmsessssisssassssanss ams” 108 - (Rk) Change Enumeration ........ ecviscnnsinnieiienssmnescssssa = 108 - (2) Change Application ...c.cceoe curirnrismminsisnsrsssnsscessesnssnnnnms 109 = 3) Conflict ReSOIUHON ......evv mm cuenrrereenresrrecrcossassosersnsssnneo men 110 = b) Amdapter implementation ..........veemcserveresmecsseessanssessssnsanarennes = 110 = 4. | EX TYY: 1.7111 FOUN IIRUIIUR IRONS § § I

H TRADITIONAL FILE SYSTEM INTEROPERABILITY. ......coveurenrrerseene == 111 = 1. Model fOr Interoperability ........ooveruererissmmmiinssressiserssrnsrssnsenrensossesene += 112.

oo 2. Data SOLE FEALUTES « «.veeeverrerranraoncseseesenssenesssssssssmsnsmsnssnsns ssmsanssscsonsas = 113 - a) NOL 2 VOLUITIE o.eevveniverennneonerssnisessssmasessmsessesasissmnssan = satsasessssssssens™ 113 -

Co b) SLOLE SHIUCEUTE cvvirirerieniesererssessiersassssnnssasrensessesosas mm sissssssssassseses” 113 - c) Not all files are Migrated .......coveeerereessierssmismsnsnmsmoncsissnasens” 114 - d) NTFS namespace access to Storage Platform filess...............- 114 - e) Expected nammespace/drive JEHETS ...irnreeresnsiss nisisserernenn” 114 -

L STORAGE PLATFORM APL... iieerivinnmnensssressesionissassesss srsssssssssnsnies = 114 - 1. OVEIVIEW .eeeereeenveensseresomessnessssestessasnsassusamssassassossssrasasassaessom sireassssssassers = 115- 2. Naming and SCOPES: cueueremrmmrreniersscrstarssssmissssssssssnssasssssssmsissssasassessses = 116 - 3. Storage Platform APT COMPONENLS ........ovurrsusrmsussssesssnes amaimnrsssassnssanee = 117 - 4. Data CIASSES vvvvvvves merremsrensssemsessesssssssssssenssssresssssassessssssses mmmnsssssassasess = 118 = 5. RUNGME FrAmMEWOIK ....oooreeeeeceevemssssesnessessressssssosssssseennces smsaesssesssssnses = 123 = a) Runtime Framework Classes. ....c...erersressccrserire sommrsrssesrsnsses™ 125 (1) TEETNCONLEXL 1... vevererrerseesroreneesiressonsiseserasses seresesesessnssesnes 123 = ©) THEIL SCATCRET ..oeoveeeetsveseersessacsnsremssssassen ssssassesessssssasss= 120 = (a) Target Type ) 126 - (b) Filters -127- (©) Preparing Searches - 127 - (@ Find Options -127- 3) Item Result Stream (“FindResult”) .....cc.—coeeeerenieninnnn= 129 - b) Runtime Framework in Operation............ceseeses mensesesscsersconiser= 130 = c) Common Programming Patterns ........cco.eceeeeereeemviersersssnsenneann 130 = 0) Opening and Closing ItemContext ObjecSts...vvneennrm 130 - 2 Searching for ODJECES ......ocvunirerevnririniiens ssnnesisesssercsnnann= 131 - (a) Search Options -132- ®) FindOne and FindOnly -133-

WwV0 2005/029313 PC Y/US2003/026144 (©) Search Shortcuts on ItemConte=xt -133- @ Find by ID or Path -134- (e) The GetSearcher Pattern -134 - (3) Updating the SEOTe.......ceviriemmiimssmnenns srnsnimniseinnens 135 - 7. Support for REIAHONSIIPS.....c.voevuriemsimnsiasisnsseasssnsssssam ssssesssssinessens © 137 - a) Base RelatioT1ship TYPES .....ccoiieieriiescensiinnsssmannsinscscssnisanians” 137 - ¢))] Relati onship Class ........cocerrareecsnsisssie mannsminsssessensesiens™ 138 - (2) TtemIZeference Class. ....ccceovierrccmecsnnss aneseneessaessossaseans™ 138 - (3) TtemnIdReference Class ...ovecrercrers wunrinnissssssssasasnsens” 139 - 4) ItemPPathReference Class.....oveeeemeees eomesreseeserseessensens= 139 = (5) Relationshipld Structure... .ceceeceniisns rinreniiuninnsrerencees 140 - (6) Virtu alRelationshipCollection Class .. ...coeeeiieniieeieenies 141 - b) Generated Relationship TYPES ..cceerereesereiiseemm sunsramrassssasssereneses= 143 = a) Generated Relationship TYPES ..c.coc—vcrevremnssenennsenene= 143 = (2) RelationshipPrototype Class ........cecemueeenenreseessenensnm 143 - (3) RelationshipPrototypeCollection Classs...........ccveeueee i= 144 = c) Relationship Support in the Item Class ......... —veeevuserseressceseeencw 144 = (1) TEEN ClASS...crneeereeenrerererriensnsnsesesserssncmmomencrsseseeserensanene= 145 = (2) RelationshipCollection Class ......ceee meeusisrareseesennenne= 145 = d) Relationship Support in Search EXDIessions. .........eeeseeeeni= 145 - (1) Traversing From Items to Relationsh3ips.........ccoeconnee i= 145 - (2) Traversing From Relationships to Items .......cc.eerereeen- 146 - (3) Combining Relationship Traversal ... ....cccccoonnnincninn= 147 = e) Examples Uses of Relationship SUPPOrt ....... .ccerecorecernnnceni 147 = 1) Searching for Relationships ........cecee sieriecricecenrveenenn= 147 =

(2) Navigating from a Relationship to the Source and Target TEMS ..c ovcvirrueemserimssissssrsssssassamsnssasesacess 14-8- (3) Navigating from S ource Items to Relationships .........- 14-9 - (4) Creating Relationships [EIT 105 1:7:) [RU 150- (5) Deleting Relationships (and TEMS) cvureemrescrenenessnsnsees™ 151- 8. «Bxtending” the Storage Platform FN 2) SURI P RR 151- a) Domain BehaVIOTS...ccees cm cererrinsssensessersnsmssssesesensssmrasassssasesess” 152- b) Value-Add Behaviors ....c ceciiessesessisesissnssecsrensasesssassesssaese” 153 - c) value-add Behaviors as Service PIOVIGETS wowieeuseerecrrneneene™ 153 - 9. Design Time Framework... oeu.cee crmreisasserssssssssnsissssssssessssssssassnsenses = 1.54 - 10. Query FOrmaliSi....vercesersensesem soissrssssssserasssmsissssmmsssnasesssessessissssisecses = 155- a) FUIEE BASICS ceceverririrssrers om mrresssessonssssmsssssermsssssansrassasersasss sassssarass™ 155- b) TYPE CASES cecveroerrnnrrcansns mesemsisssassamsssesssnessassnssasnsssnseonscnsensases” 1.57 - c) FILET SYNLAX covevnvunrrrerie = errenasressssssinsssassssnstssassssastonsanss sons sssanes™ 1.57 - 11. REMOUNE .ovrererenseresseressssmsmsnssnnee memrissmemarssssssissaissssssnsssssssssssesssissnssenss = 1.58 - a) Local/Remote Transpare=ncy in the APL......c.oiivernccnicnnienn= WS8 - b) Storage Platform Implementation of REMOUNE ........ccurearivnree™ n59 - c) Accessing Non-Storage Platform StOTeS overs ceerssirennnasseens™ A159 - d) Relationship to DFS .....ccoieneenneinineesssisinssanssssn ees” 159 - 6) Relationship to GXA/MNEIEO corocrvrsrsrsommrsissssirsssessnsssss™ 159 = 12. CONSILAIMNES..cevierreereesrrsccnromerosess memiseninsraseesssssssisstasnsnassassssssrossssnnsasaeses = 59 - 13. SRAMNE coceerresssssrensesessssssssasass ammastsssssmsssssissssssensessessssmsssssssmssssnens = 161 = a) Representing 8 SATE... «oeceerrmieremmissrsesescssnissmsimssisnsieeeensse™ 162 - b) Managing Shares .......... coeeenieessinrnmisnnnsnsisssnsssesssisisnssnsensene” 162 - c) Accessing Shares......... eee messes tases sss 162 - d) DiSCOVELADILILY cevereeren crirnrirmimmrinnenressecnesssneaisnsan arene srnesesses™ 163 -

14. Semantics Of FIN. ee crmrimrsssssmsessissrsssaes mecsssnssssemsisiusssisssssasssseess © 163- 15. The Storage Platform Contacts APL...cccuree cmemmmissimsummsissremsenennesnies © 16-4 - a) Overview of System. Storage. CONtACE ...veuverenssreresararemseeeeees 16 4- b) DOMAIN BERAVIOTS ocr eeecunmrrremamsrsssesmssmsssssssnsssasssasasnssssssssseress” 16e4- 16. Storage Platform File AP ...cieuuisrrrreinsines muussemssssssssmmrssssessnsssssnss © 16m6- a) TNEOAUCHION covers verrereseecsirmrensssnesann sessssssmssssrenssasnsnsssencasasnasssses 166- (1) Reflecting an NTFS Volume in the Storage

PLATO c.vveeeeesere everenencseessmssrasssnsarasssssssnsssasasassanssensaces” 1656 - 2) Creating Files and Directoxies in the storage

Platform NAMESPACE ...cvuevs wuumsserssssusmsssmssseremsssassnssssseess” 1466 - b) Tile SCHEMA cv eerirrrrerenersacscacresssrasn ssincssiiersmssssssssssnsssasnstssenens” 167 - c) Overview of System. Storage Files ......oominniimanensiensenn” 167- d) COE ERAMPIES.rvevrersersrrenssansensone monismssssssassiasitessssssasesssssassssess” 1 68 - (1) Opening a File and Writing to Tt ooreeeeeeenneessensrsesn= 168 = 2) USING QUETIES ..ocvverrasereres crimssssnarmsssracsseassnssusassanasissasses” 1.68 - e) Domain Behaviors ....cieeerreserssanes srsresrsrmssisrssmesnsssoncsssntasinsnsses” B69 -

J. CONCLUSION ...ooreenireriesieseresissesssasensssssssssnen siubsssmsssssessasssisssssensacasasssosess = h69 -

- L INTRODUCTION

[0056] The subje=ct matter of the present invention is de=scribed with specificity to meet statutory requiremerats. However, the description itself is rmot intended to limit the scope of this patent. Rather, the- inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or cormbinations of steps similar to the ones described in this document, in conjunction with other pxesent or future technologies.

Moreover, although the tem “step” may be used herein to conneote different elements of methods employed, the te=rm should not be interpreted as implyi mg any particular order . among or between various steps herein disclosed unless and exczept when the order of individual steps is explicmtly described.

A. EXEMPLARY COMPUTING ENVIRONMENT

[0057] Numerous embodiments of the present inventicon may execute on a computer. Fig. 1 and the following discussion is intended to provide a brief general description of a suitable computing environment in which the imnvention may be implemented

Although not required, v-arious aspects of the invention may be described in the general context of computer executable instructions, such as program modules, being executed by a computer, such as a cliert workstation or a server. Generally, program modules include routines, programs, obje=cts, components, data structures and the like that perform particular tasks or implement parti«cular abstract data types. Moreover, thee invention may be practiced with other computer system configurations, including hand hel d devices, multi processor systems, microprocessor based or programmable consumer ele ctronics, network PCs, minicomputers, mainfra-ime computers and the like. The invention may also be practiced in distributed computing emnvironments where tasks are performecd by remote processing devices that are linked through 2 communications network. In a distributed computing environment, program modules may te located in both local and remote mexory storage devices.

[0058] As shown in Fig. 1, an exemplary general purpose computing system includes a conventional personal computer 20 or the like, including a processing unit 21, a system memory 22, and_ a system bus 23 that couples various System components including the system memory to thhe processing unit 21. The system bus 23 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a localk bus using any of a varie=ty of bus architectures. The system me=mory includes read only memory (ROM) 241 and random access memory (RAM) 2.5. A basic input/output system 26 (BIOS), containingz the basic routines that help to transfer information between elements within the personal computer 20, such as during start up, is stored in ROM 24. The personaal computer 20 may further include a hard disk drive 27 for reading from and writing to a hard disk, not shown, a magnetic disk drive 28 for reading fromm or writing to a removable magnetic disk 29, and an optical disk drive 30 for reading from or writing to a removable ’ optical disk 31 such as a CD ROM or other optical media. The hard disk drive 27, magnetic disk drive 28, and_ optical disk drive 30 are connected to the system bus 23 by a hard disk drive interface 32 _, a magnetic disk drive interface 33, and an optical drive interface 34, respectively. The drives and their associated computer readable media provide non volatile storage of computer readable instructions, data structures, program modules and other data for the personal c-omputer 20. Although the exemplary environment described herein empmloys a hard disk, a removable magnetic disk 29 and a removable optical disk 31, it should be appreciated by th-ose skilled in the art that other types of computer readable media which can store data that is =accessible by a computer, such as magnetic cassettes, flash memory carcls, digital video disk=s, Bernoulli cartridges, random access memories (RAMs), read only memories (ROM s) and the like may also be used in the exemplary operating environment.

Likewise, the exesmpldry environment may also include many types of monitoring device=s such as heat sensors and security or fire alarm systems, and other sources of information— . [0059] _A number of program modules may be stored on the hard disk, magnetic disk 29, optical isk 31, ROM 24 or RAM 25, including an operating system 35, one or nore application progmrams 36, other program modules 37 and program data 38. A user may ermter commands and isnformation into the personal computer 20 through input devices such as a keyboard 40 and. pointing device 42. Other input devices (not shown) may include a microphone, joy-stick, game pad, satellite disk, scanner or the like. These and other input= devices are often connected to the processing unit 21 through a serial port interface 46 that is coupled to the ssystem bus, but may be connected by other interfaces, such as a parallel pwort, game port or uni versal seria] bus (USB). A monitor 47 or other type of display device is also connected to thes system bus 23 via an interface, such as a video adapter 48. In addition t=o the monitor 47, personal computers typically include other peripheral output devices (not shown), such as speakers and printers. The exemplary system of Fig. 1 also includes a h-ost :

© acapter 55, Small Computer System Interface (SCSI) bus 56, and an ext=emal storage device : . 62 connected to the SCSI bus 56. : [0060] The personal computer 20 may operate in a networked - environment using logical connections to one or more reIote computers, such as a remote ~computer 49. The reemote computer 49 may be another personal computer, a server, a routeer, a network PC, a p=eer device or other common network node, and typically includes man_y or all of the elements described above relative to the personal computer 20, althougin only a memory sstorage device 50 has been illustrated in Fig. 1. The logical connections= depicted in Fig. 1 imnclude a local area network (LAN) S51 and a wide area network (WAND) 52. Such networking e=mvironments are commonplace in offices, enterprise wide computer ne=tworks, intranets and ) thhe Internet.

[0061] When used in a LAIN networking environment, the per—sonal computer 20 is connected to the LAN 51 through a network interface or adapter 53. W_hen used in a WAN ’ retworking environment, the personal computer 20 typically includes am modem 54 or other : means for establishing communications over the wide area network 52. such as the Internet. :

Whe modem 54, which may be internal or external, is connected to the ssystem bus 23 via the sserial port interface 46. In a networked environment, program modules depicted relative to the personal computer 20, or portions thereof, may be stored in the rerrmote memory storage levice. It will be appreciated that the network connections shown are e=xemplary and other reans of establishing a communications link between the computers nay be used.

[0062] As illustrated in the block diagram of Fig. 2, a computer system 200 can be roughly divided into three component groups: the hardware component 202, the * Ehardware/software interface system component 204, and the applicatio ns programs component 206 (also referred to as the “user component” or “software component” in certain contexts herein).

[0063] In various embodinnents of a computer system 200, armd referring back to Fig. 1, the hardware component 202 may comprise the central processing unit (CPU) 21, the mmemory (both ROM 24 and RAM 25), the basic input/output system (BBIOS) 26, and various

Fnput/output (VO) devices such as a keyboard 40, a mouse 42, a moniteor 47, and/or a printer «(not shown), among other things. T he hardware component 202 compmrises the basic physical fnfrastructure for the computer system 200.

[0064] The applications programs component 206 comprises various soft-ware programs including but not limited to compilers, database systems, word processoms, business progyrams, videogames, and so forth. Application programs provide the means by which computer resources are utilized to solve problems, provide solutions, and process «lata for var# ous users (machines, other computer systems, and/or end-users).

[0065] The hardware/software interface system component 204 comprises (and, in © some embodiments, may solely consist of) an operating system that itself compris es, in most casees, a shell and a kemel. An “operating system” (OS) is a special program that acts as an intesrmediary between application programs and computer hardware. The hardwamre/software interface system component 204 may also comprise a virtual machine manager (WWMM), a

Co-mmon Language Runtime (CLR) or its functional equivalent, a Java Virtual Machine (WM) or its functional equivalent, or other such software components in the plac eof or in adciition to the operating system in a computer system. The purpose of a hardwar—e/software inteerface system is to provide an environment in which a user can execute application programs. The goal of any hardware/software interface system is to make the computer sysstem convenient to use, as well as utilize the computer hardware in an efficient manner.

[0066] The hardware/software interface system is generally loaded into a computer sysstem at startup and thereafter manages all of the application programs in the co-mputer sysstem. The application programs interact with the hardware/software interface System by » recjuesting services via an application program interface (APT). Some applicatiora programs : ) enable end-users to interact with the hardware/software interface system via a usesr interface suech as a command language or a graphical user interface (GUI). : [0067] A hardware/software interface system traditionally performs a v ariety of - semrvices for applications. In a multitasking hardware/software interface system v=vhere - mmultiple programs may be running at the same time, the hardware/software interface system : . de=termines which applications should run in what order and how much time shoud be ' alKowed for each application before switching to another application for a turn. The haerdware/software interface system also manages the sharing of internal memory among multiple applications, and handles input and output to and from attached hardware devices su_ch as hard disks, printers, and dial-up ports. The hardware/software interface system also se nds messages to each application (and, in certain case, to the end-user) regarding the status of= operations and any errors that may have occurred. The hardware/software int=erface system can also offload the maragement of batch jobs (e.g., printing) ‘so that the initiating applicat®on is freed from this work znd can resume other processing and/osx operations. On computers "that can provide parallel processing, a hardware/software inter—face system also manages dividing a program so that it runs on more than one processor at a time. . [0068] A hard ware/software interface system shell (ssimply referred to herein as a— “shell” is an interactives end-user interface to a hardware/soft=ware interface system. (A shuell may also be referred to as a “command interpreter” or, in an owperating system, as an “operating system shell”). A shell is the outer layer of a hard=ware/software interface syste=m that is directly accessibele by application programs and/or end—users. In contrast to a shell, a kernel is a hardware/so-ftware interface system’s innermost la_yer that interacts directly with the hardware components.

[0069] While itis envisioned that numerous embod ments of the present invention are particularly well-suited for computerized systems, nothinge in this document is intendead to : limit the invention to s-uch embodiments. On the contrary, as used herein the term “‘compeater : system” is intended to encompass any and all devices capable of storing and processing : information and/or capable of using the stored information tow control the behavior or . execution of the devices itself, regardless of whether such dev-ices are electronic, mechanical, logical, or virtual in nature.

B. TRADNRTIONAL FILE-BASED STORAGE

[0070] In mosstcomputer systems today, “files” are units of storable information- that may include the hardwos are/software interface system as well as application programs, datas sets, and so forth. In zmll modern hardware/software interface= systems (Windows, Unix,

Linux, Mac OS, virtual machine systems, and so forth), files are the basic discrete (storabmle and retrievable) units of information (e.g., data, programs, ard so forth) that can be manipulated by the hardware/software interface system. Groups of files are generally organized in “folders.” ’ In Microsoft Windows, the Macintossh OS, and other . hardware/software interface systems, a folder is a collection of files that can be retrieved, ;. moved, and otherwise manipulated as single units of informzation. These folders, intum, are .* organized in a tree-based hierarchical arrangement called a “directory” (discussed in mor-e detail herein below). In certain other hardware/software intesrface systems, such as DOS, + z/OS and most Unix-based operating systems, the terms “dir-ectory” and/or “folder” are interchangeable, and early Apple computer systems (e.g., the Apple Ile) Lased the term “catalog” instead of directory; hoswever, as used herein, all of these termss are deemed to be synonymous and interchangeable and are intended to further include all Other equivalent ’ terms for and references to hierarchical information storage structures and their folder and file components. : [0071] Traditionally, a directory (a.k.a. a directory of folders)isa tree-based hierarchical structure wherein filles are grouped into folders and folder, i turn, are arranged according to relative nodal locations that comprise the directory tree. Foor example, as

Cl ’ illustrated in Figure 2A, a DOS—based file system base folder (or “root directory”) 212 may comprise a plurality of folders 2-14, each of which may further comprises additional folders (as “subfolders” of that particular folder) 216, and each of these may also ceomprise additional folders 218 ad infinitum. Each -of these folders may have one or more files 220 although, at ‘the hardware/software interface system level, the individual files in a foslder have nothing in "common other than their location in the tree hierarchy. Not surprisinglsy, this approach of organizing files into folder hiersarchies indirectly reflects the physical omrganization of typical storage media used to store thesse files (¢.g., hard disks, floppy disks, CED-ROM, etc.). . [0072] In addition to the foregoing, each folder is a container for its subfolders and its files—that is, each folder ovens its subfolders and files. For examples, when a folder is deleted by the hardware/softwanre interface system, that folder’s subfolcders and files are also deleted (which, in the case of e ach subfolder, further includes its own subfolders and files . recursively). Likewise, each file is generally owned by only one foldem and, although a file . can be copied and the copy loc ated in a different folder, a copy of a files is itself a distinct and "separate unit that has no direct connection to the original (e.g., changes to the original file are not mirrored in the copy file at the hardware/software interface system. level). In this regard, files and folders are therefore Characteristically “physical” in nature bescause folders are the treated like physical containers, and files are treated as discrete and separate physical elements inside these containexs.

II. A NEW STORAGE PLATFORM FOR ORGANIZING=,

SEARCHING, AND SHARING DATA

[0073] The present iravention is directed to a storage platfornn for organizing, searching, and sharing data. The storage platform of the present inveration extends and broadens the data platform beyond the kinds of existing file systems and database systems discussed above, and is designecd to be the store for all types of data, including a new form eof data called Items.

A. GLOSSARY

[0074] As used herein and in the claims, the following terms have the following meanings: . [0075] An “Item” is ar unit of storable information accessible to a hardware/software interface sysstem that, unlike a simple file, iss an object having a basic se=t of properties that are commonly supported across all objects ex=posed to an end-user by the= hardware/software interface sysstem shell. Items also have propoerties and relationships that are commonly supported acrosss all Item types including featurees that allow new properties and relationships to be introduc=ed (and discussed in great detail later herein). " [0076] An “operating system” (OS) isa special prog_ram that acts as an intermediary between application programs and computer hardware. An operating system comprises, in most cases, a she-lland a kernel.

[0077] A “hardware/ssoftware interface system” is software, or a combination of hardware and software, that sexves as the interface between thee underlying hardware components of a computer syst-em and applications that execuste on the computer system. A hardware/software interface system typically comprises (and, Jin some embodiments, may solely consist of) an operating system. A hardware/software interface system may also comprise a virtual machine ma-nager (VMM), a Common Lan zguage Runtime (CLR) or its- functional equivalent, a Java Virtual Machine (JVM) or its furctional equivalent, or other such software components in tthe place of or in addition to the operating system in a computer system. The purpose of a hardlware/software interface system_ is to provide an environment in which a user can execute appli_cation programs. The goal of aany hardware/software interf=ace system is to make the computer system convenient to use, as well as utilize the computer hardware in an efficient manner. :

B. STORAGE PLATFORM OVERVIEW

[0078] Referring to Figure 3, a storage platform 300® in accordance with the present invention comprises a data sto=re 302 implemented on a databaase engine 314. In one embodiment, the database engzine comprises a relational database engine with object relational extensions. In one embodiment, the relational database emngine 314 comprises the

Microsoft SQL Server relational database engine.

[0079] The data store 3 O2 implements a data model 304 ttmat supports the

B . organization, searching, sharing, synchronization, and security of d=ata. Specific types of data are described in schemas, such as schemas 340, and the storage platform 300 provides tools 346 for deploying those schemas as well as for extending those sch.emas, as described more fully below.

[0080] A change tracki ng mechanism 306 implemented within the data store 302 provides the ability track changess to the data store. The data store 302 also provides security capabilities 308 and a promotior/demotion capability 310, both of —which are discussed more fully below. The data store 302 also provides a set of application programming interfaces 312 to expose the capabilities of the data store 302 to other storage- platform components and application programs (e.g., application programs 350a, 350b, and 350c) that utilize the . : storage platform.

[0081] The storage platform of the present invention still further comprises an application programming interfaces (API) 322, which enables appMication programs, such as application programs 350a, 350%, and 350c, to access all of the for-egoing capabilities of the storage platform and to access the data described in the schemas. "The storage platform API 322 may be used by application programs in combination with oth er APIs, such as the OLE

DB API 324 and the Microsoft "Windows Win32 API 326.

[0082] The storage platform 300 of the present invention may provide a variety of services 328 to application programs, including a synchronization service 330 that facilitates the sharing of data among users- or systems. For example, the synchronization service 330 may enable interoperability with other data stores 340 having the ssame format as data store 302, as well as access to data stores 342 having other formats. Thee storage platform 300 also provides file system capabilities that allow interoperability of the cata store 302 with existing file systems, such as the Windo ws NTFS files system 318.

[0083] In at least some embodiments, the storage platformm 320 may also provide application programs with addisional capabilities for enabling datam to be acted upon and for enabling interaction with other systems. These capabilities may bee embodied in the form of additional services 328, such as an Info Agent service 334 and a notification service 332, as well as in the form of other utilities 336.

[0084] In at lezast some embodiments, the storage platform is embodied in, or forms an integral part of, the h ardware/software interface system of a computer system. For example, and without limitation, the storage platform of the present invention may be embodied in, or form am integral part of, an operating system, a vixrtual machine manager (VMM), a Common Lamguage Runtime (CLR) or its functional ecguivalent, or a Java Virtual

Machine (JVM) or its fanctional equivalent.

[0085] Throug=h its common storage foundation, and schematized data, the storage platform of the present -invention enables more efficient application development for consumers, knowledge workers and enterprises. It offers a rich and extensible programming surface area that not only makes available the capabilities inherent in its data model, but also embraces and extends existing file system and database access methods.

[0086] In the Following description, and in various ones of the figures, the storage platform 300 of the pre sent invention may be referred to as “WinJES.” However, use of this name to refer to the storage platform is solely for convenience of description and is not intended to be limiting in any way.

C. THE D.ATA MODEL

[0087] The data store 302 of the storage platform 300 of the present invention implements a data model that supports the organization, searching, sharing, synchronization, and security of data that resides in the store. In the data model of the present invention, an “Item” is the fundameratal unit of storage information. The data xmodel provides a mechanism for declaring Items andl Item extensions and for establishing relationships between Items and for organizing Items in. Item Folders and in Categories, as describ>ed more fully below. ’ [0088] The data model relies on two primitive mechanisms, Types and

Relationships. Types are structures which provide a format which governs the form of an instance of the Type. T he format is expressed as an ordered set of Properties. A Property is a name for a value or set of values of a given Type. For example a USPostalAddress type might have the properties Street, City, Zip, State in which Street, City and State are of type String "and Zip is of Type Int32. Street may be multi-valued (i.e. a set of values) allowing the ‘address to have more than one value for the Street property. The system defines certain primitive types that cax1 be used in the construction of other types - these include String, . Binary, Boolean, Int16s, Int32, Int64, Single, Double, Byte, DateTime, Decimal and GUID.

The Properties of a Type may be defined usimg any of the primitive types or (x=vith some restrictions noted below) any of the construc ted types. For example a Locatior Type might be defined that had Properties Coordinate and Address where the Address Property is of Type

USPostaalAddress as described above. Propexties may also be required or optional. 0089] Relationships can be declared and represent a mapping between the sets of instance=s of two types. For example there m ay be a Relationship declared between the Person

Type aad the Location Type called LivesAt which defines which people live zat which locatioras. The Relationship has a name, two» endpoints, namely a source endp-oint and a target ; endpoirt. Relationships may also have an ordered set of properties. Both the Source and

Target endpoints have a Name and a Type. ¥or example the LivesAt Relatiormship has a

Source called Occupant of Type Person and. a Target called Dwelling of Types Location and in addition has properties StartDate and EncDate indicating the period of time for which the occupamt lived at the dwelling. Note that a Person may live at multiple dwellmngs over time and a dwelling may have multiple occupant s so the most likely place to put tine StartDate. and

EndDatte information is on the relationship itself.

[0090] Relationships define a mapeping between instances that is comnstrained by the types given as the endpoint types. For example the LivesAt relationship canrmot be a relatioraship in which an Automobile is the © Occupant because an Automobile= is not a Person.

[0091] The data model does allow” the definition of a subtype-super—type relationship between types. The subtype-supertype relationship also known as the BaseT=ype relationship is defined in such a way that if Type A is a BaseType for Type B it must be tthe case that every i-nstance of B is also an instance of A.. Another way of expressing this 3s that every " instanc=e that conforms to B must also conform to A. If, for example A has a property Name of Typee String while B has a property Age of Type Intl6, it follows that any instance of B . must h ave both a Name and an Age. The type hierarchy may be envisaged ass an tree with a single ssupertype at the root. The branches from the root provide the first levesl subtypes, the branch._es at this level provide the second level subtypes and so on to the leaf—most subtypes which themselves do not have any subtypes. The tree is not constrained to be of a uniform depth but cannot contain any cycles. A given Type may have zero or many ssubtypes and zero orone super type. A given instance may conform to at most one type together with that type’s super t=ypes. To put it another way, for a git ven instance at any level in the tr=ee the instance may conform to at most one subtype at that level.

[0092] A type is said to be Abstract if instances of the type must also be an instance- of a subtype of the type. 1. Items

[0093] An Iter is a unit of storable information th at, unlike a simple file, is an object having a basic sett of properties that are commonly sumpported across all objects exposed to an end-user ©r application program by the storagze platform. Items also have properties and relationskiips that are commonly supported accross all Item types including features that allow new properties and relationships to be introduced, as discussed below.

[0094] Items are the objects for common operatioms such as copy, delete, move, open, print, backup, restore, replicate, and so forth. Items amre the units that can be stored ancl retrieved, and all forms of storable information manipulate] by the storage platform exist as

Items, properties of Iterms, or Relationships between Items, each of which is discussed in greater detail herein bel ow. }

[0095] Items are intended to represent real-world and readily-understandable units of data like Contacts, People, Services, Locations, Docume=nts (of all various sorts), and so on. Fig. 5A is a block cliagram illustrating the structure of =an Item. The unqualified name o the Item is “Location”. The qualified name of the Item is “CCore.Location” which indicates that this Ttem structure is defined as a specific type of Item in the Core Schema. (The Core

Schema is discussed in more detail later herein.)

[0096] The Location Item has a plurality of prope=rties including EAddresses,

MetropolitanRegion, Neighborhood, and PostalAddresses. The specific type of property fox . each is indicated immediately following the property name= and is separated from the property - name by a colon (*:”). “To the right of the type name, the nuamber of values permitted for that property type is indicat ed between brackets (“[ J”) whereir an asterisk (“*") to the right of the colon (*:”) indicates an unspecified and/or unlimited ntamber (“many”). A “1” to the rigsht " of the colon indicates that there can be at most one value. _A zero (“0”) to the left of the coleon } indicates that the propestty is optional (there may be no valwue at all). A “1” to the left of the _ colon indicates that there must be at least one value (the property is required). Neighborhood and MetropolitanRegicsn are both of type “nvarchar” (or ecguivalent) which is a predefined data type or “simple type” (and denoted herein by the lack of capitalization). EAddresses amd

PostalAddresses, howe ver, are properties of defined types or “complex types” (as denoted .herein by capitalizatiora) of types EAddress and PostalAdd-xess respectively. A complex type i-s type that is derived from one or more simple data types and/or from other complex types. “Whe complex types for the properties of an Item also constitute “neste elements” since the

Cletails of the complex type are nested into the immediate Item to defirme its properties, and the { formation pertaining to these complex types is maintained with the Item that has these

Properties (within the Item’s boundary, as discussed later herein). These concepts of typing are well known and readily appreciated by those of skill in the art.

[0097] Fig 5B is a block diagram illustrating the complex pr=operty types

WPostal Address and EAddress. The PostalAddress property type defines that an Item of goroperty type Postal Address can be expected to have zero or one City values, zero or one «CountryCode values, zero or one MailStop values, and any number (z=ero to many) of

Postal AddressTypes, and so on and so forth. In this way, the shape off the data for a —particular property in an Item is hereby" defined. The EAddress prope=rty type is similarly defined as shown. Although optionally, used herein this Application, - another way to : —represent the complex types in the Loc ation Item is to draw the Item with the individual —properties of each complex type listed therein. Fig. 5C is a block diagram illustrating the

Location Item wherein its complex types are further described. However, it should be understood that this alternative representation of the Location Item in this Fig. 5C is for the exact same Item illustrated in Fig. SA. The storage platform of the present invention also allows subtyping whereby one property type can be a subtype of anot=her (where the one property type inherits the properties of another, parent property type) —

[0098] Similar to but distinct from properties and their prop-erty types, Items } inherently represent their own Item Ty pes that can also be the subjectt of subtyping. In other " words, the storage platform in several embodiments of the present inwvention allows an Item be a subtype of another Item (whereby the one Itern inherits the praoperties of the other, parent Item). Moreover, for various ermbodiments of the present inve=ntion, every Item is a "subtype of the “Item” Item type which is the first and foundational Ite=m type found in the

Base Schema. (The Base Schema will also be discussed in detail late=r herein.) Fig. 6A illustrates an Item, the Location Item im this Instance, as being a subt_ype of the Item Item type found in the Base Schema. In this drawing, the arrow indicates —that the Location Item (like all other Items) is a subtype of the Item Item type. The Item Item type, as the foundational Item from which all othex Items are derived, has a number of important properties such as Itemld and various &imestamps, and thereby define=s the standard properties off all Items in an operating system. In the present figure, these properties of the Item Item type are inherited by Location and there by become properties of Location.

[0099] Another way to represe=nt the properties in the Location Item inherited from the Xtem Item type is to draw Location with the individual properties of each poroperty type facom the parent Item listed therein. Fig- 6B is a block diagram illustrating the= Location Item wherein its inherited types described in addition to its immediate properties. Kt should be ‘noted and understood that this Item is the same Item illustrated in Fig. SA, although in the ‘peresent figure Location is illustrated with all of its properties, both immediate=—shown in twoth this figure and Fig. 5A—and inhex-ited—shown in this figure but not Figz. 5A (whereas in : Fig. SA these properties are referenced by showing with an arrow that the Lo«cation Item is a subtype of the Item Item type).

[0100] - Items are stand-alone Objects; thus, if you delete an Item, all of the Items iommediate and inherited properties are also deleted. Similarly, when retrievimg an Item, what i s received is the Item and all of its imrmediate and inherited properties (inclu ding the i formation pertaining to its complex property types). Certain embodiments of the present

Lhvention may enable one to request a Subset of properties when retrieving a specific Item; however, the default for many such embodiments is to provide the Item with all of its i mmediate and inherited properties when retrieved. Moreover, the properties of Items can also be extended by adding new properties to the existing properties of that Item’s type. “These “extensions” are thereafter bona fide properties of the Item and subtypes of that Item type may automatically include the extension properties.

[0101] The “boundary” of thes Item is represented by its properties (Cincluding complex property types, extensions, and so forth). An Item’s boundary also yepresents the

Rimit of an operation performed on an Item such as copy, delete, move, creates, and so on. For example, in several embodiments of th present invention, when an Item is ¢ opied, everything within that Item’s boundary is also copied. For each Item, the bo undary encompasses the following: e The Item Type of the Item zand, if the Item is a subtype of another= Item (as is the case in several embodiments of the present invention where all Items are y derived from a single Item and Item Type in the Base Schema), any applicable . subtype information (that is, in#formation pertaining to the parent Iter Type).

V0 2005/029313 PCH/US2003/026144

If the original Item being copied is a subtype of another Item, the cosy may also be a subtype of that same Itenn. e The Item’s complex-type properties and extensions, if any. If th e original

Item has properties of complex types (native or extended), the copy may also have the same complex types. e The Item’s records on “ownership relationships”, that is, the Iter x's own list of what other Items (the “Target Items”) are owned by the present Item (the “Owning Item”). This is particularly relevant in regard to Item Folders, discussed more fully below, and the rule stated below that all Items must belong to at least one Item Folder. Moreover, in regard to embedde=d items— discussed more fully below—an embedded item is considered to be part of the

Ttem in which it is embedded for Operations such as copy, delete, armd the like. 2. Item Identification

[0102] Items are uniquely identified within the global items space= with an ItemID.

The TBase.Item type defines a field ItemIID of type GUID that stores the identity for the Item.

An Item must have exactly one identity im the data store 302. a) Item References

[0103] An item reference is a data structure that contains information to locate and identify an Item. In the data model, an abstract type is defined named Iterm Reference from . whicsh all item reference types derive. Th e ItemReference type defines a vimrtual method : nameed Resolve. The Resolve method resolves the ltemReference and returns an Item. This method is overridden by the concrete subtypes of ItemReference, which implement a function that retrieves an Item given a reference. The Resolve method is invoked as part of the storage platform API 322. §)) ItexmIDReference

[0104] ItemIDReference is a subtype of ItemReference. It defines a Locator and an ’ Iterm ID field. The Locator field names (i. €. identifies) an item domain. It is processed by a locator resolution method that can resolv e the value of the Locator to an item domain. The

Iterm ID field is of type ItemID

2) XtemPathReference

[0105] ItemPathReference is a specialization of ItemReference that defines a . Locator and a Path field. The Locator field identifies an item domain. It —is processed by a locator resolution method that can resolve the value of the Locator to an item domain. The

Path field contains a (relative) path in the storage platform namespace rooted at the item domain provided by the Locator.

[0106] This type of reference cannot be used in a set operation . The reference must generally be resolved through a path resolution process. The Resolve method of the storage platform API 322 provides this functi onality. b) Referemce type hierarchy

[0107] The reference forms discussed above are represented thmrough the reference type hierarchy illustrated in. Figure 11. Additional reference t=ypes that inherit from these types can be defined in the schemas. They can be useed in 2 relationship declaration as type of thes target field. 3. Item Folders zand Categories

[0108] As discussed more fully below, groups of Items can are= organized into special Items called Item Folders (which are not to be confused with file folders). Unlike in most file systems, however, an Item can belong to more than one Item Folder, such that when an Ttem is accessed in one Item Folder and revised, this revised Item camn then be accessed: directly from another Item folder. In essence, although access to an Iterm may occur from different Item Folders, what is actually being accessed is in fact the ver=y same Item.

However, an Item Folder does not necessarily own all of its member Ite=ms, or may simply co-own Items in conjunction with other folders, such that the deletion o—f an Item Folder does not necessarily result in the deletion Of the Item. Nevertheless, in sever—al embodiments of the © present invention, an Item must belong to at least one Item Folder so thaat if the sole Item

Folder for a particular Item is deleted then, for some embodiments, the _Ttem is automatically deleted or, in alternative embodiments, the Item automatically becomes a member of a default Item Folder (e.g., a “Trash Can” Item Folder conceptually simil=ar to similarly-named folders used in various file-and-foldex-based systems).

©. WO 2005/029313 PCT/US2003/026144

[0109] As also discussed. more fully below, Items may also be=long to Categories based on common described characteristic such as (a) an Item Type (or~ Types), (b) a specific . immediate or inherited property (Or properties), or (c) a specific value Cor values) corresponding to an Item property. For example, a Item comprising speecific properties for personal contact information might automatically belong to a Contact Category, and any Item having contact information properties would likewise automatically be Jong to this Category.

Likewise, any Ite having a location property with a value of “New Yeork City” might automatically belong to a NewYoxkCity Category.

[0110] Categories are conceptually different form Item Folde=rs in that, whereas

Ttem Folders may comprise [tems that are not interrelated (i.e., without a common described characteristic), each Item in a Category has a common type, property, «or value (a “commonality”) that is described for that Category, and it is this commonality that forms the basis for its relationship to and armong the other Items in the Category. Moreover, whereas an

Item’s membership in a particulax Folder is not compulsory based on any particular aspect of that Item, for certain embodiments all Items having a commonality categorically related to a . Category might automatically become a member of the Category at thes hardware/software interface system level. Conceptu ally, Categories can also be thought ©f as virtual Item

Folders whose membership is based on the results of a specific query &such as in the context of a database), and Items that meet the conditions of this query (define=d by the commonalities of the Category) would thus comprise the Category’s membership.

[0111] Fig. 4 illustrates the structural relationship between It ems, Item Folders, and

Categories in various embodiments of the present invention. A plurality of Items 402, 404, 406, 408, 410,412, 414, 416, 418, and 420 are members of various Ite=m Folders 422, 424, 426, 428, and 430. Some Items rmay belong to more than one Item Fo Ider, e.g., Item 402 “belong to Item Folders 422 and 4-24. Some Items, e.g., Item 402, 404, 406, 408, 410, and 412 are also members of one or more Categories 432, 434, and 436, while other times, e.g., Items 414, 416, 418, and 420, may belong to no Categories (although this is largely unlikely in ~ certain embodiments where the possession of any property automaticamlly implies membership in a Category, and thus an Item vwould have to be completely featurele=ss in order not to be a member of any category in such :an embodiment). In contrast to the hierarchical structure of folders, both Categories and Iter Folders have structures more akin to directed graphs as

B WO 20055/029313 PCT/US2Z=03/026144 shown. In any event, the Items, Item Folders, and Categories are all Items (albeit= of different

Item Types). . [01712] In contrast to files, folders, and directories, the Items, Item Folde=rs, and

Categories of the present invention are not character stically “physical” in nature because they do not have conceptual equivalents of physical containers, and therefore Items may exist in more than one such location. The ability for Items to exist in more than one Item Folder location as well as being organized into Categories perovides an enhanced and enmriched degree of data mamipulation and storage structure capabilities at the hardware/software i_nterface level, beyond that currently available in the art. 4. Schemas . a) Base Schema

[0113] To provide a universal foundation f=or the creation and use of Items, various embodiments of the storage platform of the present invention comprise a Base S~chema that establishes a conceptual framework for creating and organizing Items and propemties. The

Base Schexma defines certain special types of Items and properties, and the featumres of these * special forandational types from which subtypes can be further derived. The use of this Base

Schema allows a programmer to conceptually distin_guish Items (and their respective types) from properties (and their respective types). Moreover, the Base Schema sets fomrth the ‘foundatioraal set of properties that all Items may pos sess as all Items (and their ¢ orresponding

Item T'ype s) are derived from this foundational Iter in the Base Schema (and its * corresponding Item Type). [0E14] As illustrated in Fig. 7, and in regar—d to several embodiments off the present invention, the Base Schema defines three top-level types: Item, Extension, and PropertyBase. .As shown, the Item type is defined by the properties of this foundational “Item” Iter type. In contrast, the top level property type “PropertyBase” has no predefined propertie-s and is merely thes anchor from which all other property typ-es are derived and through v=vhich all derived property types are interrelated (being commonly derived from the single= property type). The Extension type properties define which Xtem the extension extends a=s well as identification to distinguish one extension from another as an Item may have multiple extensions.

[0115] Item Folder is a subtype of the Item Item type that, in addition to the properties inherited farom Item, features a Relationship for establishing links to its members (if any), whereas botk IdentityKey and Property are subtypes of PropertyBase. CategoryRef, in turn, is a subtype Of IdentityKey. b) Core Schema

[0116] Various embodiments of the storage platform of the present invention further- comprise a Core Sch ema that provides a conceptual framew«rk for top-level Items type : structures. Fig. 8A iss a block diagram illustrating Items in the Core Schema, and Fig. 8B is a block diagram illustrating the property types in the Core Schema. The distinction made between files with dil fferent extensions (*.com, *.exe, *.bat, *.sys, etc.) and other such criteria : in file-and—folder-bassed systems is analogous to the functiora of the Core Schema. In the .° Ttem-based hardwares/software interface system, the Core Schema defines a set of core Item types that, directly (by Item type) or indirectly (by Item subtype), characterize all Items into one or more Core Schema Item types which the Item-based hardware/software interface system understands zand can directly process in a predetermi ned and predictable way. The predefined Item types reflect the most common Items in the Item-based hardware/software interface system andl thus a level of efficiency is gained by the Item-based hardware/software= interface system uncerstanding these predefined Item types that comprise the Core Schema.

[0117] In certain embodiments, the Core Schema is not extendable—that is, no "additional Item type s can be subtyped directly from the Iter type in the Base Schema except for the specific predefined derived Item types that are part of the Core Schema. By preventing extensiors to the Core Schema (that is, by preventing the addition of new Items to the Core Schema), the storage platform mandates the use of the Core Schema Item types since every subsequeent Item type is necessarily a subtype of a Core Schema Item type. This structure enables a reasonable degree of flexibility in defini ng additional Item types while also preserving the benefits of having a predefined set of cosre Item types.

[0118] For various embodiments of the present inwention, and in reference to Fig. 8A, the specific Iterma types supported by the Core Schema rmay include one or more of the following: e Categories: Items of this Item Type (and substypes derived therefrom) repre=sent valid Categories in the Item-based hardware/software interface syste m.

. e Commod. ities: Items that are identifiable things of value. - e Devices: Items having a logical structure= that supports information processing y capabiliti_es. e Documents: Items with content that is not interpreted by the Ttem-bwased : hardware=/software interface system but i_s instead interpreted by an application

To program corresponding to the document type. e Events: tems that record certain occurre=nces in the environment. : e Location s: Items representing physical locations (e.g. geographica_1 locations). e Message s: Items of communication between two or more principal s (defined below). e Principals: Items having at least one definitively provable identity aside from an Itemlad (e.g., the identification of a pe=rson, organization, group, household, . authority, service, etc.). e Statements: Items having special inform ation regarding the enviromnment including, without limitation, policies, subscriptions, credentials, a nd so forth.

Likewise, and in refererace to Fig. 8B, the specific prop-erty types supported by thes Core

Schema may include ore or more of the following: : e Certificamtes (derived from the foundational PropertyBase type in tine Base

Schema)» e Principal Identity Keys (derived from thae IdentityKey type in the Base

Schema e Postal A_ddress (derived from the Propexty type in the Base Schem a) . e Rich Texxt (derived from the Property ty pe in the Base Schema) "i e FEAddresss (derived from the Property tygpe in the Base Schema) o IdentitySSecurityPackage (derived from ®khe Relationship type in the= Base

Schema) e RoleOccupancy (derived from the Relationship type in the Base Sechema) e BasicPresence (derived from the Relationship type in the Base Schema)

These Items and Properties are further described by thesir respective properties ses forth in

Figs. 8A and 8B.

’ 5. Relationships

[0119] Relationships are binary relationships where one Item is designated as source and the other Iten as target. The source Item and the target Item are related by the relationship. The source Item generally controls the life-tixme of the relationship. That is, when the source Xtem is deleted, the relationship between the Items is also deleted.

[0120] MRelationships are classified into: Containment and Reference relationshipss.

The containment relationships control the life-time of the target Items, while the reference relationships do mot provide any life-time management semantics. Figure 12 illustrates thes ] manner in which relationships are classified.

[0121] “The Containment relationship types are further classified into Holding an=d

Embedding relationships. When all holding relationships to an Item are removed, the Iter is deleted. A holding relationship controls the life-time of time target through a reference counting mecharism. The embedding relationships enable modeling of compound Items amnd can be thought of as exclusive holding relationships. An Ktem can be a target of one or mamre holding relationships; but an Item can be target of exactly’ one embedding relationship. Ax

Ttem that is a tar_get of an embedding relationship can not be a target of any other holding or embedding relationships.

[0122] Reference relationships do not control thae lifetime of the target Item. They may be dangling - the target Item may not exist. Referen ce relationships can be used to model reference s to Items anywhere in the global Item name space (i.e. including remote data stores). = [0123] Fetching an Item does not automatically~ fetch its relationships. Applicati_ons must explicitly request the relationships of an Item. In adi dition, modifying a relationship does

E not modify the source or the target Item; similarly, addin g a relationship does not affect thhe source/target ITtesm. a) Relationship Declaration : [0124] The explicit relationship types are defined with the following elements: . AA relationship name is specified in the Name attribute. . Relationship type, one of the following: Folding, Embedding, Reference. This is specified in tke Type attribute. . Source and target endpoints. Each endpoi nt specifies a name and the type of the referenced tem.

. The source endpoint field is generally of type ItemID® (not declared) and it mus t reference an Item in the same d_ata store as the relationship ins@ance. } . For Holding and Emb-edding relationships, the target endpoint field must be of types ItemIDReference and it must reference an Item in the same sto-Te as the relationship instance. For Reference relationshipss the target endpoint can be of asny ItemReference type and can reference Items in other stor-age platform data stores. . Optionally one or moe fields of a scalar or Property Base type can be declared.

The=se fields may contain data associated with the relationship. . Relationship instance=s are stored in a global relation=ships table. . Every relationship insstance is uniquely identified by the combination (source

ItermlD, relationship ID). The relaticenship ID is unique within a given source TtemID for all relationships sourced in a given Iter regardless of their type.

[0125] The source Item is ®the owner of the relationship. While an Item designated as Owner controls the life time of thes relationship, the relationship i tself is separate from the

Iterms it relates. The storage platfornn API 322 provides mechanisms for exposing relationships associated with an Iter.

[0126] Bere is an example of a relationship declaration: <Relationship Name="Empl oyment" Base Type="Reference" > <Source Name="Em-ployee" ItemType="Contact. Pe=rson"/> - <Target Name="Empoloyer" ItemType="Contact.Ormeanization”

Reference Type="ItemIDReference"” /> <Property Name="StzartDate" Type="the storage platformTypes.Date ime" /> <Property Name="EmdDate" Type="the storage platformTypes. Date ime" /> <Property Name="0O fice" Type="the storage platformTypes. Date ime" /> </Relationship>

[0127] This is an example of a Reference relationship. Thae relationship can not be cre ated if the person Item that is refeerenced by the source reference does not exist. Also, if the= person Item is deleted, the relati onship instances between the pserson and organization are del eted. However, if the Organization Item is deleted, the relationship is not deleted and it is daragling.

b) Holding Relationship

[0128] Holding relationships are used to model reference count based life-tirme maragement of the target Items. : [0129] An Item can be a source endpoint for zero or more relationships to Itesms. An

Tterm that is not an embedded Item can be a target of in one or more holding relationstaips.

[0130] The target endpoint reference type must be ItemIDReference and it must refe=rence an Item in the same store as the relationship instance.

[0131] Holding relationships enforce lifetime management of the target endygpoint.

The= creation of a holding relationship instance and the Item that it is targeting is an at-omic ope=ration. Additional holding relationship instances can be created that are targeting ®&he same : Iterm. When the last holding relationship instance with a given Item as target endpoin®t is deleeted the target Item is also deleted.

[0132] The types of the endpoint Items specified in the relationship declarat=ion will © germerally be enforced when an instance of the relationship is created. The types of thes endpoint Items can not be changed after the relationship is established. . [0133] Holding relationships play a key role in forming the Item namespace=. They cortain the “Name” property that defines the name of the target Item relative to the seource

Iter. This relative name is unique for all the holding relationships sourced from a gixven Item. “The ordered list of this relative names starting from the root Item to a given Item forrms the _ full name to the Item.

[0134] The holding relationships form a directed acyclic graph (DAG). Wheen a hol ding relationship is created the system ensures that a cycle is not created, thus ensuring _tha_t the Item namespace forms a DAG. :

[0135] While the holding relationship controls the life time of the target Itemm, it does not control the operational consistency of the target endpoint Item. The target It em is operationally independent from the Item that owns it through a holding relationship. -Copy,

Move, Backup and other operations on an Item that is a source of a holding relationship do not: affect the Item that is a target of the same relationship — for example that is, back _ing up a

Folder Item does not automatically backup all the Items in the folder (targets of the

FoRderMember relationship).

[0136] The following is an example of a holding relationship: <Relationship Name="FolderMembers" Base Type="Holding” >

E -42-

<S oource Name="Folder" ItemType=""Base Folder"/> <T =arget Name="ltem" Item Type="Bzase. Item" : } FeferenceType="ItemIDReference="/> </Relatiorn _ship>

[0137] The FolderMembers relationship ermables the concept of a Foldesr as a generic collection of Item.=s. c) Embedding Relation _ships

[0138] F=mbedding relationships model the= concept of exclusive control of the lifetime of the target Item. They enable the concept of compound Items.

[0139] The creation of an embedding relat=ionship instance and the Iterrm that it is targeting is an ato mic operation. An Item can be a s=ource of zero or more embed ding relationship. However, an Item can be a target of orme and only one embedding re=lationship.

An Item that is a t=arget of an embedding relationshi—p can not be a target of a holding relationship.

[0140] The target endpoint reference type must be ItemIDReference an_d it must reference an Item in the same data store as the relatmonship instance.

[0141] The types of the endpoint Items spe=cified in the relationship declaration will generally be enfom-ced when an instance of the relationship is created. The types «of the endpoint Items ca_n not be changed after the relatiormship is established.

[0142] FE=mbedding relationships control tine operational consistency of the target endpoint. For exa—mple the operation of serializing Of an Item may include serialization of all the embedding rel ationships that source from that Item as well as all of their targzets; copying : an Item also copies all its embedded Items.

[0143] T_he following is an example declaration: <Relation=ship Name="ArchiveMembers" B aseType="Embedding” > <S= ource Name="Archive" ItemType="Zip. Archive" /> <I arget Name="Member" ItemType="Base.Item " _ReferenceType="TtemIDReferenc-¢" /> <P®roperty Name="ZipSize" Type="%the storage plaatformTypes.bigint" /> <P=roperty Name="SizeReduction" T_ype="the storage platformTypes.float" /> : </Relatior-xship> d) Reference Relationships

So [0144] The reference relationship does not control life time of the Item it references.

Even more, the reference relationships do not guarantee the existence of the target, nor do they guarantee the type of the target as specified in the relationship declaration. This means “that the reference relationships can be dangling. Also, the reference relationship can reference

Items in other data stores. Reference relationships can be thought of ass a concept similar to links in web pages.

[0145] An example Of reference relationship declaration is th_e following: <Relationship Name=""DocumentAuthor" BaseType="Referenace” > <Sourc ItemType="Document” ItemType="Base.Dociament"/> <Target ItemType="Author" ItemType="Base. Author"

Reference Ty pe="ltemIDReference" /> <Property Type="Role" Type="Core.CategoryRef" /> - <Property Type="DisplayName" Type="the storage : platformTypes .nvarchar(256)" /> ) </Relationship>

[0146] Any reference type is allowed in the target endpoint. “The Items that participate in a reference relationship can be of any Item type. . [0147] Reference relationships are used to model most non-1_ifetime management ‘relationships between Items. Since the existence of the target is not ermforced, the reference relationship is convenient to model loosely-coupled relationships. The= reference relationship can be used to target Items in other data stores including stores on oth- er computers. e) Rules and constraints

[0148] The followin g additional rules and constraints apply for relationships: 1. An Item must be a target of (exactly one embedding re -lationship) or (one or more holding relationships). One exception is the root Item. An Item can be a target of zero or more reference relationships 2. An Item that is a target of embedding relationship can —mot be source of holding . relationships. It can be a source of reference relationships. 3. An Item can not be a source of holding relationship if Mt is promoted from file.

It can be a source of embedding relationships and reference relationshnips.

4. An Item can that is promoted from am file can not be a target of an embedding relationship. f Ordering of Relatio nships

[01497] In at least one embodiment, the storage platform of the present invention : “supports orde ting of relationships. The ordering is achieved through a property rmamed “Order” in thes base relationship definition. There i=s no uniqueness constraint on the Order : field. The order of the relationships with the same “order” property value is not Fuaranteed, howeveritis guaranteed that they may be ordered after relationships with lower “order” value and bef ore relationships with higher “order” field value. [0150 } Applications can get the relationships in the default order by ordering on the combination « SourceltemID, RelationshipID, Ord_er). All relationship instances sourced from a given [tem are ordered as a single collection regzardless of the type of the relat onships in the collectiorm . This however guarantees that all relationships of a given type (e.£.,

FolderMemb -ers) are an ordered subset of the relat=ionship collection for a given Item.

[0150] The data store API 312 for manip-ulating relationships implement a set of operations th at support ordering of relationships. Whe following terms are introclluced to help explain the operations: } RelFZ rst is the first relationship in the orde xed collection with order value OrdFirst;

RelLczst is the last relationship in the ordered collection with order value: OrdLast;

RelX is a given relationship in the collection with order value OrdX; © RelPr-ev is a closest relationship in the coll ection to RelX with order valtae OrdPrev : smaller then OrdX; and

RelNeext is a closest relationship in the coll ection to RelX with order valwie OrdNext greater then «OrdX.

InserzBeforeFirst( SourceltemlD, Relatiore ship )

Inserts the relationship as the first relation=ship in the collection. The val ue of the “Order” property of the new relationship may be smaller then OrdFirst.

Inser-tAfterLast( SourceltemlD, Relationsrmip )

Inserts the relationship as the last relationsship in the collection. The value of the “Qrder” property of the new relationship may be gzreater then OrdLast.

InsertA( SourceltemID, ord, Relationship )

Inserts a relationship with the specified va _lue for the “Order” property.

InsereeBefore( SourceltemID, ord, Relationship )

Inserts the relationship before the relationship with the given order value. The mew ‘relationship mmay be assigned “Order” value that is between OrdPrev and ord, noninclussive.

InserzAfter( SourceliemlID, ord, Relationsizip }

Inser#ts the rel ationship after the relationship with the given order value. The new relationship “may be assigned “Order” value that iss between ord and OrdNext, non-inclwusive.

Move=Before( SourceltemID, ord, RelationsshipID )

Movees the relationship with given relation ship ID before the relationship with specified “Omrder” value. The relationship may be assi gned a new “Order” value that is between OrcIPrev and ord, non-inclusive.

MovaeeAfter( SourceltemlD, ord, RelationstaiplD )

Moves the relationship with given relatiomship ID after the relationship with specified “Order” value. The relationship may be assigned =a new order value that is between ord and

OrdNext, noOn-inclusive. [015 2] As previously mentioned, every Mtem must be a member of an Item Folder.

In terms of MRelationships, every Item must have aa relationship with an Item Folder. Ir several embodiments of the present invention, cexr-tain relationships are represented by -Relationships existing between the Items. : [01553] As implemented for various embodiments of the present invention, a

Relationship provides a directed binary relationship that is “extended” by one Item (thae source) to another Item (the target). A Relationship is owned by the source Item (the tem that extended it), and thus the Relationship is removed if the source is removed (e.g., the

Relationship is deleted when the source Item is deleted). Moreover, in certain instanc=es, a

Relationshigp may share ownership of (co-own) the target Item, and such ownership might be reflected in the IsOwned property (or its equivalent) of the Relationship (as shown in Fig. 7 for the Relaationship property type). In these embwodiments, creation of a new IsOwned

Relationship automatically increments a referenc e count on the target Item, and deletion of such a Rela_tionship may decrement the reference= count on the target Item. For these specific * embodiments, Items continue to exist if they haves a reference count greater than zero, and are automatically deleted if and when the count reacknes zero. Again, an Item Folder is ara Item that has (or is capable of having) a set of Relatiorships to other Items, these other Iterms comprising the membership of the Item Folder. ® ther actual implementations of

- Relationships are possible and anticipated by the present invention to achieve the \ functionality described herein. ; [0154] Regardless of actual implementation, a Relationship is a selectable connection from one awbject to another. The ability for an tem to belong to more than cone

Item Folder, as well ass to one or more Categories, and whether these Items, Folders, amd

Categories are public eor private, is determined by the mezanings given to the existence Cor lack - thereof) in an Item-bassed structure. These logical Relationships are the meanings assigned to a set of Relationships, regardless of physical implementation, which are specifically ‘employed to achieve the functionality described herein. W.ogical Relationships are estamblished between the Item and jts Item Folder(s) or Categories (amd vice versa) because, in esse-nce,

Item Folders and Cate=gories are each a special type of Iteem. Consequently, Item Folders and

Categories can be acte=d upon the same way as any other JTtem—copied, added to an enail message, embedded ir a document, and so and so forth without limitation—and Item Folders and Categories can be= serialized and de-serjalized (impox-ted and exported) using the s=ame mechanisms as for other Items. (For example, in XML. a 1 Items might have a serializzation format, and this formzat applies equally to Item Folders, Categories, and Items.)

[0155] The aforementioned Relationships, which represent the relationship between an Item and it Item Foolder(s) can logically extend from tEne Item to the Item Folder, from the

Item Folder to the Item, or both. A Relationship that log ically extends from an Item to an

Item Folder denotes thhat the Item Folder is public to that Item and shares its memberskip information with that Item; conversely, the lack of a logical Relationship from an Iterra to an * Item Folder denotes that the Item Folder is private to that [tem and does not share its membership informat jon with that Item. Similarly, a Relationship that logically exten ds from an Item Folder to an X'tem denotes that the Item is public and sharable to that Item Folder, whereas the lack of a logical Relationship from the Item Folder to the Item denotes that the

Item is private and ncon-sharable. Consequently, when am Item Folder is exported to amother system, it is the “‘pubMic” Items that are shared in the new context, and when an Item searches its Items Folders for Other, sharable Items, it is the “public” Item Folders that provide the

Item with informatior regarding sharable Items that belosng thereto.

[0156] Fig. "Ois ablock diagram illustrating an Jtem Folder (which, again, is an Item itself), its member Itexms, and the interconnecting Relationships between the Item FolcJer and its member Items. The Item Folder 900 has as members a plurality of Items 902, 904, and

906. Iterm Folder 900 has a Relationship 912 from itself to Item 902 which denote=s that the

Itern 902 is public and sharable to Item Folder 900, its members 904 and 906, andl any other

Item Folders, Categories , or Items (not shown) that mnight access Item Folder 900 . However, there is mo Relationship from Item 902 to the Item Folder 900 which denotes that Item Folder 900 is private to Item 902 and does not share its menmbership information with Itesm 902.

Item 904-, on the other hand, does have a Relationship 924 from itself to Item Foleder 900 which denotes that the Item Folder 900 is public and. shares its membership informmation with

Item 904-. However, there is no Relationship from tte Item Folder 900 to Item 94 which denotes that Item 904 is private and not sharable to tem Folder 900, its other memmbers 902 and 906, and any other [tem Folders, Categories, or XMtems (not shown) that might= access Item

Folder 900. In contrast with its Relationships (or lack thereof) to Items 902 and S04, Item

Folder 9Q0 has a Relationship 916 from itself to the Jtem 906 and Iter 906 has a Relationship 926 back to Item Folder 900, which together denote that Item 906 is public and sharable to

Item Fol der 900, its members 902 and 904, and any -other Item Folders, Categories, or Items (not shown) that might access Item Folder 900, and #hat Item Folder 900 is publiec and shares its membership information with Item 906. ¥ [0157] As previously discussed, the Items iin an Item Folder do not need to share a commonality because Item Folders are not “describezd.” Categories, on the other— hand, are described by a commonality that is common to all of its member Items. Consequently the - membership of a Categoxy is inherently limited to Items having the described commmonality p and, in certain embodiments, all Items meeting the Mescription of a Category are automatically made members of the Category. Thus, whereas Item Folders allow trivial type structures to be represented by their membership, Categories allow membership Bbased on the defined commonality.

[0158] Of course Category descriptions ares logical in nature, and therefore a

Category may be described by any logical representation of types, properties, anad/or values.

For exarmple, a logical representation for a Category~ may be its membership to ceomprise

Items have one of two properties or both. If these deescribed properties for the Category are “A” and “B”, then the Categories membership may ecomprise Items having prope=rty A but not

B, Items having property B but not A, and Items hawing both properties A and B.. This logical representation of properties is described by thhe logical operator “OR” wh ere the set of members described by thie Category are Items havin g property A OR B. Similar logical operands (including without limitation “AND”, “XOR”, and “NOT” alone or in combina_tion) can also be umsed describe a category as will be appreciated by those of skill in the art.

[0159] Despite the distinction between Itexm Folders (not described) and Categomries (described), =Categories Relationship to Items and Items Relationship to Categories essen tially the same wa~y as disclosed herein above for Item Folders and Items in many embodiment=s of the present imvention. [016€0] Fig. 10 is a block diagram illustrating a Category (which, again, is an Item itself), its mesmber Items, and the interconnecting Relationships between the Category amd its member Iterms. The Category 1000 has as members a plurality of Items 1002, 1004, and. 1006, all of =which share some combination of comrnon properties, values, or types 1008 as described (ceommonality description 1008”) by the Category 1000. Category 1000 has a

Relationship 1012 from itself to Item 1002 which denotes that the Item 1002 is public arad sharable to Category 1000, its members 1004 and 1 006, and any other Categories, Item

Folders, or Items (not shown) that might access Category 1000. However, there is no

Relationship from the Item 1002 to the Category 1000 which denotes that Category 1000 is private to Iteem 1002 and does not share its membership information with Item 1002. Ite-m 1004, on the= other hand, does have a Relationship 1024 from itself to Category 1000 wh ich denotes that the Category 1000 is public and shares its membership information with Item : 1004. However, there is no Relationship extended from Category 1000 to the Item 1004 which denotes that Item 1004 is private and not sharable to Category 1000, its other mermbers : 1002 and 10806, and any other Categories, Item Folders, or Items (not shown) that might access Category 1000. In contrast to its Relationships (or lack thereof) with Items 1002 and 1004, Category 1000 has a Relationship 1016 from itself to Item 1006 and Item 1006 ha sa

Relationship> 1026 back to Category 1000, which altogether denotes that Item 1006 is pumblic and sharables to Cate gory 1000, its Item members 1002 and 1004, and any other Categories,

Item Folderss, or Items (not shown) that might access Category 1000, and that the Category 1000 is public and shares its membership information with Item 1006.

[01671] Finally, because Categories and Item Folders are themselves Items, ande

Items may Relationship to each other, Categories may Relationship to Item Folders and vice versa, and Categories, Item Folders, and Items can Relationship to other Categories, Iterm

Folders, andR Item respectively in certain alternative embodiments. However, in various embodiments, Item Folder structures and/or Category structures are prohibited, at the

- hardware/software interface system level, from containing cycles. Where Item Folder and “Category structures are akin to directed graphs, the embodiments that prolibit cycles are akin to directed acyclic graphs (DAGs) which, by mathematical definition in time art of graph theory, are directed graphs wherein no path starts and ends at the same vemtex. 6. Extensibility

[0162] The storage platform is intended to be provided with an initial set of schemas 340, as described above. In addition, however, in at least some embodiments, the storage platform allows customers, including independent software vendor (ISVs), to create new schemas 344 (i.e. new Item and Nested Element types). This section addr—esses the mechanism for creating such schemas by extending the Item types and Nested Element types (or simply “Element” types) defined in the initial set of schemas 340.

[0163] Preferably, extension of the initial set of Item and Nesteca Element types is constrained as follows: © an ISV is allowed to introduce new Item types, i.e. subtype Base.l tem; an ISV is allowed to introduce new Nested Element types, i.e. substype

Base. NestedElement; ’ - an ISV is allowed to introduce new extensions, i.e. subtype Base.INestedElement; but, ) an ISV cannot subtype any types (Item, Nested Element, or Exten_sion types) defined by the initial set of storage platfornn schemas 340. '[0164] Since an Item type or Nested Element type defined by th_e initial set of storage platfort schemas may not exactly match an ISV application’s neeed, it is necessary to allow ISVs to customize the type. This is allowed with the notion of Exte=nsions. Extensions are strongly typed instances but (a) they cannot exist independently and (Cb) they must be attached to an Item or Nested Element.

[0165] In addition to addressing the need for schema extensibili_ty, Extensions are also intended to address the “multi -typing” issue. Since, in some embodiments, the storage platform may not support multiple inheritance or overlapping subtypes, applications can use

Extensions as a way to model overlapping type instances (e.g. Document is a legal document as well a secure document).

a) Item extensions

[0166] To provide Item extensibility, the data model further defines an abstract type named Base Extension. This is a root type for the hierarchy of extension types. Applications can subtype Basse. Extension to create specific extenssion types.

[0167] The Base Extension type is defined in the Base schema as follows: . <Type N ame="Base.Extension” IsAbstract="True"> —<Propety Nlame="itemID"

Type="the storage platform Typses.uniqueidentified* , Nuilable="false"

Multivalued="false"/> —<Property Name="ExtensioniD"

Type="the storage platformTyp es.uniqueidentified”

Nullable="false"

MultivValued="false"/> </Type>

[0168] The ItemID field contains the IterndD of the item that the extension is associated withw. An Item with this ItemID must exisst. The extension can not be created if the item with the g=iven ItemXD does not exist. When the Item is deleted all the extensions with the same ItemTED are deleted. The tuple (ItemID,ExtensionID) uniquely identifies an extension instamce.

[0169] The structure of an extension type is similar to that of an item type:

Extension types have fields; : “Fields can be of primitive or nested element types; and “Extension types can be sub-typed. [01701 The following restrictions apply fo-T extension types

Extensions can not be sources and targets of relationships;

Extension type instances can not exist independently from an item; and

Extension types can not be used as field types in the storage platform type definitieons

[0171] There are no constraints on the typ es of extensions that can be associated with a given Itesm type. Any extension type is allowed to extend any item type. When multiple extenssion instances are attached to an iter, they are independent from each other in both structure zand behavior.

[0172] The extension instances are stored and accessed separately from the item. Ali extension type instances are accessible from a global extension view. An efficient query can

- be composed that will return all the instan ces of a given type of extension reegardless of what typre of item they are associated with. The storage platform APIs provides a- programming ) model that can store, retrieve and modify -extensions on items.

[0173] The extension types can Ee type sub-typed using the storagse platform single inheritance model. Deriving from an extemsion type creates a new extensiox type. The structure or the behavior of an extension cannot override or replace the structure or behaviors of the item type hierarchy.

[0174] Similar to Item types, Ex tension type instances can be dire=ctly accessed © through the view associated with the extension type. The ItemID of the exteension indicates which item they belong to and can be use d to retrieve the corresponding Ite=m object from the global Item view.

[0175] The extensions are consi dered part of the item for the purposes of operational consistency. The Copy/Move , Backup/Restore and other common operations that . thes storage platform defines may operate on the extensions as part of the item. :

. WE 2005/029313 PCT/US2003/026144

[0176] Consider the following example. A Contact type is defined in the Windows

J ype Set. <Type Name="Contact" BaseType=""Base.ltem” > <Property Name="Name" : Type="String" - Nullable="false" : MultivValued="fals e"/> <Property Name="Address" ] i Type="Address"

Nullable="true"

MuttiValued="false'/> </Type>

[0177] A CRM application dev eloper would like to attach a CRM application extension to the contacts stored in the storage platform. The application developer would define a CRM extension that would contain the additional data structure that the application can mmanipulate. <Type Name="CRMExtension" BaseType="Base.Extension” > : ’ <Property Name="CustomerriD"

Type="String"

Nullable="false"

Multi alued="false"/> : </T ype> } : [0178] An HR application developer may want to also attach additional. data with the Ceontact. This data is independent from the CRM application data. Again the application . + devel-oper can create an extension <Type Name="HRExtension" EBaseType="Base.Extension" > <Property Name="Employe eID"

Type="String"

Nullable="false"

Multiv alued="false"/> </Type>

[0179] CRMExtension and HR Extension are two independent extensioms that can be attached to Contact items. They are created and accessed independently of each other.

[0180] In the above example, the fields and methods of the CRMExten sion type cannot override fields or methods of the Contact hierarchy. It should be noted that instances of the= CRMExtension type can be attaclhzed to Item types other than Contact.

: [0181] “When the Contact item is retrieved 5 its item extensions are not automatically retrieved. Given =a Contact item, its related item exte=nsions can be accessed by quexying the global extension view for extensions with the same Jtemld.

[0182] _All CRMExtension extensions in thne system can be accessed throtagh the

CRMExtension type view, regardless of which item they belong to. All item extension of an item share the sa_me item id. In the above example, the Contact item instance and the attached

CRMExtension znd BREx tension instances the sare ItemlID.

[0183] ‘The following table summarizes time similarities and differences between

Item, Extension and NestedElement types:

Item vs Item Extension vs NestedElemeent “Has sown item id Sbmaestheitemid Does not lmaveits of the item own item i d. Nested element is part of the item

Storage Item hierarchy is Ite=m extension Stored with item : stored in its own hierarchy is stored tables in its own tables

Query/Searc_h Can query item Can query item Can generally be tables ex tension tables queried omly within the containing item context

Query/Searc-h Can search across C=an search across Can generally only scope all instances of an all instances of an search within nested item type ite=m extension type element ty pe instances of a singe (containing) item

Relationshipw Can have No Relationships to No Relationships to semantics Relationships to item extensions nested eleaments items

Association sto Can be related to C an generally only Related to item via items other items via bes related via fields. Ne=sted holding, embedded exxtensions. The elements are part of : and soft extension semantics the item . Relationships is similar to exrnbedded item h semantics b) Extending Nested Element types

[0184] Nested Element types are not extended with the same mechanism as the Item types. Extensions of nested elements are stored arid accessed with the same mechanisms as fields o»f nested element types. or [0185] The data model defines a root fox nested element types named EElernent: <Type Name="Element"

IsAbstract="True"> «Property Name="ElementI|D"

Type="the storage platfornnTypes.uniqueidentifier’

Nullable="false"

Multiv alued="false"/> </Type>

[0186] The NestedElement type inherits from this type. The NestedElement element type additionally defines a field that is a multi-se't of Elements. . <Type Name="NestedElement’ BaseType="E3ase.Element"

IsAbstract="True"> <Property Name="Extensions"

Type="Base.Element” : Nullable="false"

Multi alued="true"/> </Type>

[0187] The NestedElement extensions are different from item extensions in the followdAng ways:

Nested element extensions are not extension types. They do not belong —to the ) extension type hierarchy that is rooted in the Bas.e Extension type.

Nested element extensions are stored aloxg with the other fields of the item and are not globally accessible — a query can not be composed that retrieves all instances of a given extension type.

: [#0188] These extensions are storexd the same way as other nested elerments (of the jtem) are= stored. Like other nested sets, thes NestedElement extensions are storeed in a UDT. - They are= accessible through the Extensions field of the nested element type. [#0189] The collection interfaces wised to access multi-valued properti es is also used for accessing and iterating over set of type: extensions. oo [0190] The following table surnmnarizes and compares Item Extensions and

NestedE=lement extensions.

Mtem extensions vs NestedElement extensions “Tier cxionsi on hierarchy is Stored like nested. elements stored in its own tables

Query/Search Can query item extension ~~ Can generally onl ybe : tables queried within the containing item context

Query/Search Can search across all Can generally onl y search sSecope instances of an item within nested elerxent type extension type instances of a singe (containing) item ] P=rogrammability Need special extension NestedElement extensions

APIs and special querying are like any other multi- on extensiom tables valued field of nested element; normal rested element type APY s are used

Behavior Can associa te behavior No behavior permitted (7)

Relationship No Relatioraships to item No Relationships to ssemantics extensions NestedElement extensions

Ltem ID Shares the item id of the Does not have its own item item id. NestedElemerat extension is part «of the item

D. DATABASE ENGINE

[0191] As menwtioned above, the data store is inmplemented on a database enggine. In the present embodiment... the database engine comprises 2 relational database engine t=hat implements the SQL query language, such as the Microseoft SQL Server engine, with object relational extensions. This section describes the mapping of the data model that the lata store implements to the relational store and provides information on the logical API consumed by storage platform clients, in accordance with the present esmbodiment. It is understoo=d, however, that a different mapping may be employed when a different database engin_e is employed. Indeed, in acldition to implementing the storamge platform conceptual data_ model on a relational database engine, it can also be implement=ed on other types of databas-es, e.g. object-oriented and XML databases.

[0192] An object-oriented (OO) database system provides persistence and - transactions for programming language objects (e.g. C+—+, Java). The storage platfor—m notion - of an “item” maps well to an “Object” in object-oriented systerns, though embedded collections would have —to be added to Objects. Other storage platform type concepts. , like inheritance and nested element types, also map object-oriented type systems. Object—oriented systems typically alreaclly support object identity; hence, item identity can be mappead to object identity. The itermn behaviors (operations) map we 11 to object methods. However, object-oriented systems. typically lack organizational cagpoabilities and are poor in sea=rching. .Also, object-oriented systems to do not provide support for unstructured and semi-st-ructured data. To support the cormplete storage platform data mocdel described herein, concepts like relationships, folders, amd extensions would need to be zadded to the object data modiiel. In addition, mechanisms like promotions, synchronization, notifications, and security v=vould need to be implemented.

[0193] Similar to object-oriented systems, XIVIlL databases, based on XSD (XML

Schema Definition), sugpport a single-inheritance based #type system. The item type ssystem of the present invention could be mapped to the XSD type model. XSDs also do not provide support for behaviors. Whe XSDs for items would have wo be augmented with item bwehaviors.

XML databases deal with single XSD documents and laack organization and broad seearch capabilities. As with oboject-oriented databases, to support the data model described herein, other concepts like rela-tionships, and folders would nee d to be incorporated into such XML databases; also, mechanisms like synchronization, notifications ancl security would need to be implemented. . 1. Da-ta Store Implementation Using UDTs

[0194] In the present embodiment, the relational database engine 314, which in one ‘embodiment comprises thme Microsoft SQL Server engine, supports built-in scalar types.

Built-in scalar types are ‘““native” and “simple”. They are native in the sense that the user cannot define their own t=ypes and they are simple in that they cannot encapsulate a complex structure. User-defined t=ypes (hereinafter: UDTs) provide a mech anism for type extensibility above and beyond the native scalar type system by enabling users to extend the type system by defining complex, striactured types. Once defined by a user, a TJIDT can be used anywhere in the type system that a Tbuilt-in scalar type might be used. : [0195] In accorcdance with an aspect of the present inven tion, the storage platform schemas are mapped to \IDT classes in the database engine store. Data store Items are mapped to UDT classes cleriving from the Base.Item type. Like Items, Extensions are also mapped to UDT classes zand make use of inheritance. The root Extension type is

Base Extension, from whaich all Extension types are derived.

[0196] A UDT His a CLR class — it has state (i.e., data fields) and behavior (i.e., routines). UDTs are defi ned using any of the managed languages — C#, VB.NET, etc. UDT methods and operators can be invoked in T-SQL against an instamce of that type. A UDT can be: the type of a column =in a row, the type of a parameter of a routine in T-SQL, or the type of a variable in T-SQL.

[0197] The folleowing example illustrates the basics of UFDTs. Assume that

MapLib.dll has the assembly called MapLib. In this assembly, there’s a class called Point, - under the namespace BasseTypes: namespace BaseTypes : public class Point ( //returns the distamnce from the specified point. public double Dis tance(Point p) // return the di stance between Point p and this Point other stuff in thee class 5B } Bn

~The follow ing T-SQL code binds the class Point to a SQL Server UDT called Po int. The first step invokes “CreateAssembly”, which loads the MMapLib assembly into the datalbase. The second step invokes “Create Type” to create the User Defined Type “Point” and bind it to the managed t=ype BaseTypes.Point :

CREATE ASSSEMBLY MaplLib

FROM \\miywsrvishare\MapLib.dii : go

CREATE TYPE Point

EXTERNAL. NAME ‘BaseTypes.Point’ go

Once created, the “Point” UDT can be used as a column in a table and methods can be invoked im T-SQL as shown below:

Create tabl-e Cities(

Narrm e varchar(20),

States varchar(20),

Locamtion Point) -- Retrieve the Distance of the cities -- from co-Ordinates (32,23) ‘Declare @pp point(32, 23), @distance float

Select Loc=ation::Distance(@p)

From Cities

[07198] The mapping of storage platfornn schemas to UDT classes is fairly straightfomrward at a high level. Generally, a storage platform Schema is mapped to a CLR namespac e. A storage platform Type is mapped to a CLR class. The CLR classs inheritance - mirrors th_e storage platform Type inheritance, ard a storage platform Property ms mapped to a

CLR classs property.

[07199] The Item hierarchy illustrated in. Fig. 29 is used as an example in this document=. It shows the Base.ltem type from which all Item types are derived, &long with 2 set of deri ved Item types (e.g., Contact.Person arad Contact. Employee), with in heritance indicated by arrows. 2. Item Mapping . [0 200] Given the desirability for Items to be globally searchable, and the support in the relational database of the present embodiment for inheritance and type subs- titutability, one possible implementation for Item storage in the database store would be to store all Jtems in a single table with a column of type Base.Itenn. Using type substitutability, I_tems of all types could be stored, ard searches could be filtered by Iter type and sub-type using

Yukon’s “is of (Type) Operator. - [0201] However, due to concerns about the overhe=ad associated with such an approach, in the present embodiment, the Items are divided by top-level type, such that Items of each type “family” are stored in a separate table. Under ®this partitioning scheme, a table is created for each Item type inheriting directly from Base. Jtern. Types inheriting below these are stored in the approparate type family table using type substitutability, as described above.

Only the first level of imheritance from Base.Item is treated specially. For the example Item hierarchy shown in Fig. 29, this results in the following typee family tables: : create table Contact.[TabBe!Personj { _ item Contact.Person rot nuli, {Change tracking inforrmnation} create table Doc.[Table!D»ocument] { " _ltem Doc.Document raot null, " {Change tracking inforrnation}

[0202] A “shadow” table is used to store copies o=f globally searchable properties for all Items. This table may be maintained by the Update() method of the storage platform API, through which all data «changes are made. Unlike the type #amily tables, this global Item table contains only the -top-level scalar properties of the Itemm, not the full UDT Item object.

The structure of the glawbal Item table is as follows: create table Base.[Tablel Hem] ( ltemiD unigueidentifiernot null constraint [PK_Clu_ltermlitem|D] primary key clustered,

TypelD uniqueiden tifier not null, {Additional Properties ©f Base.ltem}, {Change tracking inforamation} ol [0203] The gl obal Item table allows navigation tos the Item object stored in a type family table by exposirag an ItemID and a TypeID. The Ite=mID will generally uniquely "identify the Item within the data store. The TypelD may bee mapped using metadata, which is not described here, to a type name and the view containing. the Item.

[0204] Since finding an Item by its ItemID may bwe a common operation, both in the

N context of the global Item table and otherwise, a Getltem() function is provided to retrieve an

Item object given an Item’s ItemID. This function has thes following declaration:

Base.ltem Base.Getltem (uniqueidentifier ItemliD)

[0205] Fox convenient access and to hide implementation details to the extent possible, all queries: of Items might be against views bui 1t on the Item tables described above.

Specifically, views may be created for each Item type against the appropriate type f=amily table. These type v-iews may select all Items of the associated type, including sub-t=ypes. For convenience, in adclition to the UDT object, the views may expose columns for all of the top- level fields of that type, including inherited fields. Viev=s for the example Item hier-archy shown in Fig. 29 ar as follows: create view Contact. Person as } select _ltem.ltemiD, {Properties of Base.ltem}, {Properties= of Contact.Person}, {Change tracking information}, __Item from Contact.[TabEelPerson] ~Note that the Conta ct. Employee view uses a “where” predicate -- to restrict the set off found Items to instances of Contact. Ermnployee create view Contact. Employee as select _ltem.ltemID, {Properties of Base.ltem}, {Propertiess of Contact.Person}, {Propertiees of

Contact.Employee}, {Change treacking information}, cast (_ltem as Cont=act.Employee) from Contact.[TabBelPerson] where _item is of ((Contact.Employee) create view Doc.Doc ment as select _ltem.ltemiD, {Properties of Base.ltem}, {Properties of Doc.Document}, {Change racking information}, __Item from Doc.[Table!D ocument] --Note that the Doc.VWardDocument view uses a “where” precicate -- to restrict the set oF found Items to Instances of Doc.Word[Document create view Doc.Wor-dDocument as . select_ltem.itemiDD, {Properties of Base.ltem}, {Propertiess of Doc.Document}, {Properties of

Doc.WordDocwmument}, . {Change tracking information}, cast (_Item as Doc.\.WordDocument) from Doc.[Table!lD ocument] where _ltem is of C Doc.WordDocument)

[0206] Fo r completeness, a view may also be Created over the global Item table.

This view may initi ally expose the same columns as the. table: create view Base.lterm as select itemiD, TypeID, {Properties of Base.ltem}, {Change- tracking information} from Base.[Table!l tem]

ol WO 2005/029313 PCT/US2003/026144 3. Extension Mapping

[0207] Extensions ares very similar to Items and have some o-f the same requirements. As another root type supporting inheritance, Extension ss are subject to many of the same considerations and trade-offs in storage. Because of this, a similar type family mapping is applied to Extensions, rather than a single table approach. Of course, in other embodiments, a single table approach could be used.

[0208] In the present embodiment, an Extension is associate d with exactly one Item by ItemID, and contains an Ex tensionID that is unique in the context «of the Item. The

Extension table has the follow ing definition: create table Base.[TablelExtension)] ( ltemID uniqueidentifier not null,

ExtensioniD uniqueidentifier sot null,

TypelD uniqueidentifier not null, {Properties of Base.Extension(}, {Change tracking Information}. constraint [PK_Clu_Extension tltemID!Extension|D] primary key clustered (Item BD asc, ExtensionlD asc) )

[0209] As with Items, a function might be provided to retrie=ve an Extension given its identity, which consists of an ItemID and ExtensionID pair. This function has the following declaration:

Base.Extension Base.GetExtenssion (uniqueidentifier temID, uniqueidentifier ExtensioniD,)

[0210] A View is created for each Extension type, similar to the Item type views.

Assume an Extension hierarchy parallel to the example Item hierarctmy, with the following types: Base. Extension, Contact. PersonExtension, Contact. EmployeeEExtension. The following views may be created: create view Base.Extension as select ItemID, ExtensionID, TwpeiD, {Properties of Base.Extension}, {Change tracking information} from Base.[TablelExtension}] create view Contact.[Extension! Person Extension] as select _Extension.ltem!D, _E>«tension.Extension|D, {Properties of Base.E=xtension, {Properties of

Contact.PersonExtension }, . {Change tracking inforrmation}, _ Extension from Base.[Table!PersonExte nsion] create view Contact.[Extension! EmployeeExtension] as select _Extension.ltem|D, _Ex<tension.ExtensionID, {Properties of Base.E=xtension}, {Properties of

Contact.PersonExtension }, {Properties of Contact. EmployeeExtension}, {Change tracking inf«ormation},

pr cast (_Extension as Contact Employe ©Extension) from ERase.[Table!PersonExtension] where _Extension is of (Contact. EmployeeE>xtension) 4. Nested Element Manpping

W0211] Nested Elements are types that can be embedded in Ite=ms, Extensions,

Relationships, or other Nested Elements to form deeply nested structuress. Like Items and

Extensi=ons, Nested Elements are implemerated as UDT’s, but they are sstored within an Items and Ext-ensions. Therefore, Nested Elemerats have no storage mapping beyond that of their

Item an d Extension containers. In other wosrds, there are no tables in thee system which directly- store instances of NestedElement types, and there are no viewss dedicated specifically to Neste=d Elements. 5. Object Identity :

[0212] Each entity in the data mo del, i.e., each Item, Extension and Relationship, has a unique key value. An Item is uniqu ely identified by its ItemId. An Extension is uniquely identified by a composite key of (ltemld, Extensionld). A Re=lationship is identified by a composite key (Itemld, RelationshipIci). Itemld, Extensionld and Relationshipld are

GUID wvalues. 6. SQL Object Namirmg

[0213] All objects created in the «data store can be stored in a SQL schema name derived from the storage platform schema mame. For example, the storage platform Base schema (often called “Base”) may produce= types in the “[System.Storamge]” SQL schema such as “[Sy stem.Storage].Item”. Generated names are prefixed by a qualifier to eliminate naming conflicks. Where appropriate, an exclamation character (!) is used as a separator for each . logical part of the name. The table below eutlines the naming convent ion used for objects in the data store. Each schema element (Itexn, Extension, Relationship &and View), is listed along with the decorated naming convention used to access instances izn the data store.

Master Item | Master!Item Provides a [Sysstem.Storage].

Search: View summary of items | [M=aster!Item] a item domain.

Typed Item | ItemTypoe Provides all [AcmeCorp.Doc]). ‘| search view property data [OfficeDoc] from item and any parent type(s).

Master Master- Extension Provides a [System.Storage].

Extension summary of all. [Master!Extension]

Search View extensions in tke ’ current item domain.

Typed Extens ion!extensionType | Provides all [AcmeCorp.Doc]. extension property data feor | [Extension!StickyNot-e] search view extension.

Master Masten Relationship Provides a [System. Storage].

Relationship summary of all [Master!Relationship»]

View relationships im the current iterm domain.

Relationship | Relationship!relarionship | Provides all demta [AcmeCorp.Doc]. © | view Name associated witth a | [Relationship !Author-sFrom given relationsship | Document]

View View hsiewName Provides the [AcmeCorp.Doc]. columns/types [View!DocumentTitle=s] based on the schema view definition. 7. Column Naming

[0214] When amapping any object model into a stor-e, the possibility of naming collisions occur due to =additional information stored along with an application object. Im order to avoid naming collisions, all non-type specific columns (columns which do not nap directly to a named Pro-perty in a type declaration) is be pref-ixed with an underscore (_) character. In the present embodiment, underscore (_) characters are disallowed as the beginning character of any identifier property. Further, in owrder to unify naming between “CLR and the data store, all properties of a storage platform types or schema element . (relationship, etc.) showmld have a capitalized first character. 8. Search Views

[0215] Views are provided by the storage platform. for searching stored content. A

SQL. view is provided For each Item and Extension type. Further, views are provided tos support Relationships sand Views (as defined by the Data Model). All SQL views and underlying tables in the storage platform are read-only". Data may be stored or chan_ged using the Updates) method of the storage platform APJ, as described more fully below. : [0216] MEach view explicitly defined in a storage platform schema (defined by the schema designer, and not automatically generated by the storage platform) is accessi ble by . the named SQL wiew [<schema-name>].[ View!<view -name>]. For example, a vievw named ) “BookSales” in t he schema “AcmePublisher.Books’™ would be accessible using the rmame “[AcmePublishe-r Books].[View!BookSales]”. Since the output format of a view is= custom on a per-view ba_sis (defined by an arbitraxy query provided by the party defining the view), the columns are ~directly mapped based on the schema view definition.

[0217] All SQL search views in the storage platform data store use the foll owing ordering convention for columns: 1. 1_ogical “key” column (s) of view result such as Itemld, Elementld,

Relationshipld, —_ 2. Metadata information on type of result such as Typeld. 3 Change tracking columns such as CreateVersion, UpdateVersion, en 4. "ype specific column(s) (Properties of the declared type)

S. Type specific views (family views) also contain an object column whhich returns the object } [0218] Members of each type family are searchable using a series of Item views, with there being one view per Item type in the data store.

Co a) Item . [0219] Each Item search view contains a row for each instance of an Item of the specific type or its subtypes. For example, the view for Document could retum instances of

Document, Leg alDocument and ReviewDocument. Given this example, the Item “views can be conceptualizzed as shown in Fig. 28. (1) Master Item Search View

[0220] Each instance of a storage platform data store defines a special Item view called the Master Item View. This view provides suammary information on each Itesm in the data store. Thame view provides one column per Item type property, a column whic”h described the type of the Item and several columns which are used to provide chan _ge tracking and synchronization information. The master item view is identified in a data store using the name “[Systen. Storage]. [Master!Item]”.

- The storage platform idemntity of the Item - | Typeld Typeld The Typeld of the Item -- identifies the exact type of the Item and can be ussed to retrieve information on the type using a Meta_data catalog. _Rootltemld | Itemid The Itemld of the first ncon-embedded ancestor that controls the lifetime of thhis item. <global Global change tracking Mnformation change tracking> (2» Typed Item Search Views

[0221] Each Item type also has a search view. While simmilar to the root Item view, this view also provides access to the Item object via the “_Item” ¢ olumn. Each typed item search view is identified in a dat a store using the name [schemaNcime].[itemTypeName]. For example [AcmeCorp.Doc].[Offi ceDoc].

The storage platform id_entity of the Item <type change Type change tracking izformation tracking> . <parent props> | <property One column per parent property specific> oo <item props> <property . . specific> One column per exclusive property of this type : CLR type of Itern | CLR object — type of d=eclared Item b) Item Extensions

[0222] All Item Exten sions in a WinFS Store are also accessible using search views . a) Master Extension Search “View

[0223] Each instance ©f a data store defines a special Extension view called the

Master Extension View. This wiew provides summary informatieon on each Extension in the data store. The view has a col umn per Extension property, a co Jumn which describes the type of the Extension and sever-al columns which are used to pro~vide change tracking and synchronization information. Tae master extension view is identified ix a data store using the name “[System.Storage].[Maste-x!Extension]”.

Itemid TtemId The storage platform identity of the Item with which this extension is associated

Extensionld Extens®onld 1d of this extension instarce (GUID) _Typeld Typeld The Typeld of the Extens ion - identifies the exact type of the extension and can be used to retrieve information on the extension using the Metadata : catalog. <global change Global change tracking iraformation tracking> <ext properties> <prope-1ty . (2) Typed Extension Search Views

[0224] Each Extension type also has a search view. While similar to the master extension view, this view also provides access to the Item object via thae Extension column.

Each typed extension search vieew is identified in a data store using thes name [schemaName].[Extension!exte=nsionTypeName). For example [AcmeCorp.Doc].[Extension !OfficeDocEx t].

g WO 2005/029313 PCT/US2003/026144

Itemld Itemld The storage platform identify of the Item with which this extension is associated

Extensionld | ExtensionIdL 1d of this extension instance (GUID) <type change Type change tracking informmation tracking> <parent <property One column per parent progperty props> specific> <ext props> <property . : re he | One column per exclusive gproperty of this type _Extension CLR type of CLR object — type of decla red Extension

Extension instance c) "Nested Elements

[0225] All nested elemesnts are stored within Items, Extensi-ons or Relationships instances. As such, they are acc essed by querying the appropriate Item, Extension, or

Relationship search view. d) Re=lationships

[0226] As discussed ab-ove, Relationships form the fundameental unit of linking between Items in a storage platform data store. (1D) Master Relationship Search Wiew

[0227] Each data store provides a Master Relationship Vievwv. This view provides information on all relationship irstances in the data store. The mastesr relationship view is identified in a data store using tle name “[System.Storage].[Master! Relationship)”.

RelationshipId Relaationshipld The id of the relat=ionship instance " (GUID) _RelTypeld Rel=tionshipTypeld | The RelTypeld ofS the Relationship - identifies the type= of the relationship instance using the= Metadata catalog.

<global change Global chamnge tracking information. tracking>

TargetItemReferenc=e Identity of target endpoint _Relationship Relationship Instance of -the Relationship object for this instance (2) Relationship Instance Search Views

[0228] Each de=clared Relationship also has a search view which returns all instances of the particular relationship. ‘While similar to the raster relationship view, this view also provides name=d columns for each property of the re=lationship data. Each _. relationship instance sea rch view is identified in a data store umsing the name [schemaName].[Relatiomship!relationshipName). For exampple [AcmeCorp.Doc].[RelatGonship!DocumentAuthor].

Tony of ses ncn (eld)

RelationshipId RelationshipId The id of the relationship instance (GUID) <type change Type change tracking information tracking>

TargetltemReferenece Identity of tamrget endpoint <source name> TtemId Named prope=rty of source endpoint identity (alias for ItermnId) <target name> ItemReference or | Named prope=rty of target endpoint identity derived class (alias and casst for TargetltemReference) <rel property> <property One column per property of the relationship specific> definition _Relationship CLR type of

Relationship CLR object — type of declare Relationship instance gE 9. UNpdates

[0229] All views in the storage platform data store amre read-only. In order to create "a new instance of a data_ model element (item, extension or re-lationship), or to update an existing instance, the Pr ocessOperation or ProcessUpdategrarn methods of the storage platform API must be ussed. The ProcessOperation method iss 2 single stored procedures defined by the data store= which consumes an “operation” that details an action to be performed. The Process ®WUpdategram method is a stored procedure which takes an order—ed set of operations, known as an “updategram”, which collectively detail a set of actions to toe performed.

[0230] The operation format is extensible and provi des various operations ov-er the schema elements. Somme common operations include: 1. Item operaticens: a. Createeliern (Creates a new item in the context of an embedding or holding relationship) b. Upda_teltem (updates an existing Item) 2. Relationship operations: a. Creat eRelationship (creates an instance of a reference or holding relationship) b. UpdamteRelationship (updates a relationship instance) c. Delet=eRelationship (removes a relationship instances) 3. Extension ojoerations: a. Creat-cExtension (adds an extension to an existing Item) b. UpdaateExtension (updates an existing exteznsion) c. Deleti-eExtension (deletes an extension) 10. & Change Tracking & Tombstones

[0231] Chang=e tracking and tombstone services ares provided by the data stomre, as discussed more fully beslow. This section provides an outlire of the change tracking information exposed imm a data store. =) Change Tracking

[0232] Each ssearch view provided by the data stores contains columns used teo provide change trackin_g information; the columns are common across all Item, Exten sion and

Relationship views. Storage platform Schema Views, defimed explicitly by schema designers, do not autormatically provide change tracking information — such information is provided indirectly threough the search views on which the view itself is built.

[0233] For each element in the data store, change tracking information is available from two places — the “master” element view and the “typed” element view. For example, change tracking information on the AcmeCorp.Document. Document Item type is available fro 1m the Master Item View “[System.Storage].[Master!Item]” and typed Item search view [A<meCorp.Document].[Document]. (00)) Change Tracking in ‘Master’ Search Views

[0234] Change tracking information in the master search views provides information on the creation and update versions of an element, information on which sync pamtner created the element, which sync partner last updated the element and the vers-ion numbers from each partner for creation and ugodate. Partners in sync relationships (d=escribed below) are identified by partner key. A single UDT object named _ChangeTracking Info of type [System.Storage.Store].Change Tracking Info contains all this information. The type is defined in the System.Storage schema. _Charx geTrackinglnfo is available in all global search vieews for Item, Extension and Relationship. T he type definition of ChangeTrackingImfo is:

B <Type Name=“ChangeTrackingInfo’ BaaseType="Base. NestedElement”> ) <FieldProperty Name=“CreationLoc alTS” Type="SqlTypes.Sqlirat64” ) Nullable="False” /> <FieldProperty Name=“CreatingPar tnerKey”

Type="SqlTypes.SqglInt32” Nullable=“ralse” /> <FieldProperty Name=“CreatingPar-tnerTS”

Type="SglTypes.SqlInt64” MNullable="False” /> <FieldProperty Name=“LastUpdatel.ocalTS”

Type="SqlTypes.SqlInt64~ M™Nullable=”False” /> ’ <FieldProperty Name=*LastUpdatin. gPartnerKey”

Type="SqlTypes.SqlInt32” M™Nullable="False” /> <FieldProperty Name="LastUpdatin gPartnerTS* Type="5qlTypes.SqlIrat64”

Nullable="False” /> </Type>

These properties contain the following information: _CreationlocalTS Creation time stamp by the local machine _CreatingPartnerKey PartnerKey of the partner who created this entity.

If the entity was locally created, this is the local machine’s P artnerKey. . _CreatingPartnerTS Timestamp ©f the time at which this entity was created at th e partner corresponding to _CreatingPauxtnerKey. _LastUpdateLocalTS Local timestamp corresponding to the update time

I LLL

_LastUJpdatingPartnerKey | PartnerKey of the partner who last updated this entity. If the last update to the entity was done oo. Jocally, this is the local machine's PartnerKey. _Last®pdatingPartnerTS | Timestamp of the time at which this entity was

To updated at the partner corresponding to _LastUpdatingPartnerKey. (2) Change Tracking in “Typed” Search Vievvs [02:35] In addition to providing the same information as the global searcl view, each typed searcch view provides additional information: recording the sync state of each element in the sync tospology. <globeal change Information from global tracki ng> change tracking _ChamngeUnitVersions | MultiSet<ChangeUmiitVersion> | Description of version numbers of the cBhange units within the partictalar element _ElermentSyncMetadata | ElementSyncMetad ata Additional versicn- independent metadata about this item that is conly of interest to the : Synchronization runtime. _VerssionSyncMetadata j VersionSyncMetad ata Additional versicon-specific metadata about thhis version that is only of in®erest to the

Synchronization runtime b) Tombstones

[0236] The data store provides tombstone information for Items, Extensions and

Relationsinips. The tombstone views provide information about both live and tomEbstoned entities (iteems, extensions and relationships) in one place. The item and extension tombstone views do rot provide access to the corresponding object, while the relationship tombstone view provides access to the relationship object (the relationship object is NULL imn the case of a tombstomed relationship).

4) Item Tombstonees

[0237] Item tombstones are retrieved from the system via the view [System.Storag=e].[Tombstone!Item].

Tey fe em _Rootltenmld Itemld Ixemld of the first non-embeddingg item which contains this item. _ChangeT rackingInfo | CLR instance of Change tracking information for this item type

ChangeTrackingInfo _IsDelete-d BIT I his is a flag that is 0 for live iterms, and k_ for tombstoned items. _DeletiomWallclock | UTCDATETIME Whe UTC wall clock date time according t-o the partner which deleted the i€em. It is

NULL if the Item is live. : (2) Extension Tormbstones

[0238] Extension tombstones are retrieved from the system using the view [System.Stora_ge].[Tombstone!Extension]. Extensiom change tracking information is similar to that provided for Items with the addition of the Exc tensionld property.

Itemld Ttemld Identity of the Item which owns the

Extension

Typeld Type of the extension : _Change=TrackingInfo | CLR instance of Change tracking information for this type extension

ChangeTrackingInfo _IsDelet ed BIT ‘This is a flag that is 0 for live iteems, and 1 for tombstoned extensions. _DeletiosnWallclock | UTCDATETIME The UTC wall clock date time a<ccording to the partner which deleted the extension. It is NULL if the extension is

~

Co (3) Relationships “Tombstone

[0239] Relationship tombstones are retrieve=d from the system via the vie=w [System.Storage] — [Tombstone !Relationship]. Relaticonships tombstone informatior is similar to that provided feor Extensions. However, additionaal information is provided on t-he target - TtemRef of the reRationship instance. In addition, the relationship object is also selected.

ItemId : ItemId Identity of the Item which owmned the relationship (identity of relationship source endpoint)

Relationship 1d Relationshipid Relationshipld of the relationship

Typeld Xype of the relationship _ChangeTracckinglnfo | CLR instance of Change tracking information £or this type relationship

ChangeTrackinginfo _IsDeleted BIT ~Xhis is a flag that is O for live items, and

ML for tombstoned extensions. : _DeletionWallclock | UTCDATETIME "Hhe UTC wall clock date time= according to the partmer which deleted time relationship. It is NULL if the relationship is live.

Relationship CLR instance of a his is the relationship object for live

Relationship relationship. It is NULL for to-mbstoned relationships.

TargetltemR eference Ldentity of target endpoint (4) Tombstone Cleanup

[0240] Imn order to prevent unbounded growth of tombstone information, ®&he data store provides a tombstone cleanup task. This task determines when tombstone infformation may be discarded— The task computes a bound on tle local create / update versiomn and then truncates the tomiostone information by discarding alll earlier tombstone versions.

11. Helper APIs and Functions a [0241] The Base mapping also provides a number of helper functions. Thes functions are supplied to aid common operations over the data model. ) : a) Function [System.Storag<]. Getltem - Returns an Item object given an ltemld i item Getltem (itermid kemld) - b) Function [System.Storag €]. GetExtension // Returns an extexnsion abject given an ltemId and Extension 1d /

Extension GetExtension (Itemid itemld, Extensionld Extensio nid) c) Function [System.Storagse]. GetRelationship // Returns an relationship object given an ltemld and Relations shipld "

Relationship GetRelationship (Itemid itemld, Relationshipld Relationshipld) 12. Metadata

[0242] There are two types of metadata represesnted in the Store: instance metadata (the type of an Item, etc), and type metadata. a) Schema Metadata

Schema metadata is stored in the data store as instances of Item types from the= Meta -, schema. b) Instance Metadata

Instance metadata is used by an application to query for the type of an Item anad finds the extensions associated with an Item. Given the ItemId for an Item, an application can query the global item view to retum the type of the Item and use this value to query thwe

Meta. Type view” to return information on the declared type of the Item. For example, // Return metadata Item object for given Item instance i

SELECT m._ltem AS metadatainfoObj

FROM [System.S-torage).[ltem] i INNER JOIN [Meta).[Type] mi ON i._Typeld = m.ltem!d

WHERE i.ltemid = @ItemId

E. SECURITY

[0243] This section describes a security model fox the storage platform of the present invention, in accordance with one embodiment 1. ©verview

[0244] In accordance with the present embodimemnt, the granularity at which the security policy of the storage platform is specified and enforced is at the level of various operations on an item in a given data store; there is no ability to secure parts of an item separately from the whole. The security model specifies time set of principals who can be granted or denied access to perform these operations on an item through Access Control List s (ACL’s). Each ACL is an ordered collection of Access Comtrol Entries (ACE’s).

[0245] The se=curity policy for an item can be completely described by the discretionary access control policy and the system access control policy. Each of these is a seat of ACL's. The first set (DACL’s) describes the discretionary access granted to the various principals by the owner of the item while the second set of ACL'’s is referred to as the

SACL’s (System Accesss Control Lists) which specify how the system auditing is done when an object is manipulated in certain ways. In addition to thesse, each item in the data store is associated with a SID that corresponds to the owner of the item (Owner SID).

[0246] The primary mechanism for organizing ite=ms in a storage platform data store is that of the containme=nt hierarchy. The containment hierarchy is realized using holding relationships between items. The holding relationship betw=een two items A and B expressed as “A contains B” enabwles the item A to influence the lifeti me of the item B. Generally, an item in the data store czannot exist until there is 2 holding relationship from another item to jt —

The holding relationshi_p, in addition to controlling the lifetime of the item, provides the necessary mechanism Tor propagating the security policy for an item.

[0247] The se=curity policy specified for each itenm consists of two parts — a part tha& is explicitly specified feor that item and a part that is inherit«ed from the parent of the item in ] the data store. The expRicitly defined security policy for an—y item consists of two parts — a ~- part that governs access to the item under consideration andl a part that influences the security policy inherited by all i_ts descendants in the containment hierarchy. The security policy

WE) 2005/029313 P&CT/US2003/026144 _inheritexd by a descendant is a function of thee explicitly defined policy and “the inherited policy.

[0248] Since the security policy is propagated through holding reRationships and can also be= overridden at any item, it is necessary to specify how the effective security policy for an item is determined. In the present embo diment, an itern in the data store containment hierarc=hy inherits an ACL along every path from the root of the store to thes item.

[0249] Within the inherited ACL f or any given path, the ordering of the various

ACE’s in the ACL determines the final secumrity policy that is enforced. Thee following _ notatiosn is used to describe the ordering of 2ACE’s in an ACL. The orderin.g of the ACE’s in "an ACL that is inherited by an item is deterrmined by the following two rules —

[0250] The first rule stratifies the #ACEs inherited from the various items in a path to the iterm I from the root of the containment Inierarchy. The ACE’s inherited from a closer contairer takes precedence over the entries inherited from a distant contain_er. Intuitively, this allows an administrator the ability to overricle ACE's inherited from farthear up in the contairament hierarchy. The rule is as followws:

For all inherited ACL’s L on item I

For all items 11, I2

For all ACE's Al and A2inL, 11 is an ancestor of 12 and 12 is an ancestor of I3- and ’ Al is an ACE inherited from 11 and

A2 is an ACE inherited from 12

Implies

A2 precedes Alin LL

[0251] The second rule orders the .ACE’s that deny access to an item ahead of the

ACE’s that grant access to an item.

For all inherited ACL’s L on item I

For all items 11 : For all ACE’s Al and A2inL, 11 is an ancestor of I2 and

Al is an ACCESS_DXENIED_ACE inherited from IL and

A2 is an ACCESS_GIRANTED_ACE inherited fromm I1 . Implies

A Al precedes A2in LL

[0252] In the case of a containment® hierarchy being a tree, there is exactly one path from th_e root of the tree to the item and the i tem has exactly one inherited ACL. Under these circumstamnces, the ACL inherited by an item rmatches the ACL inherited by a £ile (item) in the existirmg Windows security model in terms of the relative ordering of the A=CE’s within them.

[0253] However, the containment hierarchy in the data store is a directed acyclic graph (DAG) because multiple holding relationships are permitted to items. Umnder these conditionss, there are multiple paths to an item from the root of the containmen t hierarchy.

Since an i tem inherits an ACL along every path each item is associated with a collection of

ACL’s as opposed to a single one. Note that this is different from the traditioral file system model, where exactly one ACL is associated with a file or folder. [0-254] There are two aspects that need to be elaborated when the cormtainment hierarchy is a DAG as opposed to a tree. A description is needed of how the e-ffective security p olicy for an item is computed when it inherits more than one ACL from its parents, and how they are organized and represented h. as a direct bearing on the administration of the security nmodel for a storage platform data stoxre.

[0255] The following algorithm eval uates the access rights for a give=n principal to a _ given iter. Throughout this document, the following notation is used to describe the ACL's " associate] with an item. : Inherited ACLs(ltemld) — the set of A.CL’s inherited by an item whose item ide=ntity is ItemlId from it’s parents in the store. i. Explicit ACL(temld) - the ACL explicitly defined for the item whose identity is Itemld.

N_ISTATUS

ACCLAccessCheck(

PSID pOwnerSid,

PDACL pDacl,

DWORD Desired Access, } HANDLE ClientToken, ‘ PPRIVILEGE_SET pPrivilegeSet, a. DWORD *pGrantesdAccess)

[02256] The above routine returns ST ATUS_SUCCESS if the desired_ access was not - explicitly denied, and the pGrantedAccess det ermines which of the rights desimred by the user were gran ted by the specified ACL. If any of the desired access was explicitly denied, the routine returns STATUS_ACCESS_DENIED . i -78-

NTSTATUS

WinFSItemAccessChecXk(

WINES_ITEMIED Itemld,

DWORD Desired Access,

HANDLE ClientToken,

PPRIVILEGE_SSET pPrivilegeSet) {

NTSTATUS S-tatus;

PDACL pExplicitACL = NULL;

PDACL pInherited ACLs = NULL;

DWORD NEumberOfInherited ACLs = 0; pExplicitACL = GetExplicitACLForltem(ItemMd);

Getlnherited ACL sForltem(ItemlId,&pInherited ~ACLs,&NumberOfInheriteed AC

Ls) :

Status = ACLAc=cessCheck( p®OwnerSid, p-ExplicitACL,

Desired Access,

CZlientToken, pPrivilegeSet, 8=GCrantedAccess); if (Status '= STATUS_SUCCESS) return Staatus; if (DesiredAccesss == Granted Access) return ST"ATUS_SUCCESS; for ( i=0; (i < NumberOfIraherited ACLs && Status == STIATUS_SUCCESS) ; i+) {

GrantedAccessF orACL = 0;

Status = ACLAccessCheck( pOwnerSid, pExplicitACL,

DesiredAccess,

ClientToken, pPrivilegeSet, &GrantedAccessIForACLY); if (Status == STATUS_SUCCESSS) { } GrantedAccess |= Grante-dAccessForACL; } }

If (Status == ST ATUS_SUCCESS) && (GrantedAcceess != DesiredAccess)) {

Status = STATUS_ACCESS_DENIED; } return Status;

[0257] The sphere of influe=nce of the security policy defi ned at any item covers all the descendants of the item in the cosntainment hierarchy defined o-n the data store. For all items where in an explicit policy is Clefined we are in effect definirg a policy that is inhemrited by all its descendants in the containr ent hierarchy. The effective #ACL’s inherited by all of the descendants is obtained by takin_g each of the ACL's inherited by the item and adding the inheritable ACE's in the explicit ACL to the beginning of the ACI. This is referred to a sthe set of inheritable ACL’s associated =with the item.

[0258] In the absence of ary explicit specification of secwurity in the containmemnt hierarchy rooted at a folder item, the security specification of the Folder applies to all thes descendants of that item in the containment hierarchy. Thus, ever—y item for which an ex_plicit security policy specification is provided, defines a region of identi cally protected items, and the effective ACL’s for all the itemss in the region is the set of inhe=ritable ACL's for that item.

This would completely define the re=gions in the case of a containrmnent hierarchy that is & tree.

If each region were to be associated with a number, then it would “be sufficient to merely~ include the region to which an item belongs along with the item.

R [0259] However, for contaminment hierarchies that are D/Gs, the points in the éontainment hierarchy at which the effective security policy chang=es is determined by tvw=vo kinds of items. The first is items fo-r which an explicit ACL has be=en specified. Typically these are the points in the containme=nt hierarchy where in the administrator has explicitly + specified an ACL. The second is items that have more than one p=arent, and the parents Bhave different security policies associate] with them. Typically, these a_re the items that are thme confluence points of security policy specified for the volume and mi ndicate the beginning of a new security policy.

[0260] With this definitior, all the items in the data store= fall into one of two categories — those that are the root Of an identically protected security region and those that ; . are not. The items that do not define security regions belong to ex =actly one security regi=on.

As in the case of trees, the effective security for an item can be speecified by specifying the region to which an item belongs alo=ng with the item. This leads tc a straight forward meodel for administering the security of a storage platform data store base=d upon the various } identically protected regions in the sstore.

2. Detailed Description of the Security Model [026 1) This section provide details of how items are secured by describing h=ow the individual rights within a Security Descriptor and its contained ACL’s affect various operations. a) Security Descriptor structure [026-2] Before describing the details of the security model, a basic discussiorn of security desecriptors is helpful. A security descriptor contains the security information. associated vaith a securable object. A security descriptor consists of a

SECURITY _DESCRIPTOR structure and its associated security information. A secumrity descriptor caan include the following security information: © R. SID’s for the owner and primary group of an object. 2. A DACL that specifies the access rights allowed or denied to particula=r users Or groups. 3. A SACL that specifies the types of access attempts that generate audit records for the object. 21. A set of control bits that qualify the meaning of a security descriptor omr } its individual members.

[02623] Preferably, applications are not ables to directly manipulate the conterts of a " security descriptor. There are functions for setting and retrieving the security informamtion in an object's security descriptor. In addition, there are functions for creating and initiali—ing a security descriptor for a new object.

[02684] A discretionary access control list IPACL) identifies the trustees tha® are allowed or clenied access to a securable object. When a process tries to access a securaable object, the s-ystem checks the ACEs in the object's D ACL to determine whether to gra=nt access to it. If the object does not have a DACL, the system grants full access to eversyone. If the object's “DACL has no ACEs, the system denies all attempts to access the object because the DACL Joes not allow any access rights. The system checks the ACEs in sequence= until it finds one or- more ACEs that allow all the requested access ri ghts, or until any of the requested access rights are denied. : [026=5] A system access control list (SACL) enables administrators to log attempts to access a secured object. Each ACE specifies the types of access attempts by a specified trustee that ecause the system to generate a record in the security event log. An ACE irm a

S- ACL can generate audit records when an access attempt fails, when it succeeds, or both. A . SACL may also raise an alarm when an unauthorized user attempts to gain 2access to an o-bject.

[0266] All types of ACEs contain the following access control information: 1. A security identifier (SID) that identifies the trustee to which. the ACE applies. 2. An access mask that specifies the access rights controlled by the ACE. 3. A flag that indicates the type of ACE. 4. A set of bit flags that deteramine whether child containers or Objects can inherit the ACE from the primary object to which the ACL iss attached.

[0267] The following table lists the three ACE types supported by all securable

Objects. #"ccess-denied Used in a DACL to deny access rights to a trustee.

ACE

Aocess-allowed | ye in 9 DACL to allow access rights to a trustee.

ACE

System-audit Used in a SACL. to generate an audit record when the trusstee attempts to

ACE exercise the specified access rights. i @ Access Mask Format

[0268] All securable objects arrange their access rights using the amccess mask format shown in the Fig. 26. In this format, the low-order 16 bits are for obmject-specific amccess rights, the next 7 bits are for standard access rights, which apply to most types of

Objects, and the 4 high-order bits are used to specify generic access rights that each object type can map to a set of standard and object-specific rights. The

ACCESS_SYSTEM_SECURITY bit corresponds to the right to access the object’s SACL. ©) Gen eric Access Rights

[0269] Generic rights are specified in the 4 high-order bits within ~the mask. Each "type of securable object maps these bits to a set of its standard and object-spoecific access rights. For example, a file object maps thee GENERIC_READ bit to the RE~AD_CONTROL and SYNCHRONIZE standard amccess rights and to the FILE REA_D_DATA,

FILE_READ_EA., and FILE_ RESAD_ATTRIBUTES object-specifTic access rights. Other types of objects map the GENEIRIC_READ bit to whatever set of =access rights is appropriate - for that type of object.

[0270] Generic access rights can be used to specify the ty=pe of access needed wheen opening a handle to an object. T his is typically simpler than specif —ying all the correspondimg standard and specific rights. The following table shows the constants defined for the genemic access rights. (3) Standard Access Rights

[0271] Each type of se=curable object has a set of access rights that correspond to operations specific to that type of object. In addition to these object-specific access rights, . there is a set of standard access rights that correspond to operationms common to most typess of securable objects. The followin gg table shows the constants defined for the standard access- rights. [REFERS SSSR

SYNCHRONIZE | wait until the object is in the signaled state. Some object types do no-t support th is access right.

b) Item Specific Rights

[0272] Imm the access mask structure of Fig. 26, item specific rights are placeciin the

Object Specific Ri. ghts section (low order 16-bits). Beccause in the present embodiment, the storage platform exposes two sets of APIs to administe=r security — Win32 and the stomrage .platform APJ, the file system object specific rights musst be considered in order to motivate the design of the storage platform object specific rights. (1) File and Directosry object specific rights : Consider the following table:

Description Description

FILE_LIST_ Right to list | FILE_READ_ Righttoread | 0x0001

DIRECTORY the contents | DATA the of the corresponding ) directory file data

FILE_ADD_FILEE | Rightto FILE_WRITE _ Right to write | 0x0002 create a file | DATA data to the file in the

I directo

FILE_ADD_ Right to FILE_APPEND_ Right to 0x0004

SUBDIRECTORY | create a DATA append data to subdirecto the file

FILE_READ_EA Right to FILE READ_EA | Righttoread | 0x0008 read extended file ’ extended attributes : file attributes

FILE_WRITE_EA\ | Right to FILE_WRITE_EAA | Right to write | 0x0010 write extended file extended attributes file attributes

FILE_TRAVERS_E | Right to FILE _EXECUTE- | For anative 0x0020 traverse the code file, the directory. right to execute the file.

FILE_DELETE_ Right to None None 0x0040

CHILD delete a directory and all the files it contains

ATTREBUTES read ATTRIBUTES file attributes directory attributes

FILE_"WRITE _ Right to FILE_WRIXE_ Right to write | 0x0M00 al ill directory attributes

[0273] Referring to the foregoing table, note that file systems make a fundamental distinc tion between files and directories, which i s why the file and directory rigshts overlap on the sarme bits. File systems define very granular rights, allowing applications t=o control behavi or on these objects. For instance they allow applications to distinguish among

Attribtates (FILE READ/WRITE_ATTRIBUTE-S), Extended Attributes and thee DATA stream associated with the file.

[0274] A goal of the security model of the storage platform of the pre=sent invention is to simplify the rights assignment model so applications operating on data store items (Contacts, Emails, etc.) generally do not have a reed to distinguish between attributes, extend ed attributes and data streams, for exampl«e. However, for files and folde=rs, the * granular Win32 rights are preserved and the sem antics of access via the storage platform are "defined so that compatibility with Win32 applications can be provided. This m_apping is discussed with each of the item rights specified telow.

[0275] The following item rights are spmecified with their associated allowable . operations. The equivalent Win32 rights backings each of these item rights is also provided. 2 WinFSIternRead

[0276] This right allows read access to all elements of the item, incluading the items “linked to the item via embedded relationships. It also allows enumeration of iteems linked to . this item via holding relationships (a.k.a., directory listing). This includes the mmames of items linked via reference relationships. This right ma ps to: i File: : (FILE_READ_DATA | SYNCHRONIZE)

Folder: (FILE_LIST_DIRECTORY | SYNCHRONIZE)

[0277] The semantics are that a securit=y application could set WinFSMtemReadData and specify the rights mask as a combination of €he file rights specified above.

3) WinFSItemReadAttributes

[0278] This right allowss read access to basic attributes of the= Item, much as file systerns distinguish between basi < file attributes and data streams. Pre=ferably, these basic attributes are those that reside in “the base item that all items derive from. This right maps to:

File: (FILE_READ_ATTRIBUJTES)

Folder: (FILE_READ_ATTRIBUJITES) @ WinFSItemWriteAttributes :

[0279] This right allows write access to basic attributes of time Item, much as file systems distinguish between basi. c file attributes and data streams. Preferably, these basic attributes reside in the base item “that all items derive from. This righ maps to: :

File: (FILE_WRITE_ATTRIB UTES)

Folder: . (FILE_WRITE_ATTRIB UTES) : 3) WinFSItemWrite

[0280] This right allows the ability to write to all elements owf the item, including ,items linked via embedded relationships. This right also allows the ability to add or delete embedded relationships to other Atems. This right maps to:

File: (FILE_WRITE_DATA)

Folder: (FILE_ADD_FILE)

[0281] In the storage pl atform data store, there is no distinct—on between items and folders, since items can also haves holding Relationships to other item.=s in the data store.

Hence, if you have FILE_ADD_ SUBDIRECTORY (or FILE_APPEIND_DATA) rights, you . :can have an item be the source off Relationships to other items.

”~ (6) WinFSItemAddLink

[0282] This right allows the ability to add Folding Relationships to itexrxs in the store. It should be noted that since the security mode=] for multiple holding Relat® onships changes the security on an item and the changes can bypasses WRITE_DAC if c=oming from a higher point i m the hierarchy, WRITE_DAC is required on the destination itercm in order to be able to create a Relationship to it. This right mapes to:

File: : (FILE__APPEND_DATA) oC } ’ Folder: (FILE__ADD_SUBDIRECTORY) (7) WinFSltemDeeleteLink [02831 This right allows the ability to deletze a holding to an item even If the right to delete that itera is not granted to the principal. This &s consistent with the file sysstem model and helps with purge. This right maps to:

File: (FILE_DELETE_CHILD) — Note that file systems do not have a file equa ivalent to this right, but ve have the notion of items having hoelding Relationships to other=s and hence carry this right for non-folders as well.

Folder: (FILE_"DELETE_CHILD) (8) Rights to dele te an item

[0284] An item gets deleted if the last holding Relationship to the item. disappears. 3 There is no explicit notion of deleting an item. Theres is a purge operation which deletes all . holding Relationships to an item, but that is a highex= level facility and not a syste=m primitive.

[0285] Any item specified using a path carm be unlinked if either one of™ two } conditions is satisfied: (1) the parent item along that: path grants write access to the subject, or © the standarcl rights on the item itself grant DELFEZTE. When the last Relation. ship is removed, the item disappears from the system. Any- item specified using the Ite mmID can be : unlinked if the standard rights on the item itself grarat DELETE.

WED 2005/029313 PCT/US2003/(26144 ] (£))] Rights to copy an item ; ) [0286] An item can be copied from a source to a destination folder if the subj ect is grantec WinFSItemRead on the item and WinFSItemWrite on the destination folder. : (10) Rights to move an item - [0287] Move file in the file system requires just the DELETE right on the sovarce file ancl FILE_ADD_FILE on the destination directory, since it preserves the ACL on #the destinamtion. However, a flag can be specified in the MoveFileEx call (MOV_EFILE_COPY_ALLOWED) that lets an application specify that it in the case o—fa cross-v>olume move, it can tolerate CopyFile semantics. There are 4 potential choices —with respect= to what happens with the security descriptor upon a move:

L Carry the entire ACL. with the file - default intra-volume move semantics. :

Co 2. Carry the entire ACL with the file and mark the ACL as protected. 3. Carry just the explicit ACEs across and re-inherit on the destination. 4. Carry nothing and re-inherit on the destination — default inter-volume move semantics — same as copy file. . [0288] In the present security model, if an application specifies the

MOVESFILE _COPY_ALLOWED flag, the fourth option is performed for both the inte=r- and intra-veolume cases. If this flag is not specified, the second option is performed unless the destination is also in the same security region (i.e., same inheritance semantics). A stomrage platform level move implements the fourth choice as well and requires READ_DATA. on the source, much as a copy would. : . (11) Rights to view the security policy on an item

[0289] An item's security can be viewed if the item grants the standard right

READ __CONTROL to the subject. } (12) Rights to change the security policy on an ' item . [0290] An item’s security can be changed if the item grants the standard right

WRITEE_DAC to the subject. However, since the data store provides implicit inheritan ce, this has implications on how security can be changed on hierarchies. The rule is that if the root of the hierarchy grants WRITE_DAC, then the security policy is changed on the entire hierarchy re_gardless of whether specific items wit hin the hierarchy (or DAG) donot grant

WP RITE_DAC to the subject. (13) Rights that don’t have a direct equivalent

[0291] In the present embodiment, FILE_EXECUTE (FILE_TRAAVERSE for di. rectories) do not have a direct equival ent in the storage platform. The model keeps these for

W~in32 compatibility, but does not have any access decisions made for iterms based on these rigehts. As for FILE_READ/WRITE_E.A, because data store items do not Ehave notions of extended attributes, semantics for this brit are not provided. However, the bit remains for

Woin32 compatibility. 3. Implementatiom

[0292] All the items that defime identically protected regions hav € an entry . asssociated with them in a security table . The security table is defined as follows:

Identity | Ordpath

[0293] The Item Identity entry, is the Item Identity of the root of =an identically protected security region. The Item Orcdpath entry is the ordpath associate=d with the root of thme identically protected security regiora. The Explicit Item ACL entry is the explicit ACL “de=fined for the root of the identically protected security region. In some cases this can be

NIULL, e.g., when a new security regiom is defined because the item has mul tiple parents belonging to different regions. The Path ACLs entry is the set of ACL’s inherited by the iteem, and the Region ACLs entry is the set of ACL’s defined for the identically protected security region associated with the itera.

BN [0294] The computation of effective security for any item in a gi ven store leverages this table. In order to determine the security policy associated with an iter, the security re=gion associated with the item is obtaimed and the ACL’s associated with_ th.at region are retrieved.

[0295] As the security policy associated with an item is changed either by directly aclding explicit ACL's or indirectly by adding holding Relationships that mes ult in the formation of new security regions, the security table is kept up to date to esnsure that the akibove algorithm for determining the effective security of an item is valid.

[029€5] The various changes to the store and the accompanying algorithms to "maintain the security table are as follows: a) Creating a new ite=m in a container

[0297] When an item is newly created im a container, it inherits all the ACL °s associated w=ith the container. Since the newly created item has exactly one parent it Belongs to the same mregion as its parent. Thus there is no meed to create a new entry in the security . table. b) Adding an explicit ACL to an item.

[02928] When an ACL is added to an item, it defines a new security region for all its descendants in the containment hierarchy that beBong to the same security region as the given item itself. Foor all the items that belong to other security regions but are descendants of the given item imn the containment hierarchy, the secumrity region remains unchanged but the effective ACL associated with the region is chan goed to reflect the addition of the neve ACL.

[02979] The introduction of this new security region can trigger further region definitions for all those items which have multiple holding Relationships with ancestors that .straddle the old security region and the newly defined security region. For all such items a : pew security region needs to be defined and the procedure repeated. oo [030-0] Figures 27(a), (b), and (c) depict a new identically protected securitsy region being carve out of an existing security region by introducing a new explicit ACL. T his is "indicated by~ the node marked 2. However, the introduction of this new region results in an additional region 3 being created because of an item having multiple holding Relationships. [030 1] The following sequence of upda tes to the security tables reflect the Factoring of the identi cally protected security regions. ©) Adding a holding Relationship to an item } [030-2] When a holding Relationship is added to an item it gives rise to one of three possibilities . If the target of the holding Relationship, i.e., the item under consideration is the : root of a seczurity region, the effective ACL associated with the region is changed anad no further modifications to the security table is required. If the security region of the sowirce of . the new holding Relationship is identical to the s ecurity region of the existing parents of the item no chamnges are required. However, if the ite=m now has parents that belong to di fferent security regmons, then a new security region is foxmed with the given item as the root of the security region. This change iis propagated to all the items in the containment hierarchy by modifying the security region associated with the item. All the items that belong to the same security region as the item undler consideration and its descend-ants in the containment hierarchy need to be changed. Once the change is made, all the= items that have multiple holding Relationships must be examined to determine if furthemr changes are required. Furtiner changes may be required if any of these items have parents of cHifferent security regions. d) Deleting a holding Relationship from an item

[0303] When a holdimng Relationship is deleted from aan item it is possible to collapse a security region with its parent region if certain conditions are satisfied. More . precisely this can be accomplished under the following conditieons: (1) if the removal of the holding Relationship results inn an item that has one parent and no explicit ACL is specified for that item; (2) if the removal of the holding Relationship resTults in an item whose parent ’s are all in the same security region and no explicit ACL is defin ed for that item. Under thes-e circumstances the security regi on can be marked to be the same as the parent. This marking needs to be applied to all the items whose security region corre=sponds to the region being collapsed. e) Deleting an explicit ACL from an item ) [0304] When an explicit ACL is deleted from an item, it is possible to collapse the security region rooted at that item with that of its parents. More precisely, this can be done if the removal of the explicit ACL results in an item whose paren_ts in the containment hierarchy belong to the same security reggion. Under these circumstances, the security region can be marked to be the same as the parent and the change applied to =ll the items whose security region corresponds to the region being collapsed. f) Modifying an ACL associated with an item

[0305] In this scenari ©, no new additions to the securi_ty table are required. The effective ACL associated with the region is updated and the ne—w ACL change is propagate«l to the security regions that are affected by it.

F. NOTIFICATIONS AND CHANGE TRACKEING

[0306] According to nother aspect of the present inveention, the storage platform provides a notifications capability that allows applications to tr-ack data changes. This featwre is primarily intended for applications which maintain volatile staate or execute business logi-c on data change events. Applications register for notifications orn items, item extensions aned item relationships. Notifications are delivered asynchronously a—fter data changes have beer committed. Applications may filter notifications by item, exten=sion and relationship type 2s well as type of operation.

[0307] According to one embodiment, the storage platform API 322 provides two kinds of interfaces for motifications. First, applications register for simple data change evermis triggered by changes to items, item extensions and item relation_ships. Second, applicationss . create “watcher” objects to monitor sets of items, item extensiomns and relationships bet wee=n items. The state of a watcher object can be saved and re-created after a system failure or af ter a system has gone off-Line for an extended period of time. A sirgle notification may reflect “multiple updates. 1. Storage Change Events

[0308] This section provide a few examples of how thee notification interfaces provided by the storage platform API 322 are used.

C a) Events . [0309] Items, ItemExtensions and ItemRelationships e=xpose data change events "which are used by appl ications to register for data change notifi-cations. The following cocle sample shows the definition of the ItemModified and ItemRemoved event handlers on the "base Item class. 1 Events public event itemx ModifiedEventHandler Item ItemModified;

Co public event itera RemovedEventHandler [tem_ItemRemowred;

[0310] All notifications carry sufficient data to retrieves the changed item from the data store. The following code sample shows how to register foarevents on an Item,

ItemExtension, or ItenmRelationship: myltem.itemModified += new itemModifiedEventHandler(thls.nitemUpdate); myltem.itemRemoved -+= new HemRemovedEventHandler(this.-onltemDelete);

[0311] Inthe present embodiment, the storage platforrm guarantees that applications will be notified if the respective item has been modified or delesed since last delivering a notification or in case Of a new registration since last fetched from the data store.

b) Watchers [£0312] In the present embodime=nt, the storage platform defines watcher classes for monitoring objects associated with a (1) Eolder or folder hierarchy, (2) ar item context or (3) a specific item. For each of the three cate gories, the storage platform proevides specific watcher classes which monitor associate items, item extensions or item_ relationships, e.g. the stor=age platform provides the respects ve FolderItemWatcher, FolderRelationship Watcher and FolcderExtensionWatcher classes.

E0313] When creating a watcher, an application may request notifications for pre- existing items, i.e. items, extensions or re=lationships. This option is mos#&ly for applications which nmaintain a private item cache. If neot requested, applications receiwe notifications for all upda_tes which occur after the watcher object has been created.

[0314] Together with deliverings notifications, the storage platform supplies a “Watche=rState” object. The WatcherState= can be serialized and saved om disk. The watcher - state car subsequently be used to recreate the respective watcher after a_ failure or when reconneecting after going off-line. The newly re-created watcher will re-g=enerate un- acknow Medged notifications. Applicationss indicate delivery of a notification by calling the “Exclude” method on the respective watcher state supplying a reference to a notification. [C0315] The storage platform del ivers separate copies of the watcher state to each event hamndler. Watcher states received orm subsequent invocations of the same event handler - presume= delivery of all previously received notifications. : [0316] By way of example, the following code sample shows thhe definition of a . Folderlt-=em Watcher. public class FolderttemWatcher : Watcher // Constructors pum blic FolderltemWatcher Constructor(Folder folder); put blic FolderltemWatcher Constructor] (Folder folder, Type itemType); pui blic FolderltemWatcher Constructor2(itemContext context, itemid folderld); pu: blic FolderltemWatcher Constructor3(Folder folder, Type itemType2,

FomlderitemW atcherOptions options); pu blic FolderltemWatcher Constructord(ltemContext context, temid folderld, Type te mType); pu blic FoldertemWatcher Constructor5(itemContext context, Itemid folderld, Type ite=mType, FolderlitemWatcherOptions eoptions); // FPProperties pu blic kemld FolderItemW atcher Foslderld {get;}

public Type Fol derltemWatcher ItemT ype {get;} public FolderitemmWatcherOptions FolderltemWatcher Options {get;} . Il Events : public event ltermChangedEventHandler FolderftemW aatcher ItemChanged; ;

[0317] The following code sample shows how to create a folder watcher ob_ject for monitoring the conte=nts of a folder. The watcher generat es notifications, i.e. events, “when new music items are added or existing music items are updated or deleted. Folder watchers either monitor a particular folder or all folders within a folder hierarchy. myFolderitemWatcher- = new FolderltemWatcher(myFolder, twpeof(Music)); myFolderitemWatcher—.ltemChanged += new ltemChanged EveentHandler(this.onltemChange-d); 2. Change Tracking and Notification Generation Mechanism. . [0318] The= storage platform provides a simple, yet efficient mechanism to track data changes and gemnerate notifications. A client retrieves notifications on the same connection used to reetrieve data. This greatly simplifies s-ecurity checks, removes latencies and constraints on possible network configurations. Notifications are retrieved by iss uing . select statements. To prevent polling, clients may use a “~waitfor” feature provided byw the database engine 314 . Figure 13 shows the basic storage pelatform notification concept. This * waitfor query may b « executed synchronously, in which «case the calling thread is blcacked until results are avai lable, or asynchronously, in which case the thread is not blocked and results are returned On a separate thread, when available.

[0319] A ceombination of “waitfor” and “select’™ is attractive for monitoring data changes which fit into a particular data range as changes can be monitored by setting_ a notification lock on the respective data range. This holds for many common storage platform scenarios. Changes to individual items can be efficiently monitored by setting notific=atiora locks on the respecti ve data range. Changes to folders arad folder trees can be monitored by setting notification leocks on path ranges. Changes to type=s and its subtypes can be meonitored by setting notificaticen locks on type ranges.

[0320] In g=eneral, there are three distinct phasess associated with processingz notifications: (1) data change or even detection, (2) subscription matching and (3) notification - delivery. Excluding synchronous notification delivery, i.e. notification delivery as pa_1t of the transaction performing the data change, the storage platform can implement tva/o forms of notification delivery: 1) Immediate Event Detection: Event detection and subscription. matching is performed as part of the update transaction. Notifications are immserted into a table monitored by the subscriber; and 2) Deferred Event Detection: Event detection and subscription natching is performed after the update transaction has been committed. Sulbsequently the actual subscriber or an intermediary detects events and generates notifications.

[0321] Immediate event detection requires additional code to be executed as part of update operations. This allows the capture of all events of interest including ewents indicating a relative state change.

[0322] Deferred event detection removes the need to add additional code to update operations. Event detection is done by the ultimate subscriber. Deferred event detection naturally batches event detection and event delivery and fits well with the quemry execution infrastructure of the database engine 314 (e.g., SQL Server). :

[0323] Deferred event detection relies on a log or trace left by update operations.

The storage platform maintains a set of logical timestamps together with tomb- stones for deleted data items. When scanning the data store for changes, clients supply a timestamp : which defines a low watermark for detecting changes and a set of timestamps —to prevent duplicate notifications. Applications might receive notifications for all change -s which " happened after the time indicated by the low watermark.

[0324] Sophisticated applications with access to core views can furthmer optimize and reduce the number of SQL statements necessary to monitor a potentially large set of items by creating private parameter and duplicate filter tables. Applications with specia 1 needs such as those having to support rich views can use the available change tracking framework to monitor data changes and refresh their private snapshots.

[0325] Preferably, therefore, in one embodiment, the storage platformmn implements a deferred event detection approach, as described more fully below. a). Change Tracking

[0326] All items, extensions and item relationship definitions carry a unique identifier. Change tracking maintains a set of logical timestamps to record cre=ation, update

. and deletion times for all data items. Tombstone entries are used to represent dele=ted data items. . [0327] Applications use that information to efficiently monitor whether a particular item, item exten sion or item relationship has been newly added, updated or delete=d since the application last zaccessed the data store. The followi ng example illustrates this m-echanism. "create table [item —extension-relationship-table-template] « identifier uniqumeidentifier not null default newid() created bigint.. not null, -- @ @dbts when created updated bigint,, not null, -- @ @dbts when last updated )

[0328] All deleted items, item extensions amnd relationships are recorded in a corresponding tombstone table. A template is showr below. create table [item —extension-relationship-tombstone tablex-template} ( identifier uniqumeidentifier not null, deleted bigint not null, -- @ @dbts when deleted, created higint not null, -- @ @dbts when creamted upated bigint not null, -- @ @dbts when last updated )

[0329] For efficiency reasons, the storage golatform maintains a set of gl _obal tables for items, item e=xtensions, relationships and pathnarxes. Those global lookup tab-les can be used by applications to efficiently monitor data rangses and retrieve associated tinmnestamp and type informatio. b) Timestamp Manager ment :

[0330] Logical timestamps are “local” to a_ database store, i.e. storage p_latform ~ volume. Timestz=amps are monotonically increasing €54-bit values. Retaining a singgle timestamp is oft en sufficient to detect whether a dataa change occurred after last cconnecting to a storage platfor-m volume. However, in most realistzic scenarios, a few more timestamps need to be kept to che=ck for duplicates. The reasons are explained below.

[0331] Relational database tables are logic al abstractions built on top off a set of physical data structures, i.e. B-Tree’s, heaps etc. Assigning a timestamp to a new=ly created or updated record i_s not an atomic action. Inserting tha t record into the underlying Gata structures may Iaappen at different times, thus applications may see records out o f order.

[0332] Figure 14 shows two transactions both inserting a new recor-dinto the same © E3-Tree. Since transaction T3 inserts its record before transaction T2’s insert is scheduled, an application scanning the B-Tree may see the records inserted by transaction _T3 before those inserted by T2. Thus, the reader may iracorrectly assume that he has seen all records created rap to time “10”. To resolve this issue, the database engine 314 provides a function which returns a low water mark up to which all updates have committed and been irserted into the respective underlying data structures. In the example above, the low watermaark returned would be “5,” assuming that the reader started before transaction T2 had beemn committed.

Khe low watermark provided by the database engine 314 allows applications to efficiently determine which items to ignore when scanning the database or a data range for data changes.

I'm general, ACID transactions are assured to last a very short time thus, low watermarks are e=xpected to be very close to the most recently dispensed timestamp. In the pr=esence of long lasting transactions, applications might have to keep individual timestamps to detect and dlliscard duplicates. c) Data Change Detection - Event Detection

[0333] When querying the data store, applications obtain a low watermark.

SS ubsequently, applications use that watermark to scan the data store for entrie=s whose c reation, update or delete timestamp is greater than the low watermark return ed. Figure 15 illustrates this process.

A [0334] To prevent duplicate notifications, applications remember tirnestamps which a—xe greater than the low watermark retiexrned and use those to filter out duplicates.

Applications create session local temporary tables to efficiently handle a largse set of .d_uplicate timestamps. Before issuing a select statement, an application insertss all duplicate ti_mestamps previously returned and del etes those which are older than the last low watermark returned, as illustrated below. . d-elete from $duplicates where ts < @oldLowWaterMark; irmsert into $duplicates(ts) values(...),...( ..); ywaitfor({ select *, getLowWaterMark() as newLowWateriark from [globailltems} where updated >= @oldLowW aterMark and updated not In (select * from $dwiplicates))

rd

G. SYNCHRONIZATION

[0335] According to another aspect of the pre sent invention, the storage pla~tform provides a synchronization service 330 that (i) allows rmultiple instances of the storagze platform (each with its own data store 302) to synchrorize parts of their content acco=rding to _a flexible set of rules, and (ii) provides an infrastructur=e for third parties to synchronize the data store of the storage platform of the present invention with with other data sources that implement proprietary protocols.

[0336] Storage platform-to-storage platform ssynchronization occurs amongs a group of participating replicas. For example, with reference ~to Figure 3, it may be desirabl-eto provide synchronization between the data store 302 of the storage platform 300 with another remote data store 338 under the control of another instance of the storage platform, perhaps running on a different computer system. The total meembership of this group is not necessarily” known to any given replica at any given timme.

[0337] Different replicas can make the changes independently (i.e. concurrently).

The process of synchronization is defined as making es very replica aware of the charmges made by other resplicas. This synchronization capability is fi nherently multi-master.

[0338] The synchronization capability of the- present invention allows replicas to: - . determine which changes another replica is aware of; . . request information about changes that. this replica is not aware of; * . convey information about changes that: the other replica is not aware of; . determine when two changes are in comflict with each other; . . apply changes locally;

LI convey conflict resolutions to other replicas to ensure convergence; &and . resolve the conflicts based on specifiecd policies for conflict resolutiomns. 1. Storage Platform-to-Storage “Platform Synchronization

[0339] The primary application of the syncharonization service 330 of the storage platform of the present invention is to synchronize multiple instances of the storage= platform (each with its own data store). The synchronization sservice operates at the level of the storage pl atform schemas (rather than the underlying tables of the database engine 2314).

Thus, for example, “Scopes” are used to define synchmronization sets as discussed below.

[0340] The synchronization service operatess on the principle of “net chan_ges”.

Rather than recording and sending individual operaticons (such as with transactional replication), the synchronization service sends the end-result of those Operations, thus often consolidating the results of multiprle operations into a single resulting change.

[0341] The synchronization service does not in general respe ct transaction boundaries. In other words, if two changes are made to a storage platform data store in a single transaction, there is no guarantee that these changes are applied at all other replicas atomically --- one may show up without the other. The exception to this principle is that if two changes are made to the same= Item in the same transaction, then these changes are — guaranteed to be sent and applied to other replicas atomically. Thus, Items are the consistency units of the synchroni zation service. a) Synchronization (Sync) Controlling A_pplications ) [0342] Any application can connect to the synchronization sesrvice and initiate a sync operation. Such an applicatieon provides all of the parameters needed to perform synchronization (see sync profile ®elow). Such applications are referred to herein as Sync

Controlling Applications (SCAS).

[0343] When synchronizing two storage platform instances, sync is initiated on one : side by an SCA. That SCA inforrms the local synchronization service to synchronize with the remote partner. On the other side, the synchronization service is awolken by the messages sent by the synchronization service from the originating machine. It responds based on the persistent configuration informati«n (see mappings below) present on the destination machine. The synchronization sexvice can be run on schedule or in re sponse to events. In’ these cases, the synchronization service implementing the schedule bexcomes the SCA. ) [0344] To enable synchr-onization, two steps need to be taker. First, the schema = “designer must annotate the storages platform schema with appropriate sync semantics (designating Change Units as described below). Second, synchronization must be properly configured on all of the machines having an instance of the storage platform that is to participate in the synchronization (as described below). b) Schema annotation

[0345] A fundamental concept of the synchronization services is that of a Change

Unit. A Change Unit is a smallest piece of schema that is individually tracked by the storage platform. For every Change Unit, the synchronization service may be: able to determine ‘whether it changed or did not chamge since the last sync.

[0346] Designating Change Units in the schema serves several purposes. First, it determines how chatty time synchronization service is on the wire. When a change is made inside a Change Unit, thes entire Change Unit is sent to the other replicas, since the synchronization service does not know which part of the Change Unit was changed. Second, it determines the granula rity of conflict detection. When two coracurrent changes (these terms - are defined in detail in sLabsequent sections) are made to the same change unit, the synchronization service maises a conflict; on the other hand, if comcurrent changes are made to different change units, timen no conflict is raised and the changes are automatically merged.

Third, it strongly affects the amount of meta-data kept by the system. Much of the "synchronization service meta-data is kept per-Change Unit; thus, making Change Units . smaller increases the ovesthead of sync.

[0347] Definingg Change Units requires finding the right trade-offs. For that reason, the synchronization service allows schema designers to participate in the process.

[0348] In one eembodiment, the synchronization service does not support Change

Units that are larger tharm an element. However, it does support t he ability for schema designers to specify sma ller change units than an element --- naneely, grouping multiple attributes of an element ®nto a separate Change Unit. In that embodiment, this is "accomplished using the &ollowing syntax: <Type Name="Appointmert' MajorVersion="1" MinorVersion="0" Extends Type="Base.ltem"

ExtendsVersi=on="1"> <Field Name="Meeting Status” Type="the storage platformTypes.umiqueidentifier ’ ’ Nullable="False"/> <Field Name="OrganizeName" Type="the storage platformTypes.rwvarchar(512)"

Nullable="False"/> <Field Name="OrganizeerEmail* Type="the storage platformTypes.xvarchar(512)"

TypeMajorVersion=""1" MuiltiValued="True"/> ‘ <ChangeUnit Name="CSU_Status”> ’ <Field Name="MeetingStatus”/> </ChangeUnit> <ChangeUnit Name="CCU_QOrganizer'/> <Field Name="Org anizerName" /> <Field Name="0rg anizerEmail” /> </ChangeUnit> </Type>

©) Sync Configuration oo [0349] A grooup of storage platform partners that wish to keep certain parts off their data in sync are referred to as a sync community. While tthe members of the communi ty want to stay in sync, they dilo not necessarily represent the data #in exactly the same way; in Other words, sync partners mmay transform the data they are synchronizing. " [0350] In a goeer-to-peer scenario, it is impractical for peers to maintain transformation mappi_ ngs for all of their partners. Instead, the synchronization service takes the approach of defin-ing “Community Folders”. A commmunity folder is an abstractiora that represents a hypothetsical “shared folder” that all commun ity members are synchronizing with.

[0351] This notion is best illustrated by an example. If Joe wants to keep My

Documents folders of his several computers in sync, Joe CJefines a community folder called, say, JoesDocuments. Then, on every computer, Joe configures a mapping between thme hypothetical JoesDoc- uments folder and the local My Documents folder. From this point on, when Joe’s computerss synchronize with each other, they talk in terms of documents irm

JoesDocuments, rather than their local items. This way, amll Joe's computers understarmd each “other without having to know who the others are -— the Ceommunity Folder becomes tie lingua franca of the s—ync community.

[0352] Configuring the synchronization service consists of three steps: (1) defining mappings between loecal folders and community folders; C2) defining sync profiles that determine what gets ssynchronized (e.g. whom to sync with and which subsets should ®be sent and which received); and (3) defining the schedules on whhich different sync profiles should _ run, or running them manually. (6) Community Foldex— - Mappings

[0353] Commmunity Folder mappings are stored sas XML configuration files ©n - individual machines. Each mapping has the following schema: /mappings/communit—yFolder

This element names the community folder that thi s mapping is for. The name follows . the syntax rulees of Folders. * /mappings/localFoldeer } This element names the local folder that the mapp ing transforms into. The narme follows the sy=ntax rules of Folders. The folder mest already exist for the mapping to be valid. The items within this folder are considexred for synchronization per thhis mapping.

/mappings/transformcetions

This element clefines how to transform items from t he community folder to the local folder and back. If absent or empty, no transformat ions are performed. In particular, this means thamt no IDs are mapped. This configuration is primarily useful for creating iE a cache of a Felder. /mappings/transformecations/maplDs ~ This element xequests that newly generated local IDs be assigned to all of the items mapped from the community folder, rather than reu sing community IDs. The Syrac . Runtime will maintain ID mappings to convert itenms back and forth. /mappings/transformecations/localRoot

This element mrequests that all root items in the commmunity folder be made children of the specified root. /mappings/runAs

This element controls under whose authority requessts against this mapping are processed. If absent, sender is assumed. /mappings/runAs/sencier

The presence «of this element indicates that the send_er of messages to this mappings must be impersonatecd, and requests processed under his credentials. : (2) Profiles

[0354] A Syme Profile is a total set of parameters needed to kick off synchronization. It iss supplied by an SCA to the Sync Runtime to initiate sync. Sync profiles for storage platform-t-o-storage platform synchronization contain the following information: . Local Folder, to serve as the source and des®ination for changes; . Remote Folder name to synchronize with - Chis Folder must be published from the remote partner by way of a mapping as defined above; . Directdon - the synchronization service supgports send-only, receive-only, and send-receive sync;

B . Local Filter — selects what local informatiomn to send to the remote partner. - Expressed as a the storage platform query over the local folder; ny id Remote Filter — selects what remote information to retrieve from the remowte partner - expressed ass a storage platform query over the commmunity folder; . Transformations -— defines how to transfora—n items to and from the local format; . Local security --- specifies whether the charges retrieved from the remote ’ endpoint are to be appolied under the permissions of the remote endpoint (impersonated) or the user initiating the sync locally; and

. Conflict resolution policy -— specifies whether conflicts should be rejected,

Rogged, or automatically resolved] - in the latter case, it specifies whieh conflict resolver to mse, as well as the configuration parameters for it.

[0355] The synchronizaation service provides a runtime CLIR class that allows simple

Building of Sync Profiles. Profil es can also be serialized to and froma XML files for easy sstorage (often alongside scheduless). However, there is no standard place in the storage platform where all the profiles are stored; SCAs are welcome to construct a profile on the =spot without ever persisting it. Mote that there is no need to have a local mapping to initiate ssync. All sync information can toe specified in the profile. The mapping is, however, mequired in order to respond to sync requests initiated by the remote side. (3® Schedules : [0356] In one embodiment, the synchronization service does not provide its own ] sscheduling infrastructure. Insteamd, it relies on another component to peform this task - the “Windows Scheduler available with the Microsoft Windows operatin g system. The ssynchronization service includes a command-line utility that acts as an SCA and triggers ssynchronization based on a sync profile saved in an XML file. This utility makes it very easy - mo configure the Windows Scheduler to run synchronization either om schedule, or in response mo events such as user logon or logoff. : d) Conflict Handling : [0357] Conflict handlirag in the synchronization service is clivided into three stages: «1) conflict detection, which occurs at change application time - this step determines if a echange can be safely applied; (20) automatic conflict resolution and logging - during this step

Bb «(that takes place immediately after the conflict is detected) automatic conflict resolvers are consulted to see if the conflict can be resolved - if not, the conflict can be optionally logged; ~ =and (3) conflict inspection and resolution - this step takes place if some conflicts have been 2 Jogged, and occurs outside of thes context of the sync session - at this time, logged conflicts «can be resolved and removed froem the log. oo (1D Conflict Detection

N [0358] In the present embodiment, the synchronization service detects two types of «conflicts: knowledge-based and econstraint-based. (a) Knowledge-based con flicts

[0359] A knowledge-based conflict occurs when two replicas make independent changes to the samse Change Unit. Two changes are called independent if they are made without knowledge= of each other —- in other words, the ver-sion of the first is not covered by the knowledge of thhe second and vice versa. The synchronization service automatically “detects all such corflicts based on the replicas’ knowledge as described above.

[0360] It -is sometimes helpful to think of conflicts as forks in the version history of a change unit. If neo conflicts occur in the life of a change wanit, its version history is a sinciple chain --- each charm ge occurring after the previous one. In tthe case of a knowledge-based conflict, two changes occur in parallel, causing the chain to split and become a version tree. (b) Constraint-b sased conflicts

[0361] There are cases where independent changes violate an integrity constraimnt when applied toget-her. For instance, two replicas creating a file with the same name in the same directory coumld cause such a conflict to occur.

[0362] A constraint-based conflict involves two imdependent changes (just like a knowledge-based one), but they do not affect the same change unit. Rather, they affect different change umits but with a constraint existing between them.

[0363] TIne synchronization service detects constaraint violations at change application time ard raises constraint-based conflicts autormatically. Resolving constrairat- ‘based conflicts usuvsally requires custom code that modifies the changes in such as way ass to not violate the con straint; The synchronization service doe s not provide a general-purpose _- mechanism for doi ng so. 2) Conflict Processing : [0364] When a conflict is detected, the synchroni zation service can take one of three actions (selected b y the sync initiator in the Sync Profile): (1) reject the change, returnin_g it back to sender; (2)» log a conflict into a conflict log; or (3) xesolve the conflict automatically.

[0365] If the change is rejected, the synchronizatXon service acts as if the change did not arrive at the replica. A negative acknowledgement is sent back to the originator. Th is resolution policy iss primarily useful on head-less replicas (such as file servers) where logging conflicts is not fea_sible. Instead, such replicas force the others to deal with the conflicts by rejecting them.

[0366] S-ync initiators configure conflict resolution in their Sync Profiles. The synchronization service supports combining multiple conflict resolvers in a single profilee in the following ways — first, by specifying a list of conflict resolvers to be tried orme after another, until one of them succeeds; and second, by associating conflict resolver=s with conflict types, e.g. directing update-upd ate knowledge-based conflicts to one resolver, but all the other conflicts to the log. (a) Automatic Conflict resolution

[0367] The synchronization service provides a number of default conflict resolvers.

This list includes: . local-wins: disregard incoming changes if in conflict with locallyw stored data; . remote-wins: disregard 1ocal data if in conflict with incoming chaanges; . last-writer-wins: pick either local-wins or remote-wins per chang=e unit based on the timestamp of the change (note that the synchronization service in general does not rely on clock values; this conflict resolver is the sole exception to that rule); . Deterministic: pick a winner in a manner that is guaranteed to be= the same on all replicas, but not otherwise meaningful — one embodiment of the synchronizamtion services uses lexicographic comparisons of partner IDs to implement this feature. . [0368] In addition, ISVs can implement and install their own conflict mesolvers. - Custom conflict resolvers may accept configuration parameters; such parameter—s must be specified by the SCA. in the Conflict Resolution section of the Sync Profile.

[0369] When a conflict resolver handles a conflict, it returns the list of operations that need to be performed (in lieu of the conflicting change) back to the runtime=. The a. synchronization service then applies these operations, having properly adjusted remote knowledge to include what the conflict handler has considered.

[0370] It is possible that another conflict is detected while applying th eresolution.

In such a case, the new conflict must be resolved before the original processing resumes.

B [0371] When thinking of conflicts as branches in the version history o fan item, - conflict resolutions can be viewed as joins -—— combining two branches to form asingle point.

Thus, conflict resolutions tum version histories into DAGs. (b) Conflict Logging

[0372] A very particular kind of a conflict resolver is the Conflict Logeger. The synchronization service logs conflicts as Items of type ConflictRecord. These r-ecords are related back to the items that are in conflict (unless the items themselves have toeen deleted).

Each conflict record contains: the incorning change that caused the conflict; thee type of the conflict: update-update, update-delete, delete-update, insert-insert, or constraint; ancl the version of the incoming change and the knowledge of the replica sending it.

Logged conflicts are available £or inspection and resolution as describexd below.

© Conflict insp»ection and resolution

[0373] Thee synchronization service provides an API for applications to examinee the conflict log and to suggest resolutions of the conflicts in it. The API allows application =o * "enumerate all conflActs, or conflicts related to a given Item . It also allows such applicaticons .to resolve logged conflicts in one of three ways: (1) remote wins --- accepting the logged cliange and overwra ting the conflicting local change; (2) local wins --- ignoring conflictirg parts of the logged «change; and (3) suggest new change —— where the application propose=s a merge that, in its opinion, resolves the conflict. Once conflicts are resolved by an application, the sym chronization service removes them frommn the log. (d) Convergence of replicas and

Propagation of Conflict Resolutions

[0374] In complex synchronization scenarios, thes same conflict can be detected at multiple replicas. Xf this occurs, several things can happera: (1) the conflict can be resolved on one replica, and the resolution be sent to the other; (2) &the conflict is resolved on botlm replicas automatica 1ly; or (3) the conflict is resolved on boath replicas manually (through the conflict inspection _API). ‘ [0375] To» ensure convergence, the synchronization service forwards conflict resolutions to other- replicas. When a change that resolves a conflict arrives at a replica, the synchronization sex-vice automatically finds any conflict reecords in the log that are resolved by this update and eliminates them. In this sense, a conflict resolution at one replica is binding on all the other replicas.

[0376] If different winners are chosen by differemt replicas for the same conflic=t, the "synchronization sexrvice applies the principle of binding conflict resolution and picks ones of the two resolutions to win over the other automatically. T he winner is picked in a . deterministic fashion that is guaranteed to produce the same results at all times (one embodiment uses r«eplica ID lexicographic comparisons).

[0377] If different “new changes” are suggested by different replicas for the sa—me conflict, the synchronization service treats this new confli«ct as a special conflict and uses=s the

Conflict Logger to prevent it from propagating to other regplicas. Such situation commomnly arises with manual conflict resolution.

2. Synchronizing to non-storage platform data sstores : [0378] According to another aspect of the storage platform o-f the present invention, the storage platform provides an architecture for ISVs to implement S=ync Adapters that allow the storage platform to synchronize to legacy systems such as Microsoft Exchange, AD,

Hotmail, etc. Sync Adapters benefit from the many Sync Service prowided by the synchronization service, as described below.

[0379] Despite the name, Sync Adapters do not need to be inziplemented as plug-ins into some storage platform architecture. If desired, a “sync adapter” ¢ an simply be any application that utilizes the synchronization service runtime interfaces to obtain services such as change enumeration and application. " [0380] In order to make it simpler for others to configure anc run synchronization to a given backend, Sync Adapter writers are encouraged to expose the standard Sync Adapter interface, which runs sync given the Sync Profile as described above. The profile provides configuration information to the adapter, some of which adapters pass to the Sync Runtime to control runtime services (e.g. the Folder to synchronize). a) Sync Services a © [0381] The synchronization service provides a number of syrc services to adapter writers. For the rest of this section, it is convenient to refer to the mac=hine on which the storage platform is doing synchronization as the “client” and the non-sstorage platform . ‘backend that the adapter is talking to as the “server”.

A) Change Enumeration a [0382] Based on the change-tracking data maintained by the synchronization service, Change Enumeration allows sync adapters to easily enumerates the changes that have occurred to a data store Folder since the last time synchronization witli this partner was "attempted.

[0383] Changes are enumerated based on the concept of an “ anchor” -- an opaque structure that represents information about the last synchronization. T he anchor takes the form of the storage platform Knowledge, as described in the proceedirag sections. Sync adapters utilizing change enumeration services fall into two broad cate=gories: those using “stored anchors” vs. those using “supplied anchors”.

[0384] The distinction is based on where the information about the last sync is stored --- On the client, or on the server. It is often easier for adapters to store this "informatio m on the client —- the backend is often not capable of conveniently storing thi=s - information. On the other hand, if multiple clients synchronize to the same backend, storing this information on the client is inefficient and in so-me cases incorrect — it makes one client unaware of the changes that the other client has alre-ady pushed up to the server. If an adapter wants to usse a server-stored anchor, the adapter needs to supply it back to the storage . platform at the time of change enumeration.

[0385] In order for the storage platform to maintain the anchor (either for locall or remote sto rage), the storage platform needs to be maade aware of the changes that were successfully applied at the server. These and only these changes can be included in the anchor. D uring change enumeration, Sync Adapters use an Acknowledgement interface to report which changes were successfully applied. A the end of synchronization, adapter=s using suppelied anchors must read the new anchor (vwhich incorporates all of the success#ully- applied changes) and send it to their backend.

[0386] Often, Adapters need to store adap®er-specific data along with the items they insert into the storage platform data store. Common examples of such data are remote I Ds - and remote versions (timestamps). The synchronization service provides a mechanism for storing this data, and Change Enumeration provides a mechanism to receive this extra d_ata along with. the changes being returned. This eliminates the need for adapters to re-quer=y the database ir most cases. @) Change Appl ication

[0387] Change Application allows Sync Adapters to apply changes received from their backend to the local storage platform. Adaptexrs are expected to transform the charges to the storage platform schema.

[0388] The primary function of change application is to automatically detect conflicts. As in the case of Storage Platform-to-Sto-rage Platform sync, a conflict is defsned as two ove=rlapping changes being made without knowledge of each other. When adapters use Chang e Application, they must specify the anchmor with respect to which conflict detection i s performed. Change Application raises aa conflict if an overlapping local chzange that is not covered by the adapter’s knowledge is de=tected. Similar to Change Enumeration, adapters may use either stored or supplied anchors. Change Application supports efficient storage of adapter—specific meta-data. Such data may Eoe attached by the adapterto the changes being appmlied, and might be stored by the synchronization service. The data might be returned on nexxt change enumeration. 3) Conflict Resolutzion

[0389] T he Conflict Resolution mechanisms =described above (logging and automatic resolution options) are available to sync adapters as well. Sync adapters meay specify the policy for conflict resolution when applying changes. If specified, conflicts may be passed on to thee specified conflict handler and resok ved (if possible). Conflicts cam also be logged. It is possi. ble that the adapter may detect a commflict when attempting to apply alocal change to the backend. In such a case, the adapter may still pass the conflict on to the Sync

Runtime to be resolved according to policy. In additiomn, Sync Adapters may request that any conflicts detected by the synchronization service be semnt back to them for processing. This is particularly conve mient in the case where the backend ms capable of storing or resolving conflicts. b) Adapter implementatiosn

[0390] While some “adapters” are simply applications utilizing runtime interfaces, adapters are encoiaraged to implement the standard adempter interfaces. These interfac es allow

Sync Controlling Applications to: request that the adapter perform synchronization according to a given Sync Profile; cancel on-going synchronizati-on; and receive progress reporting (percentage compete) on an ongoing sync. 3. Security

[0391] T he synchronization service strives to- introduce as little as possible x nto the "security model implemented by the storage platform. MRather than defining new rights for synchronization, e=xisting rights are used. Specifically. . anyone who can read a data store Item can enumerate changes to that xtem; . anysone who can write to a data store Ite=m can apply changes to that item; and . anyone who can extend a data store Iter, can associate sync meta-data with - that item.

[0392] T he synchronization service does not “maintain secure authorship information. Whe=n a change is made at replica A by u:ser U and forwarded to replica_ B, the fact that the chang=e was originally made at A (or by UD) is lost. If B forwards this chagnge to replica C, this is doone under B’s authority, not that of A. This leads to the following limitation: if a repl.ica is not trusted to make its own clhanges to an item, it cannot forvward : changes made by others. " [0393] WW” hen the synchronization service is Hnitiated, it is done by a Sync

Controlling Appliczation. The synchronization services impersonates the identity of thes SCA and performs all ogoerations (both locally and remotelsy) under that identity. To illustrzate, observe that user TJ cannot cause the local synchronizaation service to retrieve changes from a remote storage platform for items that user U does no® have read access. 4. Manageability

[0394] Monitoring a distributed community «of replicas is a complex problen. The synchronization service may use a “sweep” algorithm to collect and distribute information about the status of the replicas. The properties of the ssweep algorithm ensure that information about aall configured replicas is eventually= collected and that failing (non- responsive) replica s are detected.

[0395] Tlmis community-wide monitoring infeormation is made available at exsery replica. Monitoringg tools can be run at an arbitrarily-chosen replica to examine this monitoring information and make administrative decissions. ‘Any configuration changes must be made directly at the affected replicas.

H. TRADITIONAL FILE SYSTEM INTEROPERABILITY

[0396] Ass mentioned above, the storage platform of the present invention is, in at least some embodirments, intended to be embodied as ean integral part of the hardware/software —interface system of a computer system. For example, the storage p latform of the present inveration may be embodied as an integral part of an operating system, s uch as ‘the Microsoft Windows family of operating systems. “In that capacity, the storage platform

API becomes a part of the operating system APIs throx_igh which application programs- interact with the op erating system. Thus, the storage pelatform becomes the means through ‘which application programs store information on the o perating system, and the Item baased data model of the st-orage platform therefore replaces thhe traditional files system of such an - operating system. or example, as embodied in the Microsoft Windows family of ope rating systems, the storages platform might replace the NTFS file system implemented in that operating system. Presently, application programs acc -ess the services of the NTFS file system through the "Win32 APIs exposed by the Windows family of operating systems .

[0397] Recognizing, however, that completely replacing the NTFS file system wikith the storage platform of thes present invention would require recoding of existing Win32-bassed . application programs and that such recoding may be undesirabl-e, it would be beneficial fom the storage platform of the present invention to provide some irteroperability with existingz file systems, such as NTFS. In one embodiment of the present invention, therefore, the storage platform enables application programs which rely on thme Win32 programming mociel to access the contents of both the data store of the storage platfeorm as well as the traditionaal

NTFS file system. To this end, the storage platform uses a namu ing convention that is a superset of the Win32 narming conventions to facilitate easy int_eroperability. Further, the storage platform supports accessing files and directories stored in a storage platform volurmne. through the Win32 APL 1. Model for Interoperability

[0398] Accordiragto this aspect of the present inventi=on, and in accordance with the exemplay embodiment di scussed above, the storage platform imnplements one namespace mn which non-file and file itesms can be organized. With this mode=], the following advantages are achieved: 1. Folders in the data store can contain both file armd non-file items, thus presenting a single namespace for file and schematized data. MToreover, it also provides a uniform security, sharing and administration model for all user— data. 2. Since file znd non-file items are both accessible= using the storage platform . APIs and no special rules are imposed for files in this approacin, it presents a cleaner programming model for application developers to work against. 3. All namespace operations pass through the storage platform and hence are * handled synchronously. I't is important to note that deep property promotion (driven off of= file contents) still happens as~ynchronously, but the synchronous operations provide a much m._ore predictable environment for users and applications.

[0399] As a comsequence of this model, in the presen—t embodiment, search capabilities may not be povided over data sources that are not migrated into the storage . platform data store. This includes removable media, remote se-xvers and files on the local disk. A Sync Adapter is provided which manifests proxy item s (shortcuts + promoted metadata) in the storage platform for iterns residing in foreign file systems. Proxy items deo not attempt to mimic files either in terms of the namespace hierarchy of the data source or in terms of security.

[0400] The symmetry achieved on the namespace an«d programming model betweemn file and non-file content provides a better path for applications to migrate content from file systems to more: structured items in the storage platform data sstore over time. By providing za . native file item type in the storage platform data store, applica tion programs can transition file data into the storage platform while still being able to mamipulate this data via Win32.

Eventually, application programs might migrate to the storage platform API completely and - structure their data in terms of storage platform Items rather thman files. 2. Data Store Features

[0401] In order to provide the desired level of interoperability, in one embodiment, the following features of the storage platform data store are implemented. a) Not a volume

[0402] The storage platform data store is not exposecd as a separate file system volume. The storage platform leverages FILESTREAMs directly hosted on NTFS. Thus, there is no chan ge to the on-disk format, thereby obviating ansy need to expose the storage _ platform as a new file system at the volume level.

[0403] Instead, a data store (namespace) is constructed corresponding to an NTFS volume. The database and FILESTREAM:s backing this portion of the namespace is located on the NTES volume with which the storage platform data sto-we is associated. A data store i. corresponding to the system volume is also provided. } } b) Store Structure

E [0404] The structure of the store is best illustrated with an example. Consider, as arm ae : example, the directory tree on the system volume of a machines named HomeMachine, as illustrated in Fig. 16. In accordance with the file system interoperability feature of the present invention, corresponding to the c:\ drive, there is a stomrage platform data store exposed to the Win32 APIs via a UNC share, called, for exam ple, “WinFSOnC.” This make=s the associated d ata store accessible via the following UNC narme: \\HomeMachine\WinFSOnC.

[0405] In this embodiment, files and/or folders need to be migrated from NTFS to the storage platform explicitly. So, if a user desires to move th-e My Documents folder into the storage platform data stores in order to avail his or herself of a_1l the extra _ search/categorization features: offered by the storage platform, thes hierarchy would look as shown in Fig. 17. It is important to note that these folders are actmally moved in this example.

Another point to note is that the namespace moves into the storage platform, the actual streams are renamed as FILESSTREAMSs with appropriate pointer—s hooked up within the storage platform. : ©) Not all files are migrated

[0406] Files that correspond to user data or that need thee searching/categorization that the storage platform prov ides are candidates for migration in_to the storage platform data store. Preferably, in order to limit issues of application program «compatibility with the storage platform, the set of fil es that are migrated to the storage platform of the present invention, in the context of the Microsft Windows operating systeem, are limited to the files in the MyDocuments folder, Intemet Explorer (IE) Favorites, IE History, and Desktop .ini files : in the Documents and Setting s directory. Preferably, migrating “windows system files is not permitted. . d) NTFS namespace access to Stora_ge Platform files

[0407] In the embodiment described herein, it is desiralole that files migrated into ) the storage platform not be accessed via the NTFS namespace even though the actual file . streams are stored in NTFS. This way, complicated locking and ssecurity considerations that arise from a multi-headed implementation are avoided. - e) Expected namespace/drive letters=s

[0408] Access to filess and folders in the storage platform is provided via a UNC name of the form \\<machine name>\<WinfsShareName>. For thme class of applications that require drive letters for operation, a drive letter can be mapped tc this UNC name.

L STORAGE PEATFORM API

[0409] As mentioned above, the storage platform comp=tises an API that enables application programs to access the features and capabilities of thes storage platform discussed above and to access items stowed in the data store. This section d_escribes one embodiment of a storage platform API of the storage platform of the present inveention.

[0410] Figure 19 illustrates the basic architecture o=f the storage platform API, in accordance with the present embodiment. The storage platfcarm API uses SQLClient 1900 to talk to the local data store 302 and may also use SQLClient M900 to talk to remote data stores (e.g., data store 340). “The local store 302 may also talk to th_e remote data store 340 using; - either DQP (Distribute=d Query Processor) or through the the storage platform synchronization service (“Sync”) described above. The storage platform API 322 also acts as the bridge API for data store notifications, passing applicatiomn’s subscriptions to the notification engine 332 and routing notifications to the application (e.g., application 350a, 350b, or 350c), as alsoe described above. In one embodiment, the storage platform API 3222 may alsa define a limited “provider” architecture so that it camn access data in Microsoft "Exchange and AD. 1. Overview

[0411] The d ata access mechanism of the present emmnbodiment of the storage platform API of the present invention addresses four areas: q_uery, navigation, actions, events.

Query

[0412] In ones embodiment, the storage platform dasa store is implemented on a relational database enpgine 314; as a result, the full expressive= power of the SQL language is inherent in the storage platform. Higher level query objects gorovide a simplified model for * querying the store, but may not encapsulate the full expressiwse power of the storage. a Navigation : [0413] The storage platform data model builds a ric=h, extensible type system on ®he . underlying database ab»stractions. For the developer, the storamge platform data is a web of items. The storage platform API enables navigation from iterm to item via filtering, relationships, folders, etc. This is a higher level of abstractior than the base SQL queries; at the same time, it allow s rich filtering and navigation capabili—ties to be used with familiar

CLR coding patterns.

Actions

[0414] The storage platform API exposes common =actions on all items — Create,

Delete, Update; these are exposed as methods on objects. In addition, domain specific actieons such as SendMail, Che ckFreeBusy, etc. are also available as -methods. The API frameworks uses well defined patte-rns that ISVs can use to add value by defining additional actions.

Events

[0415] Data in the storage platform is dynamic. To let appli cations react when data in the store is changed, the API exp «ses rich eventing, subscription, Zand notification } capabilities to the developer. 2. Naming and Scopes {0416] It is useful to distinguish between namespace and naming. The term namespace, as it’s commonly used, refers to the set of all names avai_lable within some system. The system could be an X?AL schema, a program, the web, tlhe set of all ftp sites (and theeir contents), etc. Naming is the process or algorithm used to assigm unique names to all en: tities of interest within a namespace. Thus, naming is of interest be=cause it is desirable to urmambiguously refer to a given unit within a namespace. Thus, the teem “namespace,” as us=-ed herein, refers to the set of all names available in all the storage platform instances in the "universe. Items are the named entities in the the storage platform narmespace. The UNC na ming convention is used to ensure uniqueness of item names. Ever_y item in every the storage platform store in the univers eis addressable by a UNC name.

[0417] The highest organizational level in the the storage pl atform namespace is a . . * semvice — which is simply an instanc e of the storage platform. The next level of organization . is =a volume. A volume is the largest autonomous container of items. Each storage platform instance contains one or more volumes. Within a volume are items. It—ems are the data atoms in the storage platform.

[0418] Data in the real world is almost always organized according to some system .thaat makes sense in a given domain. Underlying all such data organiz ation schemes is the no~tion of dividing the universe of our data into named groups. As discussed above, this - nowion is modeled in the storage platform by the concept of a Folder. A Folder is a special type of Item; there are 2 types of FoRders: Containment Folders and Virtual Folders.

[0419] Referring to Fig. 18, a Containment Folder is an item which contains holding

Re lationships to other Items and is the equivalent of the common concept of a file system fol der. Each Item is “contained” within at least one containment folde=r.

[0420] A Virtual Folder is = more dynamic way of organizirmg a collection of Items; it i=s simply a name given a set of Itemms — the set is either enumerated explicitly or specified by aquery. The Virtual Folder is itself an Item and can be thought of _as representing a set of (noon-holding) Relationships to a set «©f Items.

[0421] Sometimes, there is the need to model a tighter notion of containrmnent; for h example, a Woard document embedded in an email message is, in a sense, bound more tightly . to its container than, for example, a file contained with in a folder. This notion is emxpressed by the concept of Embedded Items. An Embedded Item hans a special kind of relationship which references another Item; the referenced Item can be bound to or otherwise manipu_lated only within the context of the containing Item.

[0422] Finally, the storage platform provides the notion of categories as =a way of classification of liems and Elements. Every Item or Element in the storage platform can have associated with it one or more categories. A category iss, in essence, simply a name that is tagged on to time Item/Element. This name can be used in searches. The storage platform data model allows the definition of a hierarchy of categories, thus enabling a tree-like classification of data,

[0423] An unambiguous name for an item is the triplet: (<serviceName, <volumeID>, <ItemID>). Some items (specifically, Folders and VirtualFolders) amre collections of other items. This gives rise to an alternative way of identifying item_s: (<serviceName>, <volumelD>, <itemPath>).

[0424] The storage platform names include the notion of a service context: a service context is a nadme which maps to a (<volumeName>, <—path>) pair. It identifies an_ item or a set of items — For instance, a folder, virtual folder, etc. "With the concept of service contexts, the UNC name for any item in the the storage platform namespace becomes: serv iceName>\<serviceContext>\<itemPath—

[0425] Users can create and delete service cosntexts. Also, the root direct=ory in each volume has a pre-defined context: volume-name$. ’ [0426] An ItemContext scopes a query (for example, a Find operation) b~y limiting the results rettmrned to those Items that live within a specified path. 3. Storage Platform API Compoments

[0427] Fig. 20 schematically represents the v-arious components of the st-orage . platform APL, in accordance with the present embodiment of the invention. The s torage platform API consists of the following components: (1) data classes 2002, which mrepresent the storage platform element and item types, (2) runtirme framework 2004, which manages object persistence and provides support classes 2006; znd (3) tools 2008, which ar—e used to ~ generate CLR classes from the storage platform scherreas.

[028] According to one aspect of the present invention, at design time, the s chema author subsmits a schema document 2010 and code £or domain methods2012 to the set of storage platform API design time tools 2008. These: tools generate the client side data classes 12002 and the store schema 2014 and store class defi nitions 2016 for that schema. “Dor main” refers to a particular schema; for instance, we talk a bout domain methods for classes ir the

Contacts s chema, etc. These data classes 2002 are vised at runtime by the application developer, in concert with the storage platform API runtime framework classes 2006, £-o ‘manipulates the storage platform data.

[0429] For purposes of illustrating various. aspects of the storage platform AFI of the present invention, examples are presented based on an exemplary Contacts schemas. A pictorial representation of this exemplary schema is illustrated in Figures 21A and 2188. 4. Data Classes

[0430] According to an aspect of the present invention, each Item, Item Extemnsion, arid Element type, as well as each Relationship, in the storage platform data store has =a corresponcling class in the storage platform API. Roughly, the fields of the type map teo the fields of time class. Each item, item extension, and el ement in the storage platform is av—ailable as an object of the corresponding class in the storage platform API The developer can query for, create. modify, or delete these objects.

[0431] The storage platform comprises an initial set of schemas. Each schema defines a s et of Item and Element types, and a set of” Relationships. The following is ore embodime nt of an algorithm for generating data classes from these schema entities: ‘ For each schema S:

For each Item, I, in S a class named System. Storage.S.I is generated. This class- has a the following members: e Overloaded constructors, including c onstructors that allow a new item's initial folder and name to be specified. e A property for each field in I If the fHeld is multi-valued, the property will be a collection of the corresponding Elem ent type. ¢ An overloaded static method which ffinds multiple items matching the fi-lter (for example, a method named “Find_All”). * An overloaded static method which finds a single item matching a filter— (for example, a method named *“FindOne™”).

» A static method which finds an item given its id (for example, a method named “FindByID”). e A static method whi ch finds an item given its name relative to an ItemContext (for example, a method named “FindByName”). * A method which saves changes to the item (for example , a method named “Update™). e Overloaded static Create methods which create new inst_ances of the item.

These methods allow the item's initial folder to be specified in various ways. . For each Element, E, in S a class named System. Storage.S.E is generated. This class has the following members: ¢ A property for each field in E. If the field is multi-valu-ed, the property will : be a collection of the corresponding Element types.

For each Element, E, in S a class named System.Storage.S.ECoMlection is generated.

This class follows general .INET Framework guidelines for strorgly typed collection . classes. For Relationship based element types, this class will alszo include the following members: * An overloaded method which finds multiple Item objeccts that match a filter which implicitly includes the item in which the collection appears in the source role. The overloads include some that allow filteering based on Item . sub-type (for example, a method named “FindAllTarge=tltems™). * An overloaded method which finds a single Item objec=t that matches a filter ~~. which implicitly includes the item in which the collectmon appears in the source role. The overloads include some that allow filteer based on Item sub- type (for example, a method named “FindOneTargetItem™). ® An overloaded method which finds objects of the neste=d element type that match a filter which implicitly includes the item in which the collection appears in the source role (for example, a method nameed “FindAllRelationships”). ¢ An overloaded method whichs find objects of the neste=d element type that match a filter whic h implicitly includes the item in which the collection appears in the source role (for example, a method nameed “FindAllRelationshipsFor Target”). * An overloaded method which finds a single object of thhe nested element type that matches a filter which implicitly includes the item in which the collection appears in the source role (for example, a meesthod named “FindOneRelationship”).

i » An overloaded method which finds a single object of the nestedll element type : that matches a filter w hich implicitly includes the item in whicin the " collection appears in the source role (for example, a method narmed . “FindOneRelationshipForTarget”).

For Relationship, R, in S a clas s named System.Storage.S.R is generated. This class will have one or two sub-classess, depending on if one or both relationship roles specify an end point field.

Classes are also generated in this manner for each Item Extension that has been created. © [0432] The data classes exist in the System.Storage.<schemaName> n=mespace, where <schemaName> is the name of the corresponding schema — such as Contaacts, Files, etc. For example, all classes corresponding to the Contacts schema are in the

System.Storage.Contacts namespace.

[0433] By way of example, with reference to Figs. 21A and 21B, the c-ontacts schema results in the following classes, contained in the System.Storage.Contact namespace: o Items: Item, Folder, WellKnownFolder, LocalMachineDataFolder, User-DataFolder,

Principal, Service, GroupService, PersonService, PresenceService, ContaactService,

ADService, Person, User, Group, Organization, HouseHold + Elements: NestedElementBase, NestedElement, IdentityKey, Security), EAddress,

ContactEAddress, TelehoneNummber, SMTPEAddress, InstantMessaging- Address,

Template, Profile, FullName, FF amilyEvent, BasicPresence, WindowsPre=sence,

Relationship, TemplateRelationaship, LocationRelationship,

FamilyEventLocationRelations hip, HouseHoldLocationRelationship, RoeleOccupancy,

Co EmployeeData, GroupMemberShip, OrganizationLocationRelationship,

HouseHoldMemberData, FamilyData, SpouseData, ChildData

[0434] By way of further exazmple, the detailed structure of the Person type, as . defined in the Contacts schema, is shown in XML below: <Type Name="Person" MajorVersion="1" MinorVersion="0"

ExtendsType="Core.Principal” Exte ndsVersion="1"> . <Field Name="Birthdate" Type="the storage platformTypes.datetime” . Nullable="true" TypeMajorVersion="1"/> <Fiold Name="Gender® Type="Base.CategoryRef® ] Nullable="true" MultiValued="falsse" ) TypeMajorVersion="1"/> <Field Name="PersonalNames” Type="Contact.FullName"

Nullable="true" MultiValued="true"

TypeMzjorVersion="1"/> <Field Narre="PersonalEAddresses” Type="Core.EA.ddress"

Nullablea="true® MultiValued="true"

TypeMaajorVersion="1"/> <Field Name="PersonalPostalAddresses"

Type="CCore.PostalAddress" Nullable="true"

MuiltiVa lued="true” TypeMajorVersion="1"/> <Field Nanme="PersonalPicture® Type="the storage pEatformTypes.image"

Nullablea="true" TypeMajorVersion="1"/> <Field Nane="Notes" Type="Core.RichText" Nullable>="trus"

MuitiVa Jued="true" TypeMajorVersion="1"/> <Field Nanme="Profession" Type="Base.CategoryRef’*

Nullablex="true" MultiValued="true"

TypeMajorVersion="1"/> <Field Nanme="DataSource" Type="Bass.ldentityKey™

Nullable2="true" MultiValued="true" .

TypeMamjorVersion="1"/> <Field Name="ExpirationDate" Type="the storage pla_tformTypes.datetime*

Nullablex="true" TypeMajorVersion="1"/> <Field Name="HasAllAddressBookData" Type="the storage platformTypes.bit"

Nullablea="true" TypeMajorVersion="1"/> <Field Namme="EmployeeOf" Type="Contact.Employe eData"

Nullables="true" MultiValued="true"

TypeMaajorVersion="1"/> </Type>

[043-5] This type results in the following class (only the public members are showmn): partial public class Person : i

Systerm,Storage.Core.Principal,

Systermn.Windows.Data.|DataUnit { public System.Data.SqlTypes.SgiDateTime

Birt-hdate { get; set; } public System.Storage.Base.CategoryRef

Gemnder { get; set: } public System.Storage.Contact.FullNameCollection

Per-sonaiNames { get; } public System.Storage.Core.EAddressCollaction

Per-sonalEAddresses { get; } public -System.Storage.Core.PostalAddressCollectiorn

Per-sonalPostalAddresses { get; } public System.Data.SqlTypes.SqiBinary

Per—sonalPicture { gst; set; } public ~System.Storage.Core.RichTextCollection

Not=es { get; } public =System.Storage.Base.CategoryRefCollection

Pro fession { get; } . public SSystem.Storage.Base.ldentityKeyCollection

Dat aSource { get; } : public &System.Data.SqiTypes.SqiDateTime

ExpirationDate { get; set; } public SSystem.Data.SqTypes.SqlBoolean

HassAllAddressBookData { get; set; } ‘ public SSystem.Storage.Contact.EmployeeDataCollection

Em gployeeOf { get; } public BPerson(); public KPerson( System.Storage.Base.Folder folder, string name ); public sstatic new System.Storage.FindResult

Finc3All( System.Storage.ltemStore store ); public sstatic new System.Storage.FindResult

FincAll(

System. Storage. ltemStore store, ssstring filter ); public sstatic new Person

FincdOne(

SSystem.Storage.ltemStore store,

string fililter ); public new evant . System.W indows.Data.PropertyChangedEventHandler-

ProperityChangedHandler; public static reew Person

FindBylD(

System.Storage.Iltem Store store, long itemm_key ); } . [0436] A s yet another example, the detailed structure of the TelephoneNumbe=r type, as defined in the Contacts schema, is shown in the XML be=low: <Type Name="TelepohoneNumber” Extends Type="Core.EAddress"

MajorVersion=="1" MinorVersion="0" ExtendsVersion="1"> <Field Name=="CountryCode" Type="the storage platform ypes.nvarchar(50)"

Nullable=" true" MultiValued="false"

TypeMajomVersion="1"/> <Field Name="AreaCode" Type="the storage platform Typ=es.nvarchar(256)" : Nullable=" true” TypeMajorVersion="1"/> <Field Name=="Number" Type="the storage platformTypes- .nvarchar(256)"

Nullable=" true" TypeMajorVersion="1"/> <Field Name=="Extension® Type="the storage platformTypes.nvarchar(256)" . Nullable=" true" TypeMajorVersion="1"/> * <Field Name="PIN" Type="the storage platformTypes.nva_rchar(50)"

Nullable="-true" TypeMajorVersion="1"/> </Type>

[0437] This type results in the following class (on 1y the public members are skown): partial public class T elephoneNumber :

System.Stora ge.Core.EAddress,

System. Windows. Data.lDatalUnit { public Syster.Data.SqiTypes.SqiString CountryCode { get; set; 3 public System.Data.SqlTypes.SqiString AreaCode { gst; set; 3 public System .Data.SqiTypes.SqlString Number { get; set; } public Systen—.Data.SqITypes.SqlString Extension {get set, 3 public System.Data. SqiTypes.SqiString PIN { get; set; B public T elephoneNumber(); - public neew event

System .Windows.Data.PropertyChangedEventHandler

P~ ropertyChangedHandler; 1 [0438 ] The hierarchy of classes resulting farom a given schema directly reflects the hierarchy of t=ypes in that schema. As an example, consider the Item types defined in the

Contacts schema (see, Figs. 21A and 21B). The class hierarchy corresponding to thi sin the storage platfaerm API would be as follows:

Objec t

DataClass

ElementBase

RootItemBase

Item

Principal

Group

Household . Organization

Person

User

Service

PresenceService

ContactService

ADService

RootNestedBase ... (Element clamsses) [04390 Yet another schema, the schema theaat allows representing all the audio/video media in the system (ripped audio files, audio CDs, IDVDs, home videos, etc.), enabl=es

: users/applications to store, organize, search through, and manipulate different kinds of audio/video media. The base media document schema is generic enougkn to represent any media, and the extensions to this base schema are designed to handle domain-specific : properties separately for audio and video media. This schema, and marmy, many others, are envisioned to operate directly or indirectly under the Core Schema. 5. Runtinse Framework

[0440] ‘The basic storage platform API programming model is object persistence. . Application programs (or “applications”) execute a search on a store an d retrieve objects representing the data in the store. Applications modify the retrieved objects or create new objects, then cause their changes to be propagated into the store. This pmrocess is managed by an ItemContext object. Search es are executed using an ItemSearcher ob_ject and search results are accessible via a FindResult object. a) Runtime Framework Classes : [0441] According to another inventive aspect of the storage pl -atform API, the 3 runtime framework implements a number of classes to support the oper=ation of the data : classes. These framework classes define a common set of behaviors for the data classes and, together with the data classes, provide the basic programming mode] for the storage platform

APL Classes in the runtime framework belong to the System. Storage n=amespace. In the present embodiment, the framework classes comprise the following ma#n classes:

ItemContext, ItemSearcher, annd FindResult. Other minor classes, enurrm values, and delegates . may also be provided.

QQ) ItemContext

[0442] An ItemContext object (i) represents a set of item domains that an application program wants to search, (ii) maintains state information for each object that © represents the state of the data as retrieved from the storage platform, ard (iii) manages the - transactions used when interacting with the storage platform and any fil=e system with which . the storage platform may interoperate.

[0443] As an object persistence engine, ItemContext provides the following services: 1. Deserializes data read from the store into objects.

ER 2. MiZaintains object identity (the same obje=ct is used to represent a gi ven item no m_atter how many times that item is inclmuded in the result of querie=s). : 3. Tmacks object state.

[0444] XternContext also performs a number Of services unique to the st=orage . platform: . 1. Generates and executes the storage plat=form update gram operaticmns necessary to persist changes. 2. Creates connections to multiple data stores as necessary to enable the seamless nzavigation of reference relationships and to allow objects retrieve-d from a multi-domain search to be modified anc saved. 3. Trsures that file backed items are prope—xly updated when changes to the oMbject(s) representing that item are saveed. 4. Manages transactions across multiple s®orage platform connectiors and, when updating data stored in file backed items and file stream propertiess, the transacted file system. 5. Performs item creation, copy, move, amd delete operations that tallke storage platform relationship semantics, file ba=cked items, and stream typoed properties ito account.

[0445] Appendix A provides a source code li sting of the ItemContext c=lass, in accordance with one embodiment thereof. (2) ItemSearcher

[0446] The ItemSearcher class supports simple searches, which return ~whole Item objects, streams of Item objects, or streams of values projected from Items. Item Searcher encapsulates the= core functionality that is common to zal] of these: the concept of™ a target type and parameterizeed filters that are applied to that target type. The ItemSearcher also allows

E searchers to be pore-compiled, or prepared, as an optimization when the same seamrch will be executed multip_le types. Appendix B provides a source code listing of the ItemSSearcher class } and several closeely related classes, in accordance with. one embodiment thereof. (a) Target Type

[0447] The search target type is set when corstructing an ItemSearcher—. The target type is a CLR ty=pe that is mapped to a queryable externt by the data store. Specifically, itis a

. CLR type that is mapped to item, relationship, and item extension types as well as "* schematized views.

[0448] When retrieving a searcher using the ItermnContext.GetSearcher method, thme © searchers targest type is specified as a parameter. When a static GetSearcher method is invoked on an -tem, relationship, or item extension type (=e.g. Person.GetSearcher), the target

Eg .type is the itenn, relationship, or item extension type. ’ [0449] Search expressions provided in an ItemS- earcher (for example, the search filter and through find options, or projection definitions) are always relative to the search target type. These expressions may specify properties of ~the target type (including properties . of nested elements) and may specify joins to relationship and item extensions as described : elsewhere.

[0450] The search target type is made available. via a read only property (for example, an ItemSearcher. Type property). . (b) Filters [04511 The ItemSearcher contains a property tos specify filters (for example, a property named “Filters” as a collection of SearchExpresssion objects) that define the filter used in the search. All filters in the collection are combimned using a logical and operator _# ‘when the search is executed. The filter may contain parameter references. Parameter valt aes g . are specified through the Parameters property. (¢) Preparing Searches

[0452] In situations where the same search is tc be executed repeatedly, possibly with only parameter changes, some performance improvement can be gained by pre-

Lo compiling, or preparing, the search. This is accomplishe d with a set of prepare methods cn the ItemSearcher (for example, a method to prepare a Fi nd that returns one or more Items, perhaps named “PrepareFind”, and a method to prepare= a Find that returns a projection, : * perhaps named “PrepareProject”). For example: ltemSearcher searcher = ...; . PreparedFind pf = searcher.PrepareFind(); result = pf.FinclAll(); result = pf.FinAl(); (d) Find Optieons

[0453] There are a number of options that can be applied toa simple search. These may be specified, for example, in a FindOptions object and passed to the Find methods. For : example: : ltemSearcher searcher = Perso n.GetSearcher( context );

FindOptions options = new FincAOptions(); options.MaxResults = 10; options SortOptions.Add( “Pers -onalNames.Surmame”, SortOrder.Ascencling );

FindResult result = searcher.FirdAll( options };

[0454] As a converience, sort options may also be passeci directly to the Find methods: [temSearcher searcher = Persoen.GetSearcher( context );

FindResult result = searcher.FimndAll( new SortOption( “PersormalNames.Surname”, SortOrder.Ascendin qg))

[0455] The Delayl_oad option determines if the values of large binary properties are loaded when the search results are retrieved or if loading is delaye d until they are referenced.

The MaxResults option detexmines the maximum number of results that are returned. This is equivalent to specifying TOXP in a SQL query. It is most often used in conjunction with sorting.

[0456] A sequence of SortOption objects can be specifie«d (for example, using a

FindOptions.SortOptions property). The search results will be sorted as specified by the first

SortOption object, then by ass specified by the second SortOption object, etc. The SortOption specifies a search expression that indicates the property that will bee used for sorting. The expression specifies one of tthe following: 1. a scalar property in the search target type; 2. a scalar property in a nested element that is reachable from the search target . type by travemsing single valued properties; or 3. the result of aan aggregation function with a valid ar-gument (for example, Max : applied to a s calar property in a nested element tha t is reachable from the search target type by traversing a multi-valued progperty or a relationship).

For example, assumi ng the search target type is System.Steorage.Contact.Person: 1. "Birthdate" — valid, birthdate is a scalar property off the Person type; 2. "PersonalNarmes.Sumame" — Invalid, PersonalNames is a multi-valued property and no aggregation function was used; : 3. "Count(PerscwnalNames)" — Valid, the count of PerssonalNames.

4, "Case(Contact. MemberOfHou 1sehold). Household HouseholdES Addresses. Start } Date" — Invalid, uses relationship and multi-valued propertiess without an aggregation function. 5. "Max(Cast(Contact MemberOfHousehold). Household. House=holdEAddresses.

StartDate)" ~ Valid, most recent household e-address start date. (3) Item Result Stream (“FindResult”) ) [0457] The ItemSearcher (for exancaple, through the FindAll metho d) returns an object that can be used to access the objects retumed by the search (for exarmple, a “FimdResult” object). Appendix C provides a source code listing of the FinedResult class and several closely related classes, in accordance with one embodiment thereof.

[0458] There are two distinct methods for getting results from a FandResult object: _ usimag the reader pattern defined by IObjectRReader (and IAsyncObjectReade=r) and using the enumerator pattern as defined by IEnumeralole and IEnumerator. The enumerator pattem is starxdard in the CLR and supports language constructs like C#’s foreach. Fear example: [termnSearcher searcher = Person.GetSearcher( context ); seavrcher.Filters.Add( “PersonaiNames.Sumame = ‘Smith™ );

FindResult result = searcher.FindAll(); foreaach( Parson person in result) ...;

[0459] The reader pattern is supported because it allows results to be processed more efficiently by eliminating a data copy in some cases. For example:

ItermSearcher searcher = Person.GetSearcher( context); seaarcher.Filters Add( “PersonalNames.SurNama e = 'Smith™ );

Fin <dResutt result = searcher.FindAli(); whi le( result. Read() ) { “ Person person = (Person)resuit.Current; }

[0460] In addition, the reader patte=m supports asynchronous oper ation: iternSearcher searcher = Person.GetSearcher( context); searcher. Filters. Add( “PersonalNames.SurNanmwse = ‘Smith™ );

Fire dResult result = searcher. FindAll();

IAyrsncResult asyncResult = result.BeginRead( new AsyncCallback( MyCallback J» ); voi d MyCallback( IAsyncResult asyncResult ) { if( result. EndRead( asyncResuilt ) ) . {

Person person = (Person)resutt.Current;

} : [0461] In the present embodiment, a FindResult should be closed when it is no : longer needed. This can be done by calling the Close method or using language constructs such as C#'s using statement. For example: ltemSearcher searcher = Person.GetSearcher( context ); searcher. Filters.Add( “PersonalNames.SurName = ‘Smith™ }; using( FindResult result = searcher. FindAll(}) ) { while( result. Read() ) {

Person person = (Person)result.Current; , } b) Rumtime Framework in Operation

[0462] Figure 22 illustrates the runtime framework in operation. The runtime framework operates as follows: 1. An application 350a, 350b, or 350c binds to an item in the s torage platform. 2. The framework 2004 creates an ItemContext object 2202 comrresponding to the bound item and returras it to the application. 3. The application submits a Find on this ItemContext to get a collection of Items; the returned collection is conceptually an object graph 2204— (due to relationships). 4. The application changes, deletes, and inserts data. 5. The application saves the changes by calling the Update() nethod. c) Common Programming Patterns

[0463] This section pro vides a variety of examples of how thes storage platform API

N _ framework classes can be used to manipulate items in the data store. a (1» Opening and Closing ItemCont_ext Objects : [0464] An application gets the ItemContext object it will use to interact with the data store, e.g. by calling a static ItemContext.Open method and provicling the path or paths that identify the item domains th at will be associated with the ItemCommtext. Item domains scope the searches performed using the ItemContext such that only the- domain item and the items contained in that item will be subject to the search. Examples are as follows:

Open zn ItemContext with the DefaultStore storage platform share on the local computer ftemCoantext ic = ltemGContext.Open();

Open .an ItemContext with a given storage pL atform share ltemContext ic = ltemContext.Open( @ "\myserveer!\DefaultStore* );

Open an ItemContext with an item under a s&orage platform share

ItemContext ic = ItemContext.Open({ @"\\myserv-er\WinFSSpecs\apiimé” );

Open an ItemContext with multiple item dormains ltemCeontext ic = ltemContext.Open( @"\myserv-erl\My Documents”, @"\janet\My Documents" , @"\jane2\My Documents ); [@465] When an ItemContext is no lorager needed, it must be closed.

Explicitly Close an temContext ltemC-ontext ic = itemContext.Open(); ic.Closse();

Close- using statement with an ItemContext using( ltemContext ic = itemContext.Open() ) : { } 2) Searchin g for Objects [®0466] According to another aspect off the present invention, the storage platform

API prowides a simplified query model that enables application programmers to foxrm queries based om various properties of the items in the «fata store, in a manner that insulates the applicati_on programmer from the details of the query language of the underlying database engine.

[0467] Applications can execute a seamch across the domains specified wlhen the

ItemCommtext was opened using an IternSearche x object returned by the

TtemComtext.GetSearcher method. Search resul ts are accessed using a FindResult Object.

Assume the following declarations for the exarmples below: ltemC=ontext ic = ...; itemS= earcher searcher = null;

FindRz esult result = null;

Item item = null;

Relati onship relationship = null;

ItemE-xtension itemExtension = null;

[0468] The basic sear<h pattern involves using an Item&Searcher object retrieved from an ItemContext by calling the GetSearcher method.

Search for all items of a giveen type searcher = ic.GetSearcher( typoeof( Person ) ); result = searcher.FindAll{); foreach( Person p in result) ... ;

Search for items of a given type that satisfy a filter searcher = ic.GetSearcher( typeof( Person ) ); searcher.Filters.Add( *PersonaalNames.Surname = 'Smith™ ); result = searcher.FindAll(); foreach( Person p in result) ..—;

Use a parameter in a filter s#tying searcher = ic.GetSearcher( typoeof( Parson } ); searcher.Filters.Add( "Birthdat e < @Date" ); searcher.Parameters["Date"] — someDate; result = searcher.FindAll(); foreach( Person p inresult ) .._;

Search for relationships of za given type and satisfying a filter searcher = ic.GetSearcher( typoeof( EmployesEmployer ) }; searcher.Filters.Add( "StartDate <= @Date AND (EndDate >= @Daate OR isnull(EndDate))" ); searcher.Parameters['Date"] = someDate; result = searcher.FindAl); = Foreach{ EmployeeEmployer «¢ in result) ...;

Search for items with relationships of a given type and satisfying a filter searcher = ic.GetSearcher( tyyoeof( Folder } ); searcher.Filters.Add( "Member Relationships.Name like 'A%"™ ); // See [ApiRel] result = searcher.FindAll();

Foreach( Folder f in result) ...5

Search for item extensions ©f a given type and satisfying a fil ter searcher = ic.GetSearcher( typoeof( ShellExtension ) ); searcher. Filters. Add{ "Keywor«ds.Value = 'Foo™ ); result = searcher.FindAll(); foreach( ShellExtension se in wesult) ...;

Search for items with item extensions of a given type and sat sfying a filter searcher = ic.GetSearcher( tyroeof( Person ) ); searcher.Filters.Add( "Extensions.Cast(@ Type). Keywords.Value = 'Foo™ ); // See [ApiExt} searcher.Parameters["Type"] = typeof( SheliExtension }); result = searcher.FindAll(); foreach( Person p in result) ... ; (a) Search Options

[0469] Various optioris can be specified when executin_g a search, including sorting, delay loading, and limiting the number of results.

oo Sort search results - s=searcher = ic.GetSearcher( typeof( Persor ) ); - =searcher.Filters.Add( "PersonalNames.Su mame = 'Smith™ );

SSearchOptions options = new SearchOptions(); } «options. SortOptions.Add( new SortOption "Birthdate®, SortOrder.Ascendirmg) ); . mesult = searcher. FindAll( options ); . ~¥oreach( Person p in result) ...; «// A shortcut is available: =searcher = ic.GetSearcher( typeof( Persor) ); _searcher.Filters.Add{ "PersonalNames.Susrmame = 'Smith™ ); ] result = searcher.FindAll{ new SortOption € "Birthdate, SortOrder.Ascendirg) ); —foreach( Person p in result) ...; _Limit result count searcher = ic.GetSearcher( typeof( Persom ) ); searcher.Filters.Add( "PersonalNames.Sumame = 'Smith™ );

SearchOptions options = new SearchOpti ons(); options.MaxResults = 10; result = searcher.FindAll( options ); foreach( Person p in result) ...; b) FindOne and FindOnly

[0470] On occasion retrieving only the first result is useful, esspecially when specifying sort criteria. In addition, some searches are expected to returm only one object and are= not expected to return no objects.

Search for one object searcher = ic.GetSearcher( typeof( Person) ); searcher. Filters.Add( *PersonalNames.Starmame ='Smith™ );

Person p = searcher.FindOne( new SortOption( “Birthdate” SortOrder.Asc=ending ) ) as Person; if(pl=null)..;

Search for single object that is expected to always exist searcher = ic.GetSearcher( typeof( Person) ); searcher. Filters. Add( "PersonalNames[SLirname ="'Smith’ AND Givennama e 'John’]" ); ry { - Person p = searcher.FindOnly(); ) catch({ Exception e ) { ) (c) Search Shortcuts on ItermmContext

[0471] There are also a number of shortcut methods on ItemC ontext that make exe=cuting simple searches as easy as possible.

Search using the E ttemContext.FindAll shortcut result = ic. FindAll{ typeof( Person ), "PersonalNames.Sumame = 'Smith™ ); foreach( Person p i nresult) ...;

Search using the MtemContext.FindOne shortcut

Person p = ic.Findne( typeof( Person ), “PersonalNames.Surmame = ‘Smith’ ) as Person; (d Find by ID or Path

[0472] In addition, Items, relationships, and item extensions can be retrieved by providing their id(s)~ Items may also be retrieved by path.

Get items, relationships, and item extensions given their i«i(s) item = ic.FinditemB3yld( iid ); relationship = ic.FirndRelationshipByld( iid, rid ); itemExtension = ic_ FinditemExtensionByld( iid, eid );

Get items given am path {// Single domain orly item = ic.FindltemB3yPath( @"temp\foo.bxt" ); // Single or multi-domain } result = ic.FindAlllt-emsByPath{ @ "temp\foo.txt* ); foreach( Item | in reesuit) ...; (e The GetSearch er Pattern

[0473] There are many places in the storage platform API where it is desirable to provide a helper method that executes a search in the contex € of another object or with specific parameters. The GetSearcher pattern enables these scenarios. There are many

GetSearcher methocls in the API. Each returns an ItemSearc ier pre-configured to perform a . given search. For exc ample: searcher = itemContext.GetSearcher(); searcher- = Person.GetSearcher(); searcher = EmployeeEmployer.GetSearcherGivenErmployer{ organization ); searcher = person.GetSearcherForReports();

You can add additioenal filters before executing the search: searcher = person.GetSearcherForReports(); searcher Filters. Add( "PersonalNames.Surname='Srmnfth™ );

You can choose how you want the results:

FindResault findResult = searcher.FindAll();

Person person = searcher.FindOne();

We’) 2005/029313 PE&ECT/US2003/026144 (3) Up Qating the Store

[0474] Once an object has been retrieved by a search it may be m=odified by the appli cation as needed. New objects may also be created and associated with existing objects.

Once= the application has made all the chamnges that form a logical group, th e application calls : Itern®Context. Update to persist those chan ges to the store. According to yet another aspect of . the swtorage platform API of the present imvention, the API collects changess to an item made a. by ar application program and then organizes them into the correct updates required by the database engine (or any kind of storage emgine) on which the data store is implemented. This : enables application programmers to make changes to an item in memory, v=vhile leaving the complexity of data store updates to the A PL

Save Changes to a Single Item : Pearson p = ic.FindlitemByld( pid ) as Pers on; p. DisplayName = "foo"; p."TelephoneNumbers.Add( new TelephorisNumber( "425-555-1234" ) ); ic—Update(); - Saave Changes to Multiple Items ’ Heousehold h1 = ic.FindtemByld( hid1 ) ass Household;

Heousehold h2 = ic.FinditemByld( hid2 ) ass Household;

Pearson p = ic.FindltemByld( pid ) as Pers on; : h¥ MemberRelationships.Remove( p ); h=.MemberRelationships.Add({ p ); ic Update();

C_xeate a new Item

Folder f = ic.FinditemByld( fid } as Folderzg

Pearson p = new Person(); p. DisplayName = "foo"; f.Felationships.Add( new FolderMember( p, "foo" )); ic—Update(); // Or using a shortcut...

Folder f = ic.FinditemByld( fid ) as Folders

Person p = new Person(); p. DisplayName = "foo"; f.;mvlemberRelationships.Add( p, "foo" ); ic Update();

Delete relationships (and possibly the target Item) searcher = ic.GetSearcher( typeof( Folder—Member) ); searcher. Filters. Add( "Sourceltemld=@fic3"); searcher. Filters. Add( “Targetltemld=@ pid"); searcher. Parameters. Add( "fid”, fid ); sesarcher.Parameters.Add( "pid®, pid );

WYO 2005/029313 PCT/US2003/026144 i foareach( FolderMember fm in searcher.FinndAll() ) fm.MarkForDelete(); ic=.Update(); // Or using a shortcut...

F older f = ic.FinditemByld( fid } as Folder: f. MemberRelationships.Remove( pid ); ic=.Update();

Aa dd an Item Extension it em item = ic.FinditemByld( iid );

MyExtension me = new MyExtension(}; rre.Foo = "bar"; iteem.Extensions.Add({ me ); ic.Update();

Delete Item Extensions : s earcher = ic.GetSearcher( typeof( MyExtension ) ); s earcher.Filters.Add( "ltemld=@iid" }; s-earcher.Parameters.Add( "iid", iid ); foreach( MyExtension me in searcher.FiredAll() ) me.MarkForDelete(); ic=.Update();

Jf” Or using a shortcut...

It=em i = ic.FinditemByld( iid ); i. Extensions.Remove( typeof( MyExtension ) ); ic.Update(), 6. Security

[0475] With reference to sectiom ILE above (Security), in the present embodiment of the storage platform API, there are five methods available on the Item Contex_t for retrieving and modifying the security policy associated with an item in the store. These are: 1. GetltemSecurity; 2. SetltemSecurity; 3. GetPathSecurity; 4. SetPathSecurity; and 5. GetEffectiveltemSecurity .

[0476] GetltemSecurity and SetltemSecurity provide the mechanism to retrieve and modify the explicit ACL associated with the item. This ACL is independent of tie paths that exist to the item and will be in play independent of the holding relationships whi ch have this iter as the target. This enables the admimistrators to reason about the item security inde=pendent of the paths that exist to the item if they so desire.

[0477] The GetPathSe«curity and SetPathSecurity proviclie the mechanism for retrieving and modifying the ACCL that exists on an item because of a holding relationship from another folder. This ACL. 3s composed from the ACL’s of tne various ancestors to the - item along the path under consiederation along with the explicit A_CL if any supplied for that path. The difference between th is ACL and the ACL in the previous case is that this ACL, remains in play only as long as the corresponding holding relationship exists while the explicit item ACL is independent of any holding relationship to zn item.

[0478] The ACL's that can be set on an item with SetIteemSecurity and

SetPathSecurity is restricted to -inheritable and object specific ACCE’s. They cannot contain any ACE marked as inherited.

[0479] The GetEffecti veltemSecurity retrieves the varicous path based ACL's as well as the explicit ACL on the item. This reflects the authorizat®on policy in effect on the given item. 7. Support. for Relationships

[0480] As discussed a bove, the data model of the storagze platform defines “relationships” that allow itemss to be related to one another. Wien the data classes for a schema are generated, the following classes are produced for each relationship type: 1. A class that represents the relationship itself. Thiss class is derived from the

Relationship class and containss members specific to the relationship type. 2, A strongly type "virtual" collection class. This class is derived from

VirtualRelationshipCollection znd allows relationship instances —to be created and deleted.

[0481] This section describes the support for relationshaps in the storage platform

APL a) Base Relationship Types

[0482] The storage pl atform API provides a number of types in the System.Storage namespace that form the found ation of the relationship API. Theese are: 1. Relationship — tthe base type of all relationship cl=asses 2. VirtualRelationsshipCollection — the base type for all relationship collections 3. ItemReference, JtemIdReference, ItemPathRefere=nce — Represent the item reference types; the relationship among these types is illustrated in Fig. 11.

’ (1) Relationship Class

[0483] The following is the base class for relationship cla=sses. public abstract class Relationship : StoreObject { // Create with default values. protected Relationship( itemIDReference targetitemReference=);

Jf Informs the relationship that it has been added to a relationship collection. The object // will interrogate the collection to determine the source item, item context, etc. internal AddedToCollection( VirtualRelationshipCollection collection ); // The relationship’s id. public Relationshipid Relationshiplid { get; } // The id of the source item. public Itemid Sourceltemid { get; } /! Get the source item. public Item Sourceltern { get; } /! Reference to the target item. public ItemidReference TargetitemReference { get; } // Get the target item (calls TargetitemReference.Getlitem()). public tem Targetitem { get; } // Determines if the item Context already has a connection to the t=arget item’s domain (calls . // TargetitemReference. IsDomainConnected). public bool IsTargetDomalinConnected { get; } // The name of the target item in the namespace. The name must be unique across all the // source item's holding relationships. . public OptionalValue<string> Name {get; set;} // Determines if this is a holding or reference relationship. : public OptionalValue<bool> IsOwned {get; set;} } 2) ItemReference Class ’ : [0484] The following is the base class for item reference types. public abstract class itemReference : NestedElement : { // Create with default values. protected itemRefererxce(); // Returns the item referenced. public virtual Item Getltem(); - /! Determine if a connection to the referenced item’s domain has oeen established. - public virtual bool IsDromainConnected();

: [0485] ItemReference objects may identify items that exist in a store other than the one whe=re the item reference itself resides. Each derived type specifies how a reference two a ‘remote store is constructed and used. Implementations of Getltem and IsDomainConnected in derivead classes use the ItemContext’s multi-domain support to load items from the necessary domain and to determine if a connection to the domain has already been established. ©) ItemIdReference Class

[0486] The following is the ItemIdRefrence class — an Item reference that uses -an item id ®o identify the target item. public class ltemidReference : ItemReference { // Construct a new ltemidReference with default values. public ItemidReference();

J €onstruct a new ltemldReference to the specified item. The domain associated with the

J/ Mem is used as the locator. public ItemidReference( Item item );

If Construct a new temidReference with a null locator and the given target item id. puablic temidReference( Itemid itemid ); /! Construct a new ItemlidReference with the given locator and item id values. piablic kemidReference( string locator, Itemid itesmid ); // "The id of the target item. public ftemld itemlid {get; set;} /! «A path identifying that WinFS item that contains the target item in its domain. If null, // &he domain that contains the item is not known. puablic OptionalValue<string> Locator {get; set;} // Determine if a connection to the referenced item’s domain has been established. public override bool isDomainConnected(); /f Retrieves the referenced item. puLablic override tem Getitem(); }

[0487] Getltem and IsDomainConnected use the ItemContext's multi-domain support to load items from the necessary domain and to determine if a connection to the domain has already been established. This feature is not implemented yet. @ ItemPathReference Class

[0488] The ItemPathReference Class is an item reference that uses a path to ide=ntify the targestitem. The code for the class is as follows:

peublic class ltemPathReference : ltermnReference { // Construct an item path reference with default values. public itemPathReference(); // Construct an item path reference with no locator and the given path. public temPathReference( strirag path );

JI Construct an item path reference with the given locator and path. public temPathReference( strirmg locator, string path ); // A path identifying that WinFS iteem that contains the target item in it's domain. public OptionalValue<string> L_ocator {get; set;} . // The path of the target item relative to the item domain specified by locator. public string Path {get; set;} // Determine if a connection to the referenced item’s domain has been established. public override bool IsDomainConnected(); // Retrieves the referenced item. public override item Getitem();

Fr 0 3

[0489] Getltem and IsDommainConnected use the ItemContex ts multi-domain support to load items from the necessary domain and to determine if a connection to the domain has already been established. 3) Relationshipld Structure

[0490] The RelationshipId Structure encapsulates a relationship id GUID. goublic class Relatlonshipld & {/ Generates a new relations id GRUID. public static Relationshipld NeswRelationshlipld(); // Initialize with a new relationshigo id GUID. public Relationshipld(); // Initialize with the specified GUI D. public Relationshipld{ Guid id ); // Initialize with a string representation of a GUID. public Relationshipld( string ici ); // Returns a string representatiore of the relationship id GUID. public override string ToStrincg(); // Converts a System.Guid instarice into a Relationshipld instance. - public static implicit operator IRelationshipld(Guid guid); // Converts a Relationshipid instexnce into a System.Guid instance. public static implicit operator &uid(Relationshipld relationshipld)e;

oo : [0491] This value type wraps a guid so that parameters and properties can be strongly typed as a relationship id. Optional Value<Relationsshipld> should be used when & relationship id is nu Tlable. An Empty value, such as provided by System.Guid. Empty, is not exposed. A Relatiorashipld cannot be constructed with an em. pty value. When the default constructor is used to create a Relationshipld, a new GUID iss created. (6) VirtualRelationshipC «llection Class

[0492] The VirtualRelationshipCollection class implements a collection of : relationship objects that includes objects from the data store, plus new objects that have be=en added to the collection, but not including objects that have been removed from the store.

Objects of a specified relationship type with a given source i tem id are included in the . collection.

CC [0493] This is the base class for the relationship collection class that is generated for each relationship type. That class can be used as the type of a property in the source item type to provide access arad easy manipulation of a given item's rek ationships.

[0494] Enwmerating the contents of a VirtualRelati «onshipCollection requires that a potentially large nuamber of relationship objects be loaded from the store. Applications should use the Count prope=rty to determine how many relationships: could be loaded before they . enumerate the contents of the collection. Adding and removi ng objects to/from the collectzmion ) does not require relationships to be loaded from the store.

[0495] Fox efficiency, it is preferable that applicaticons search for relationships th at satisfy specific critesria instead of enumerating all of an item™s relationships using a

VirtualRelationship-Collection object. Adding relationship o»bjects to the collection causes= the represented relation ships to be created in the store when ItennContext.Update is called.

Removing relations hip objects from the collection causes the represented relationship to b e deleted in the store “when ItemContext. Update is called. The= virtual collection contains thes correct set of objects regardless of whether or not a relations hip object is added/removed through the Item. Relationships collection or any other relationship collection on that item—

[0496] The following code defines the VirtualRelagionshipCollection class: public abstract class WirtualRelationshipCollection : ICollection /f The collection willl contain the relationships of the specified type owned by the item // identified by item] d.

i protected ViirtualRelationshipCollection( ltemContext itexnContext,

Itemid itemid,

Type relationshipType ); // The enumerator will return all the objects retrieved from the store minus any object thats // with the sate Deleted in addition to objects that have the state Inserted. public iEnurmerator GetEnumerator(); // Returns a_ count of the number of relationship objects that would be retumed by the // enumerateor. This count is computed without needing to retrieve all objects from the sto re. public int Count { get; } // Always re-tums false. public bool Collection.IsSynchronized(} { get; } /1 Always re-turns this object. public objec=t ICollection.SyncRoot { get; } // Searches the store for the necessary objects. public void BRefresh(); // Adds the =spacified relationship to the collection. The Object must have the state // Constructed or Removed. If the state is Constructed, itis changed to Added. If the statee // is Removeed, it is changed to Retrieved or Modified as appropriate. The relationships // source itemm id must be the same as the source item id provided when the collection was // constructed. protected veoid Add( Relationship relationship ); // Removes the specified relationship from the collection. The object's state must be // Added, Retrieved or Modified. If the object's state is Added, it will be set to // Constructed. If the object's state is Retrieved or Modified, it will be set to Removed. // The relati=onship’s source item id must be the same ass the source item id provided whesn // the collection was constructed. protected veoid Remove( Relationship relationship }; /] The objects that have been removed from the collection. public IColleaction RemovedRelationships { get; } // The objects that have been added to the collection. public Collection AddedRelationships { get; } // The objects that have been retrieved from the store. “¥his collection will be empty until // after the W/irtualRelationshipCollection is enumerated or Refresh is called (getting this /! property’ss value does not cause the collection to be filled). public Collection StoredRelationships { get; } // Asynchro nous methods. public |1AsyrcResult BeginGetCount( IAsyncCallback callback, object state ); public int EmndGetCount( IAsyncResult asyncResult }; public 1AsyricResult BeginRefresh( IAsyncCallback callback, object state ); public void EndRefresh( IAsyncResult asyncResult );

b) Generated Relationship» Types

[0497] When generating classes for a storage platform schema, a class iss generated foreach relationship declaration. In addition to a class that represents a relationship itself, a g relationship collection class is also generated for each relationship. These classes are used as the type of properti_es in the relationship’s source or taeget item classes.

[0498] Thais section describes the classes that are generated using a number of “prototype” classes. That is, given a specified relationship declaration, the class that is . generated is described. It is important to note the class ,, type, and end point namess used in the prototype classes axe place holders for the names speci fied in the schema for the relationship, and should not be taken literally. (6) Generated Relationship Types :

[0499] This section describes the classes that are generated for each relantionship type. For example: <Relationship Name ='ReilationshipPrototype" BaseType="Holding"> <Source Nam e="Head" ltemType="Foo"/> <Target Name=="Tail' ltemType="Bar" ReferenceTyp e="ltemIDReference”/> <Property Narne="SomeProperty” Type="WinFSTypees.String" /> </Relationship>

Given this mrelationship definition Relationship &Prototype and

Relationship ProtorypeCollection classes would be germerated. The RelationshipP-rototype - class represents the relationship itself. The Relationsh=ipPrototypeCollection clas-s provides access to the Relat onshipPrototype instances that hav=e a specified item as the sowmurce end + point. ‘ 2) RelationshipPrototype Class

[0500] This is a prototypical relationship classs for a holding relationshi> named "HoldingRelations hipPrototype" where the source endl point is named "Head" anad specifies the "Foo" item types and the target end point is named “Tail” and specifies the "Bax" item type. Itis defined as follows: public class RelationshipPrototype : Relationship public RelatieonshipPrototype( Bar tailitem ); public RelatieonshipPrototype{ Bar tallitem, stringy name ); public RelatieonshipPrototype( Bar talfitem, strinag name, bool IsOwned ); public RelatieonshipPrototype( Bar taifitem, bool 1 sOwned ); } public Relat/eonshipPrototype( ltemlidReference t=ailtemReference );

// Get the Head item (calls base.Sourceltem). public Foo Headltem {£ get; } /] Get the Tail item (calls base.Targetitern). : public Bar Ta/Mtem { cet; } // Represents additional properties declared in the schema for the rel ationship. These are

If generated just as for properties in an item or nested element type. : public string SomeProperty {get; set;} public static ltemSear-cher GetSearcher( itemContext itemContexxt ) public static ItemSear-cher GetSearcher( Foo headltem ); public static FindRes ult FindAll( string filter ); public static Relation. shipPrototype FindOne( string filter ); public static Relation shipPrototype FindOnly( string filter ); } 3) RelationshipPrototypeCollect ion Class

[0501] This is a prototypical class, generated with the Relat-ZonshipPrototype class, that maintains a collection of RelationshipPrototype relationship inst ances owned by a specified item. It is defined as follows: public class RelationshipPr-ototypeCollection : VirtualRelationshipCol lection { ) public RelationshipPsototypeCollection( ltemContext itemContemxt,

Iteemid headitemid ); * public void Add( Relation shipPrototype relationship ); . public RelationshipProtoype Add( Bar bar ); public RelationshipProto#ype Add( Bar bar, string name ); public RelationshipProto#ype Add( Bar bar, string name, bool IsOwrmed ); } public RelationshipProtogype Add( Bar bar, bool IsOwned ); public void Remove( RelationshipPrototype relationship ); public void Remove( Bar bar ); public void Remove( item Id barltemid ); public void Remove( Relaationshipld relationshipid ); public void Remove( strirmg name ); } " c) Relationship Support in the Item Class

[0502] The Item class contains a Relationships property that provide access to the relationships in which that item is the source of the relationship. The Relationships property has the type RelationshipC ollection.

a Item Class

[0503] The following code shows the relationship contex_t properties of the Item class:. public abstract class Item : Store=Object {

J/ Collection of relationships ~where this item is the source. public RelationshipCollection Relationships {get;} } oo (Zz) RelationshipCollection Clamss

[0504] This class prov-ides access to the relationship instances where a given item is the source of the relationship. It is defined as follows: public class RelatlonshipCollection : VirtualRelationshipCollectiora { public RelationshipCollecition( temContext itemContext,

Itemid head temid ); public vold Add( Relationship relationship ); . public Relationship Add( Bar boar ); public Relationship Add( Bar boar, string name ); : public Relationship Add( Bar bar, string name, bool IsOwned ); public Relationship Add( Bar boar, bool IsOwned ); public void Remove( Relationsship relationship ) public void Remove( Bar bar )5 public void Remove( itemld bamritemld ); public void Remove( Relationsshipld relationshipld ); public void Remove( string name ); } d) Relationship Support in Search F-xpressions

[0505] It is possible to specify the traversal of a join bet—ween relationships and related items in a search expresssion. (B®) Traversing From Items to Relationships

[0506] When the current context of a search expression is a set of items, a join between the items and relations hip instances where the item is thes source can be done using

"the Item. Rel ationships property. Joining to relation=ships of a specific type can be specified using the search expression Cast operator. . [050°7] Strongly typed relationship collect dons (e.g. Folder MemberRelationsships) } can also be used in a search expression. The cast to- the relationship type is implicit. [050 8] Once the set of relationships has beeen established, the properties of tiaat relationship are available for use in predicates or ass the target of a projection. When u sed to specify the target of a projection, the set of relation ships would be returned. For exarrple, the following st atement would find all persons related to an organization where the StartI Date property of the relationships had a value greater then or equal to ‘1/1/2000’.

FindResult result = Person.FindAll( context, "Rexlationships.Cast(Contact. EmployeeOfOr ganization). StartDate > '1/1/2000™ );

If the Person type had a property EmployerContext of type

EmployeeS# deEmployerEmployeeRelationships (ass generated for an EmployeeEmployer relationship type), this could be written as:

Find esult result = Person.FindAll{ context, "ErnployerRelationships.StartDate > '1/1/2000" ); 2) Traversing From Relationships to Items

[05009] When the current context of the se=arch expression is a set of relationships, a . join from a xelationship to either end point of the rezlationship can be traversed by specifying the name of the end point. Once the set of related tems has been established, the progperties of those iterms are available for use in predicates or as the target of a projection. Wher used to specify the target of a projection, the set of items would be returned. For example, thee following statement would find all Employee OfOr s2anization relationships (regardless of organizatiory) where the employee’s last name is name “Smith”:

FindResult result = EmployesOfOrganization.FindAll( context, “Ermployee.PersonaiNames{SurName="Smitk’]" ); [051.0] The search expression Cast operat or can be used to filter the type of she end point item. Xor example, to find all the MemberOfEFolder relationship instances where the member is a Person item with the surname “Smith” *:

FindIResult result = MemberOffolder.FindAll( context, “Member.Cast(Contact.Person).PersonalNanmaes[Surname="Smith’]" };

€)] Combiming Relationship Traversal [0®511] The previous two patterns, traversing from items to relation ships and from relationships to items, can be combined to achmieve arbitrarily complex travemsals. For example, to find all organizations with an employee that has the Surname “SSmith”:

BE Fi ndResult result = Organization.FindAll( context, =‘EmployeeRelationships.” + “Employee.” + “*PorsonalNames[SurName = 'SmithT" };

[08512] The example below would fird all Person items representim g people who live in a Inousehold that is in the “New York” area (TODO: this is no longer supported... what is thae alternative). : FimndResult result = Person. FindAll( contes<t, } - ““Relationships.Cast{Contact.MemberOft—ousehold).” + ‘ “Household.” + * “Relationships.Cast{Contact.LocationOfFousehold).” + = “MetropolitonRegion = ‘New York™ ); e) Examples Uses of Relationship Support [0#513] The following are examples Of how the relationship support= in the storage platform AP] can be used to manipulate relationships. For the examples beleow, assume the followings declarations: temCoentextic = ...; ltemid ¥Fid = ...; // a folder item's id - Folder —folder = Folder.FindByld( ic, fid ); temld =sid = ...; // a source item's id.

Item samurce = ltem.FindByld( ic, sid ); : temld #&id = ...; // an target item's id. tem tamrget = tem.FindByld( ic, tid ); lfemSe archer searcher = null; a Searching for Relationships

[0514] It is possible to search for soumrce or target relationships. Filters can be used to select relationships of a specified type and t hat have given property valuess. Filters can also be used tc select relationships based related item type or property values. Fox example, the following searches can be performed:

All relsationships where a given item is the source searche=r = Relationship.GetSearcher( folder );

foreach( Relationship relationship in searcher.FINCRAII() ) -..;

All relat jonships where a given item is the source that have a name that matchees "A%" ’ searcher = Relationship.GetSearcher( folder ); . searcher .Filters.Add( "Name like 'A%™ ); foreach( Relationship relationship in searcher.FincAAII() } ...;

All FoldierMember relationships where a givem item is the source . searcher = FolderMember.GetSearcher( folder ); foreach({ FolderMember folderMember in searcher-.FindAll() ) J

All FoldlerMember relationships where a givem item is the source and a name Mike '’A%’ searcher = FolderMember.GetSearcher( folder ); searcher .Filters.Add( "Name like 'A%"™ ); foreach( FolderMember folderMember in searcher _FindAll() } ...;

All Folc3erMember relationships where the tar-get item is a Person searcher = FolderMember.GetSearcher( folder ); searcher .Filters.Add{ "Memberltem.Cast(Person)'” ); foreach( FolderMamber folderMember in searcher .FindAll() } ...;

All FolcderMember relationships where the target item is a Person with the Sumname "Smith" searcher = FolderMember.GetSearcher{ folder ); searcher Filters.Add( "Memberitem.Cast(Person).- PersonalNames.Surname="Smith™ ); foreach( FolderMember folderMember in searcher .FindAll() ) ...; [05 15] In addition to the GetSearcher API shown above, each relationsknip class supports st atic FindAll, FindOne, and FindOnly _APL In addition, a relationship ype can be _ specified when calling ItemContext.GetSearcher, ItemContext.FindAll,

ItemConte=xt.FindOne, or ItemContext.FindOnly - @ Navigating from a Relationship to the Samurce ’ and Target Items [05 16] Once a relationship object has been retrieved through a search, i_tis possible to "navigat=e" to the target or source item. The ba se relationship class provides Sourceltem and Target Ttem properties that return an Item object. The generated relationship «class provides the equivalent strongly typed and named properties (e.g. FolderMembemr FolderItem and Folder-Member.Memberltem). For example=

Navigate to source and target item for relatioraship with the name "Foo" searcher = Relationship.GetSearcher(); searcher Filters.Add( "Name="Foo"' ); foreach( Relationship relationship in searcher.Fin«dAli() ) { {tem source = relationship.Sourceltem; tem -target = relationship. Targetitem;

} ~ Navigate to the target item : searche-r = FolderMember.GetSearcher( folder ); searche-r.Filters.Add( "Name like 'A%™ ); foreach & FolderMember folderMember in searchesr.FindAll() ) dq ltem nraember = folderMember.Targetltem; } » - 3 . .

[05517] Navigating to a target item womks even if the target item is not im the domain where the relationship was found. In such casess, the storage platform API opens & connection to the targcet domain as needed. Applications can determine if a connection would be required before retmieving the target item.

Check for target item in an unconnected dormain searchear = Relationship.GetSearcher( source ); foreachm ( Relationship relationship In searcher.Fi ndAll() ) { if( relt onship.JsTargetDomainConnected ) { item member = relationship.Targetitem; } } (3) Navigatimg from Source Items to ’ Relationsships : [® 518] Given an item object, it is pos sible to navigate to the relationships for which that item is the source without executing an explicit search. This is done using tlhe

Item.Relantionships collection property or a strongly typed collection property such as

Folder.M-emberRelationships. From a relationship, it is possible to navigate to thee target item.

Such nav-igation works even if the target item is not in the item domain associate=d with the _ source itesm’s ItemContext, including when thes target item is not in the same store as the target itexm. For example:

Navigate from a Source Item to Relationship to Target Items

Console WriteLine( "item {0} is the source of thea following relationships:", source.lteanid ); foreach { Relationship relationship in source.ReRationships ) : { item &arget = relationship.Targetitem;

Conseole.WriteLine( * {0} ==> (1}*, relationshi>.Relationshipld, target.itemid ); }

Navig ate from a Folder Item to Foldermem ber Relationships to Target Items

Conso le.WriteLine( "Item {0} is the source of th « following relationships:*, folder.lterld };

foreach( FolderMember folderMembenr in folder. MemberRelationships ) . Item target = folderMember.GetMermberltem(); , Console. Writalne( * {0} => {1}*, folderMember.Relationshipld, target.ltedmlid ); oo [0519] An item may have many relationships, so applications should use caution when enurnerating a relationship coll ection. In general, a search should be used to identify particular relationships of interest insstead of enumerating the entire coll ection. Still, having a collection based programming mode for relationships is valuable enough, and items with many relationships rare enough, that the risk of abuse by the developer is justified.

Applications can check the number of relationships in the collection and use a different programming model if needed. For example:

Check the size of a relationship collection if( folder.MemberRelationships.Courzt > 1000 ) : Console. WriteLine( *Too many rela-tionships!” }; le . { 0 ee - [0520] The relationship collections described above are "virtumal” in the sense that they are not actually populated with objects that represent each relatiormship unless the application attempts to enumerate th e collection. If the collection is entamerated, the results . reflect what is in the store, plus what has been added by the applicatiora but not yet saved, but not any relationships that have been removed by the application but not saved. “@ Creating Relationships (and Items)

[0521] New relationships zure created by creating a relationship object, adding it to a relationship collection in the source item, and updating the ItemContexxt. To create a new item, a holding or embedding relationship must be created. For examp»le:

Add a new item to an existing foX der

Bar bar = new Bar(); folder.Relationships.Add( new FoldearMember( bar, "name" ) ); ic.Update(); #1 Or

Bar bar = new Bar(}); folder.MemberRelationships.Add( neaw FolderMember( bar, "name" ) ); ic.Update();

{Or

Bar bar =new Bar(); folder.MBemberRelationships.Add( bar, name ); > ic.Update();

Add an existing item to an existing folder _ folder.MRemberRelationships.Add( target, "name" ); : ic.Upda-ts();

Add ame existing item to a new folder

Folder exxistingFolder = Ic.FinditemByld( fid ) as Folder;

Folder mewFolder = new Folder(); existing Folder.MemberRelationships.Add( newFoldRer, "a name" ); newFolcder.MemberRelationships.Add( target, “a namme” ); ic.Upda te();

Add a mew item to a new folder

Folder eexistingFolder = I¢.FinditemByld( fid ) as Folder;

Folder rmewfFolder = new Folder(); existing Folder.MemberRelationships.Add( newFolcler, "a name” );

Bar bar =new Bar(); newFoleder.MemberRelationships.Add( bar, “a name" }; ic.Upda_ts(}; 5) Deleting Re=lationships (and Items)

Delete aholding relationship - /1 if the source item and relationship ids are known...

Relatiorshipld rid = ...;

Relatiorship r = ic.FindRelationshipByld( fid, rid ); r.Mark—orDelete; ic.Updaxte(); // Otherwise... folder.MlemberRelationships.Remove( target ); ic.Updemte(); 8. “Extending” the Storage Platform API

[02522] As noted above, every storage platform schema results in a set of classes.

These classes have standard methods such as Finc3* and also have properties for geetting and setting fie=1d values. These classes and associatecd methods form the foundation of the storage platform API.

a) IDomain Behaviors

[0523] In addition to “these standard methods, every scheema has a set of domain specific methods for it. We call these domain behaviors. For exarmple, some of the domairm ~ behaviors in the Contacts schemma are: ¢ Is an email address valid? e Given a folder, get the ecollection of all members of the folder. e Given an item ID, get a-n object representing this item e Givena Person, get his online status s Helper functions to cresate a new contact or a temporary ceontact e And soon.

[0524] It is important. to note that while we make a disti_nction between “standarci” : behaviors (Find*, etc) and domain behaviors, they simply appear— as methods to the programmer. The distinction bestween these methods lies in the fact that standard behavior—s are generated automatically fromm the schema files by the storage platform API design time tools while domain behaviors amre hard-coded. : [0525] By their very mature, these domain behaviors should be hand-crafted. Thi=s ‘leads to a practical problem: th einitial version of C# requires tha_t the entire implementation of a class be within a single files. Thus, this forces the auto-generated class files to have to be edited to add domain behaviors. By itself, this can be a problem.

[0526] A feature callesd partial classes has been introduced in C# for problems sumch as these. Basically, a partial claass allows the class implementatiomn to span multiple files. A= partial class is the same as a reggular class except that its declarati on is preceded by the keyword partial: partial public class Person : DeriveeditemBase { // implementation }

[0527] Now, domain “behaviors for Person can be put in a different file like so: partial public class Person { public EmailAddress Prima ryEmailAddress { get { /*implementation*/ }

b) V alue-Add Behaviors

[0528] Data classes with domain behaviors form a foundation t hat application developers build on. However, it is neither possible nor desirable for dat=a classes to expose every conceivable behavior related to that data. The storage platform all ows a developer to build on the base functionality Offered by the storage platform API. The basic pattern here is to write a class whose methods take one or more of the the storage platform data classes as parameters. For example, the value add classes for sending email using Microsoft Outlook or using Microsoft Windows messenger can be as below:

MailMessage m = MailMessage.FindOne(...);

OutlookEMailServices.SendMessage(m);

Person p = Person.FindOne(...);

WindowsMessagerServices m = new WindowsMessagerServices(p); m.MessageReceived += new Mes sageReceivedHandler( f ); m.SendMessage("Hello");

[0529] These value-add classes can be registered with the stora_ge platform. The registration data is associated with the schema metadata the storage platform maintains for every installed storage platform type. This metadata is stored as storage —platform items and can be queried.

[0530] Registration of value-add classes is a powerful feature; —for example, it allows the following scenario: Right click on a Person object in the Shell explomrer and the set of actions allowed could be derived from the value-add classes registered faor Person. .©) V alue-add Behaviors as Service Provide=rs

[0531] In the present embodiment, the storage platform API preovides a mechanism whereby value-add classes can be registered as “services” for a given typoe. This enables an application to set and get service providers (= value add classes) of a giv=en type. Value-add classes wishing to utilize this mechanism should implement a well know=n interface; for example: interface IChatServices void SendMessage(string msg); event MessageReceivedHa ndler MessageReceived; lass WindowsMessengerServices : IChatServices

. ) glass YahooMessermgerServices : IChatServices : .-

[0532] A_llthe storage platform API data classes implement the

ICachedServicePreovider interface. This interface exteends the System.IServicePro~vider interface as follow~s: interface ICachedSearviceProvider : System.IServiceProvieder void SetServiece(System.Type type, Object provider); void Remote SService(System. Type type);

Using this interfac-e, applications can set the service gorovider instance as well as request a service provider off a specific type.

[0533] Teo support this interface, the storage platform data class maintairs a hashtable of service providers keyed by type. When & service provider is requeste=d, the implementation fir—st looks in the hashtable to see if aa service provider of the specafied type has been set. If nos, the registered service provider infrastructure is used to identify a service provider of the specified type. An instance of this provider is then created, added to the hashtable, and renmmed. Note that it is also possible —for a shared method on the data class to request a service p=rovider and forward an operation t-o that provider. For example-, this could be used to provide a Send method on the mail message class that uses the e-mail system specified by the usser. 9. Design Time Framework

[0534] Thhis section describes how a storage= platform Schema gets turne dinto storage platform AaPl classes on the client and UDT lasses on the server, in accomrdance with the present embod—iment of the invention. The diagramm of Fig. 24 shows the components involved.

[0535] With reference to Fig. 24, the types min the schema are contained &n an XML file (box 1). This f=ile also contains field level and itemm level constraints associated with the schema. The storamge platform Class generator (xfs2c=s.exe — box 2) takes this file and generates the partizal classes for the store UDTs (box 5) and partial classes for the client classes (box 3). Foer each schema domain, there exist additional methods — which we call domain behaviors. There are domain behaviors that make sense on the stoxe (box 7), on the * client (box 6), and in both places (box 4). The code in boxes 4, 6, and 7 are hand written (not — auto generated). The partial classes in boxes 3, 4, and 6 together form tlhe complete class implementation for the storage platform API domain classes. Boxes 3, 4, sand 6 are compiled (box 8) to form the storage platform API classes — box 11 (actually, the st orage platform API is the result of compiling boxes 3, 4, and 6 that result from all initial schemma domains). In " addition to the domain classes, there also exist additional classes which implement value-add behavior. These classes make ase of one or more classes in one or more sachema domains.

This is represented by box 10. The partial classes in box 4, 5, and 7 together form the ‘complete class implementation for server UDT classes. Boxes 4, 5, and 7 are compiled (box - 9) to form the server side UDT" assembly — box 12 (actually, the server sicle UDT assembly is ‘ the result of compiler-plus-ing boxes 4, 5, and 7 that result from all initial schema domains).

The DDL Command Generator module (box 13) takes the UDT assembly~ (box 12) and the

Schema file (box 1), and instal 1s them on the data store. This process involves, among other * things, the generation of tables and views for the types in each schema. 10. Query Formalism

[0536] When reduced to the basics, the application’s pattern whee=n using the storage platform API is: Open an Item®Context; use Find with a filter criterion to retrieve the desired objects; operate on the objects; and send changes back to the store. This s ection is concerned with the syntax of what goes into the filter string.

[0537] The filter string provided when finding the storage platfcerm data objects describes the conditions that thee properties of the objects must meet in oreder to be returned.

The syntax used by the storage: platform API supports type casts and relationship traversal. a) Filter Basics } [0538] A filter string is either empty, indicating that all objects Of the specified type are to be returned, or a boolean expression that each returned object must satisfy. The expression references the object's properties. The storage platform API nantime knows how these property names map to the storage platform type field names and, ultimately, to the

SQL views maintained by the the storage platform store.

[0539] Consider the following examples: // Find all people

FindResult res! = Person.FindAll (ctx)

// Find all people that have a Gender property value equal // to "Male”

FindResult res2 = Person.FinAAll(ctx, "Gender="Male™) // Find all people that have a Gender property value equal // to *Male" and that were borra in the last millennium.

FindRestilt res3 = Person.Finc3All( ctx, “Gender='"Male’ And Birthdate < 1/1/2001")

[0540] The properties of nested objects can also be u=sed in the filter. For example=: // Find all people that were modified in the last 24 hours

FindResult res1 = Person.FincdAll( ctx,

String.Format("ltem.Mo«dified > *{0}",DateTime.Now.Subtract(_ new TimeSpan(24,0,0))));

[0541] For collectHons, it is possible to filter members using a condition in square- brackets. For example:

JI Find all people with the first mame “John” and the last name : J/ “Smith”

FindResult res1 = Person.FinclAll( ctx, *ParsonalNames[Given Name='John’ And Surname="Smith’]")s . JI Find all people with a real tirne address from provider x’ // and with an online status category of 'y’'

FindResult res2 = Person.FincdAll( ctx, “PersonalRealtimeAddr ess[ProviderURI='x}.BasicPresence." + "Online Status.Categgory='y™)

The following example lists all people born since 12/3 1/1999: ltemContext ctx = itemContext.Open("Work Contacts");

FindResult results =

Person.FindAll( ctx, "Biathdate > '12/31/1999" ); foreach( Person person in res ults )

Console.WriteLine(persson.DisplayName); ctx.Close();

[0542] Line 1 creates a new ItemContext object to access the "Work Contacts" om - the storage platform share On the local computer. Lines 3 and. 4 get a collection of Person objects where the Birthdate property specifies a date more rec ent then 12/31/1999, as - specified by the expression "Birthdate > ’12/31/1999™. The exxecution of this FindAll operation is illustrated in Fi gure 23.

: b) Type Casts

[0543] It is often the case that the type of a value stored in a property is derived from the properties dec lared type. For example, the PersonalEA eddresses property in Person contains a collection of types derived from EAddress such as ENnlailAddress and

TelephoneNumber. In order to filter based on telephone area cole, it is necessary to cast from the EAddress type to thxe TelephoneNumber type: /{ Find all people with a pthone number in the 425 area code

FindResult res1 = Persons. FindAll( ’ cix, : "PersonalEAddresses." + "Cast(System.Storage.Contact.TelephoneNumber)).” + : "AreaCode="425"); // Alternatively, you could pass the type name as follows:

FindResult res1 = Person.FindAll{ ctx,

String. Format("PersonalEAddresses.Cast({0})). AreaCode="425"", typeof(Telephon eNumber).FullName )) c) Filter Syntax . [0544] Below is a description of the filter syntax suppcmrted by the storage platform : API, in accordance with one embodiment.

Filter ::= EmptyFilter | Condition

EmptyFilter : :=

Condition ::= SimpleCondition | CompoundCondit=ion : ParenthesizedCondition

SimpleConditi on ::= ExistanceCheck | Comparison

ExistanceCheck ::= PropertyReference

Comparison :: = PropertyReference ComparisonOp Constant

CompoundCondi tion ::= SimpleCondition Boolean(Dp Condition

ParenthesizedCondition ::= '(’ Condition ’)°

ComparisonOp ::= 'l=° | tot | [ | tet | rs ~~ ] [rey | tem?

BooleanOp ::= ‘And’ | ’s&&’ | ‘or’ | ‘||’

Constant ::= StringConstant | NumericConstatarmt

StringConstant ::= ‘‘’‘ (any Unicode character—)* '''

Note: embedded ' characters are escaped by duplication

NumericConstant ::= 0-9*

PropertyReference ::= SimplePropertyName | CompoundPropertyName

SimpleProper—tyName ::= (all Unicode char—acters except ‘.’ and speace)*

Filter? , Filter ::= ¢ [’ Condition '}° : CompoundProroertyName ::= (TypeCast | RelationshipTraversal | SimplePropertyName) *.

PropertyReference

TypeCast ::== 'Cast(’ TypeName '}’

Relationship>Traversal ::= TraversalToSoumrce | TraversalToTarget : TraversalToSSource ::= ‘Source(’ FullRelaationshipName ’)’

TraversalToX arget ::= ‘Target (’ PullRelaationshipName ‘)‘

TypeName ::= a fully qualified CLR type name

FullRelatiorashipName ::= SchemaName ’.’ RelationshipName

SchemaName = := the storage platformName . RelationshiroName ::= the storage platfor—mName . the storage platformName ::= as defined in [SchemaDef] 11. Remoting a) Local/Remote Transparemcy in the API

[0545] Data access in the storage platform is tax-geted to the local storage platfeorm instance. The local insstance serves as a router if the quersy (or part thereof) refers to rem_ote data. The API layer timus provides local/remote transparemicy: there is no structural diffemrence in the API between lo cal and remote data access. It is pumely a function of the requested scope.

[0546] The sstorage platform data store also impelements distributed queries; thmus, it is possible to connect to a local storage platform instances and perform a query which includes “items from different v=olumes, some of which are on the Rocal store and others on the remote _ store. The store union_s the results and presents it to the application. From the point of view of the storage platform API (and hence the application deve=loper) any remote access is completely seamless eand transparent.

[0547] The sstorage platform API allows an app lication to determine if a given

ItemContext object (a_s returned by the ItemContext.Opem method) represents a local or remote connection using the IsRemote property — this is a property on the ItemContext object. Among other t=hings, the application may wish to provide visual feedback to helgo set user expectations for gperformance, reliability, etc.

b) Storage Platform Implementation of Remoting

[0548] The storage platform data stores talk to each other using a special OLEDB provider which runs over HTTP (the default OLEDB provider uses TDS). Ima one embodiment, a distributed query goes through the default OPENROWSET fuanctionality of thee relational database engine. A special user defined function (UDF):

DeoRemoteQuery(server, queryText) is provided to do actual remoting. c) Accessing Non-Storage Platform Stores

[0549] In one embodiment of the storage platform of the present inwention, there is no» generic provider architecture that allows any store to participate in storage= platform data ac cess. However, a limited provider architecture for the specific case of Micr-osoft Exchange an d Microsoft Active Directory (AD) is provided. This implies that developers can use the storage platform API and access data in AD and Exchange just as they wouldll in the storage platform, but that the data they can access is limited to the storage platform scchematized typoes. Thus, address book (= collection of the storage platform Person types) is supported in

AID, and mail, calendar and contacts are supported for Exchange. d) Relationship to DFS

[0550] The storage platform property promoter does not promote past mount points.

Ewen though the namespace is rich enough to access through mount points, gmueries do not pass through them. The storage platform volumes can appear as leaf nodes ira a DFS tree. €) Relationship to GXA/Indigo

[0551] A developer can use the storage platform API to expose a “GEXA head” on top of the data store. Conceptually, this is no different from creating any othe-r web service.

Thee storage platform API does not talk to a storage platform data store using «GXA. As mentioned above, the API talks to the local store using TDS; any remoting is handled by the loc=al store using the synchronization service. 12. Constraints

[0552] The storage platform data model allows value constraints on types. These coxstraints are evaluated on the store automatically and the process is transpa=vent to the user.

Note that constraints are checked at the server. Having noted this, sometimes , it is desirable to give the developer the flexibility to verify that the input data satisfies the cOnstraints without incurring the overhead of a round trip to the server. This is especially useful in interactive applica tions where the end user enters the dzata which is used to populate a-n object. The storages platform API provides this facility.

[0553] Reecall that a storage platform Schema is specified in an XML file, w hich is used by the storages platform to generate the appropriates database objects representing, the schema. It is also Cased by the design time framework of the storage platform API to auto generate classes.

[0554] Heere’s a partial listing of the XML file used to generate the Contacts schema: <Schema Name=="Contacts" MajorVersion="1" MinorVersion="8"> <ReferenceciSchema Name="Base" MajorVersi on="1" /> <Type Name=—"Person® MajorVersion="1l" Min orVersion="0"

ExteradsType="Principal" ExtendsVer sion="1°"> <Field Nadme="Birthdate" Type="the storazage platformTypes.datetime*

Nullablee="true" Multivalued="false" /> <Field Name="Gender" Type="the storage platformTypes.nvarchar(1l_6)"

Nullable="true® Multivalued="false® /=> <Field Naame="PersonalNames" Type=*FullNJame" TypeMajorVersion="1_"

Nullable="true® Multivalued=*true" /> <Field Namme="PersonalEAddresses" Type='"~ EAddress"

TypeMajorVersion="1" Nullable="true" BMultivValued="true" /> <Field Namue="PersonalPostalAddresses" T'ype="PostalAddress"

TypeMajorVersion="1* Nullable="true® EMultivValued="true" /> <Check>exmporession</Check> </Type> </Schema>

[0555] Time Check tags in the XML above spec=ify the constraints on the Persson type.

There can be more than one check tag. The above constraint is generally checked in th-e store.

To specify that the constraint can also be checked explic=itly by the application, the abcove

XML is modified 1E ke so: <Schema Name=="Contacts® MajorVersion="1" M inorVersion="8"> <Referenced Schema Name="Base" MajorVersion="1" /> <Type Name= "Person" ...> <Field Nammne="Birthdate® Type="the stora-ge platformTypes.datetim-e’

Nullable ="true" Multivalued="false" />= <Check InZApplication="true”>expression< _/Check> </Type> </Schema>

Note the new “InApoplication” attribute on the <Check> element, which is set to true. T_his causes the storage pwlatform API to surface the constraint in the API through an instancee me=thod on the Person class called Valiclate(). The application can call th3is method on the object to ensure that the data is valid arad, preventing a potentially useles -s round trip to the ser-ver. This returns a bool to indicate the results of validation. Note that the value constraints are= still applied at the server regardless of whether the client calls <object>. Validate() method or mot. Here's an example of how Validate can be used: ltermContext ctx = ltemContext.Open(); /f Create a contact in the user's My Contacts folder.

Folder f = UserDataFolder.FindMyPersonamlContactsFolder( ctx );

Person p = new Person( f);

J] Set the person’s birthdate. p.Birthdate = new DateTime( 1959, 6, 9 ); // Add a name categorized as a personal mame

FuBIName name = new FullName( FullNane.Category.PrimaryName }; narme.GivenName = "Joe"; naenme.Surname = "Smith*; p.PersonalNames.Add( name ); /ivallidate the Person object if (po.Validate() == false) : // data does not represent a valid pe»rson // ssave changes p.Ulpdate(); che .Close(); ~ [0556] There exist multiple access paths to the the storage plat®orm store — the storage platform API, ADO.NET, ODBC, OLEDB, and ADO. This raise=s the question of authoritative constraint checking — that is, how can we guarantee that da—ta written from, say,

OIDBC, go through the same data integzrity constraints as would data written from the storage " platform API. Since all constraints are= checked at the store, the constrai nts are now authoritative. Regardless of what API path one uses to get to the store, a_1l writes to the store arez filtered through the constraint checks at the store. 13. Sharing

[0557] A share in the storage platform is of the form: \W\<DNS Name>\<Context Service>>, where <DNS Name? is the DNS name of the machine, and <Context Se-rvice> is a comtainment folder, virtual folder, or am item in a volume on that machire, For example, asssume that the machine “Yohns_Deskt-op” has a volume called Johns_Irmaformation, and in this volume there exists a folder called Contacts_Categoriees; this folder contains a folder called Work, which has the work contacts for John: \Wohns_Desktop»\Johns_Information$\Contacts_Czaategories\Work "This can be shared as ““wNorkContacts”. With the definitiom of this share, \\Johns_Desktop\WorkC ontacts\TaneSmith is a valid stora_ge platform name, and identifies the Person item JaneSmmith. a) Representing a Share

[0558] The share item type has the following proaperties: the share name, and the share target (this can be anon-holding link). For example, the aforementioned share’s naxne is WorkContacts and tax-get is Contacts_Categories\Work «on the volume Johns_Informati on.

Below is the schema fragment for the Share type: «Schema xmins="http://schermnas.microsoft.com/winfs/2002/1 1/18/schema"

Name="Share" MajorVearsion="1" MinorVersion="0"> <ReferencedSchena Name="Base" MajorVersion="1"/> <ReferencedSchena Name="the storage platform Types “* MajorVersion="1"/> <Type Name="Shame" MajorVersion="1" MinorVersion="€"

Extends Type="Bamse.ltem" ExtendsVersion="1"> <Field Name="Namae" Type="the storage platformTypes. nvarchar(512)"

TypeMajorVersieon="1"/> <Field Name="Tarcm et” Type="Base.RelationshipData® Ty.speMajorVersion="1/> </Type> </Schema> " b) Managing Shares

[0559] Becaus-e a share is an item, shares can be managed just as with other itencas.

A share can be created, «deleted, and modified. A share is also secured the same way as other storage platform items. cD Accessing Shares

[0560] An app lication accesses a remote storage platform share by passing the share name (e.g. \Johns_Desk<top\WorkContacts) to the storage platform API in the

TtemContext.Open() method call. temContext.Open returis an ItemContext object instan ce.

The storage platform AFI then talks to the local storage platform service (recall that accessing remote storage platform. shares is done via the local storagze platform). In turn, the local storage platform services talks to a remote storage platform service (e.g. on machine

Johns_Desktop) with the given share name (e.g. WorkContactts). The remote storage plastform service then translates Worl Contacts into Contacts_Categorie=s\Work and opens it. After that, query and other operations are performed just like other scopes. d) Discoverability

[0561] In one emb odiment, an application program an discover shares availab-1le on a given <DNS Name>, in thee following ways. According to t_he first way, the storage platform API accepts a DNS name (e.g. Johns_Desktop) as th e scope parameter in

ItemContext.Open() method. The storage platform API then c=onnects to the storage platform store with this DNS name as part of a connection string. With this connection, the only possible thing an applicatior can do is call ItemContext. Find Ali(typeof(Share)). A storage platform service then unions all the shares on all the attached volumes and returns the collection of shares. According to the second way, on alocal machine, an administrator can easily discover the shares ora a particular volume by FindAll(t—ypeof(Share)), or a particular folder by FindAll(typeof(Sh. are), “Target(ShareDestination).I = folderld”). 14. Sema ntics of Find

[0562] The Find* rnethods (regardless of whether they are called on the

ItemContext object or on an individual item) generally apply t-o Items (including embedded items) within a given contex_t. Nested elements do not have a BRind — they cannot be searched independently of their containing Items. This is consistent wit_h the semantic desired by tthe storage platform data model , where nested elements derive their “identity” from the containing item. To make this notion clearer, here are exampl_es of valid and invalid find operations: a) Show me all telephone numbers in the system whic=h have an area code of 206?

Invalid, since the find is being done on telephomne numbers — an element — without reference to an item. b) Show me all telephone numbers within all Persons which have area code of 266?

Invalid, even though a Person (=item) is refereraced, the search criterion dees not involve thaat item. ¢) Show me all telephone numbers of Murali (= one sangle person) which have a_tea : code of 206?

Valid, since there is a search criterion on an Iter (a Person named “Mural 5").

The exception to this rule is for nested element types derived directly or indir—ectly from the

Base. Relationship type. These types can be quesried individually through relastionship classes.

Such queries can be supported because the storage platform implementation e2mploys a "master- link table" to store Relationship eleme nts instead of embedding then inside item

UDTs. 15. The Storage Platform «Contacts API

[0563] This section gives an overview of the storage platform Contacts APL The schema behind the Contacts API is shown in Figures 21A and 21B. a) Overview of Sysstem.Storage.Contact

[0564] The storage platform API incl udes a namespace for dealing —with items and elements in the Contacts schema. This namesp ace is called System.Storage.Contact.

[0565] This schema has, for example , the following classes: eo Items: UserDataFolder, User, Person, .ADService, Service, Group, O=xganization, “Principal, Location eo “Elements: Profile, Postal Address, EmailAddress, TelephoneNumber_, “RealTimeAddress, EAddress, FullNam e, BasicPresence, GroupMemloership, “RoleOccupancy b) Domain Behavi ors

[0566] Below is a list of domain behaviors for the Contacts schema . When viewed from a high enough level, domain behaviors faall into well-recognizable categmories: e Static Helpers, for example, Person.CreatePersonalContact() to create a new personal contact; eo Instance Helpers, for example user. AuttoLoginToAllProfiles(), which logs in a user (instance of User class) into all profiles that are marked for auto logir; eo CategoryGUIDs, for example, Categor y.Home, Category. Work, etc; » Derived properties, for example, email _Address.Address() - returns a - string that combines the username and domain fie=1ds of the given emailAddress (=instance of

Email Address class); and

W=Q 2005/029313 PC T/US2003/026144 ® Derived collections, for exampele, person.PersonalEmail Addresses — given an instance of Person class, get her personal email addresses.

[0567] The table below givess, for each class in Contacts that has domain behaviors, a list os f these methods and the category they belong to.

BasicFresence Category URIs UnknownCategoryURI,

OfflineCategoryURI, BusyCategoryURI,

AwayCategoryURI, OnlineCategoryURI

Static helpers ConvertPresenceStateToString — format presence state as a localized string (actually localization needs to be added; just does a friendly English string now).

Category Category GUIDs Home, Work, Primary, Second ary, Cell, Fax,

Pager

EmaileAddress Derived properties | Address — combines username and domain

Static helpers IsValidEmail Address

Folder Derived propertiees | GetChildItemCollection — makes an item collection based on the Targets of the

FolderMembership.

Static helpers GetKnownFolder — specialized queries to get well-known folders

AddToPersonalContacts — adds an item to the well-known personal contacts folder

Static helpers GetltemFromID — does ID based query

Relaticonship Instance helpers BindToTarget - returns Item fox Target "| Person Derived collections | PersonalRealtimeAddresses,

PersonalEmail Addresses, ’

PersonalTelephoneNumbers

Derived propertiees | OnlineStatus, OnlineStatusIcon Source,

PrimaryEmail Address, Primary SecurityID

Static helpers CreatePersonalContact,

CreateTemporaryContact — creates new person in well-known folder

GetCurrentUser — get’s Person for currently logged in user

Securit VID Derived propertiess | UserName, DomainName, } DomainUserName

Teleph-oneNumber | Instance helpers SetFromUserInputString — parses telephone number string into parts

[eee meme string into parts - profiles that are mamrked for autologin 16. Storagze Platform File API

[0568] This section gives an overview of the the storage platform File AP], in . accordance with one embodixnent of the present invention. ’ . a) Introduction n Reflecting an NTFS Voluame in the Storage

Platform ) [0569] The storage platform provides a way of indexirg over content in existing

NTFS volumes. This is accormplished by extracting (“‘promotin=") properties from each file stream or directory in NTES and storing these properties as Itermns in the storage platform. ~~ [0570] The storage platform File schema defines two tem types — File and

Directory — to store promoted file system entities. The Director=y type is a subtype of the

Folder type; it is a containme=nt folder which contains other Directory items or File items.

[0571] A Directory item can contain Directory and Fil _e¢ items; it cannot contain items of any other type. As far as the storage platform is concexxned, Directory and File items : are read-only from any of thes data access APIs. The File Systemm Promotion Manager (FSPM) service asynchronously promotes changed properties irito the storage platform. The . properties of File and Directory items can be changed by the Win32 APL. The storage platform API can be used to Tread any of the properties of these items, including the stream associated with a File item. 2) Creating Files and Directories in the storage platform Namespace . [0572] When an N'XFS volume gets promoted to a storage platform volume, all the “files and directories therein aure in a specific part of that volume=. This area is read-only from the storage platform perspective; the FSPM can create new dire=ctories and files and/or change properties of existings items.

[0573] The rest of the namespace of this volume can contain the usual gamut of the storage platform item types — Principal, Organization, Documemnt, Folder, etc. The storage platform also allows you to cre=ate Files and Directories in any part of” the the storage platform namespace. These “native” Filess and Directories have no counterpart in the NTFS file system; they are stored entirely in the storage platform. Furthermore, changes to properties are visible immediately. -. [0574] However, the programming model remains the same : they are still read-only - as far as the the storage platforam data access APIs are concerned. The “native” Files and

Directories have to be updated using Win32 APIs. This simplifies the= developer's mental model, which is: 1. Any storage platform item type can be created anywhere in th € namespace (unless prevented by permissions, of course); 2. Any storage platform item type can be read using the storage gplatform API; 3. All storage platform ite ms types are writable using the storages platform API with the exception of File and Dwirectory; 4. To write to File and Dixectory items regardless of where they are in the namespace, : use the Win32 API; and 5S. Changes to File/DirectOry items in the “promoted” namespaces may not appear immediately in the storage platform; in the “non-promoted” n=amespace, the changes are reflected immediately in the storage platform. b) File Schema

[0575] Fi gure 25 illustrates the schema on which the File AWPI is based. c) Overview of System.Storage.Files

[0576] The storage pl atform API includes a namespace for clealing with file objects.

This namespace is called Syste-m.Storage.Files. The data members of the classes in

System. Storage.Files directly reflect the information stored in the stomrage platform store; this information is “promoted” frorm the file system objects or may be cre ated natively using the

Win32 APL. The System.Storage. Files namespace has two classes: Fileltem and

Directoryltem. The members Of these classes and methods thereof camn be readily divined by looking at the schema diagram in Figure 25. Fileltem and DirectoryIatem are read-only from the storage platform API In order to modify them, one has to use the Win32 API or classes in

System.IO.

d) Code Exampwles . [0577] In this section, three code e=xamples are provided illustreating the use of the . classes in System.Storage. Files. 4) Opening a File and Writing to It

[0578] This example shows how te do “traditional” file manipumlation. ltem Context ctx = ltemContext.Open();

Fileltem f = Fileltem.FindByPath(ctx, @ \My Do«cuments\bilig.ppt"); // example of handling file properties — ensure that file is // not read-only if (1f.1 sReadOnly) { FileStream fs = f.OpenWrite(); y II Read, write, close file stream fs ctx.C-lose();

Line 3 uses the FindByPath method to open the file. Line 7 shows the usee of the promoted property, IsReadOnly, to check if the file is writeable. If it is, then in line= 9 we use the

OpenWrite() method on the Fileltem object to get the file stream. . 2) Using Queries . [0579] Since the storage platform store holds properties promot—ed from the file systesm, it 1s possible to easily do rich queriess on the files. In this examples, all files modified in thes last three days are listed: /l Uist ali files modified in the last 3 days

FindResult result = Fileitem.FindAll{ lili oe op,

DateTime.Now.AddDays(-3)); preach ( Fileltem file in result ) -

[0580] Here's another example of wising queries — this one finds- all writable files of a certain type (= extension): /! Finn dall writable .cs files in a particular directo ry. // Equivalent to: dir c:\win\src\api\*.cs /a-r-d

Direc#oryltem dir =

Directoryltem.FindByPath(ctx, @"c:\win\s ac\api’);

FindResult result = dir.GetFiles( *Exxtension="cs’ and IsReadOnly=false"); preach ( File file in result) : we e) Domain Behaviors : [#0581] In one embodiment, in addition to the standard properties and rnethods, the file class also has domain behaviors (hand coded properties and methods). These behaviors are generally based on methods in the corresponding System.IO classes.

J. CONCLUSION : [®0582] As the foregoing illustrates, the p resent invention is directed to a storage platform for organizing, searching, and sharing data. The storage platform of tlme present inventiox extends and broadens the concept of data storage beyond existing file systems and database systems, and is designed to be the store for all types of data, including structured, non-strucctured, or semi-structured data, such as re lational (tabular) data, XML, =and a new form of cata called Items. Through its common storage foundation and schema-tized data, the storage platform of the present invention enables more efficient application deveelopment for consume=rs, knowledge workers and enterprises. It offers a rich and extensible ampplication programming interface that not only makes available the capabilities inherent irm its data model, b ut also embraces and extends existing files system and database access rnethods. It is understoeod that changes may be made to the embodiments described above witloout departing from the broad inventive concepts thereof. Accordingly, the present invention i s not limited to the pamrticular embodiments disclosed, but is intended to cover all modificatio ns that are within th_e spirit and scope of the invention as defi med by the appended claims.

[583] Asis apparent from the above, alll or portions of the various sysstems, methods,. and aspects of the present invention ma be embodied in the form of gprogram code (i.e., instructions). This program code may be stoxred on a computer-readable medium, such as a magmnetic, electrical, or optical storage mediurm, including without limitatiomn a floppy diskette, CD-ROM, CD-RW, DVD-ROM, DVD-RAM, magnetic tape, flash memory, hard _ disk drivee, or any other machine-readable storage amedium, wherein, when the p—rogram code is loaded into and executed by a machine, such as a computer or server, the mac hine becomes an apparatus for praacticing the invention.

The presemnt invention may also be embwodied in the form of program co de that is transmitted over some ®&ransmission medium, such a=s over electrical wiring or -cabling, through fiber optics, ove=r a network, including the Internet or an intranet, or via any other form of transmission, whereein, when the program code i s received and loaded imto and executed by a machine, such as & computer, the machine becomes an apparatus for practicing the invention.

When implermnented on a general-purpose gprocessor, the program code combines with the processor to prcvide a unique apparatus that operates analogously to spec—ific logic circuits. [Remainder of Page Intentio=nally Left Blank]

APPENDIX A namespace System.Storage { abstract class temContext : IDisposable, IServiceProvider { {tem Context Creation and Management Members /l Applications cannot create ltemContext objects directly nor camn they derive / classes from ItemContext. interal itemContext(); . // Create ItemContext that can be used to search the specified [Daths or, if no path /l is specified, the default store on the local computer. public static ItemContext Open(); public static temContext Open( string path ); public static itemContext Open( params string[] paths ); // Retum the paths specified when the ltemContext was created . public string[] GetOpenPaths(); // Create a copy of this I1temContext. The copy will have indeperdent transaction, caching // and update state. The cache will initially be empty. It is expected that using a /l cloned temContext would be more efficient then opening a new ltemContext using the // same item domain(s). public itemContext Clone(); // Close the ItemContext. Any attempt to use the ltemContext after it is closed will ’ // result in an ObjectDisposedException. : public void Close(); : void IDisposable.Dispose(); /l True if any domain specified when the ltemConext was opene=d resolved to a remote /l computer. public bool IsRemote { get; } // Returns an object that can provide the requested service type . Returns null if the

JI requested service cannot be provided. The use of the IServiceeProvider pattern allows // API that are not normally used and could confuse developers to be factored out of // the ltemContext class. {temContext can provide the following Binds of services:

JI ItemSerialization, IStoreObjectCache public object GetService( Type serviceType ); :

Update Related Members // Saves changes represented by all modified objects and all objects passed to / MarkForCreate or MarkForDelete. May throw UpdateCollision Exception if an update

JI collision is detected. . public void Update();

JI Saves changes represented by the specified objects. The objeacts must have either // been modified or passed to MarkForCreate or MarkForDelete, otherwise Argument- // Exception is thrown. May throw UpdateCollislonException if ar update collision is // detected. public void Update( object objectToUpdate ); public void Update( IEnumerable objectsToUpdate );

JI Refreshes the content of the specified objects from the store. If the object has // been modified, the changes are» overwritten and the object is no longer co msidered

I modified. Throws ArgumentException if anything other then an item, item «extension,

I/ or relationship object is specified. public void Refresh( object objectToRetfresh ); public void Refresh( IEnumerakblie objectsToRefresh ); // Raised when an update detects that data has been changed in the store [»etween when a : // modified object was retrieved and an attempt was made to save it. i no exvent handler

Il is registered, the update throwss an exception. If an event handler is registered, it : Jl can throw an exception to abort the update, case the modified object to owerwrite ’ /! the data in the store or merge the changes made in the store and in the object. public event ChangeCollisionE ventHandler UpdateCollision;

If Raised at various points during update processing to provide update progress

Jl information. public event UpdateProgressEwenthandler UpdateProgress;

Il Async versions of Update public IAsyncResuit BeginUpdate( IAsyncCallback callback, abject state ); public IAsyncResult BeginUpdate( object objectToUpdate,

IAsyncCalll»ack callback, object state ); public IAsyncResuit BeginUpdate( iIEnumerable objectsToUpdate, : IAsyncCalib» ack caliback, . object state );

B public void EndUpdate( IAsync Result result );

Il Async versions of Refresh public IAsyncResuit BeginRefrexsh( object objectToRefresh, . 1AsyncCallback callback, object state» ); public IAsyncResult BeginRefresh( IEnumerable objectsToRefresh,

IAsyncCallboack callback, object states ); public void EndRefresh( IAsyncResult result );

Transaction Related Members // Begins a transaction with the specified isolation level. The default isolatiors level

Il is ReadCommited. In all cases, a distributed transaction is started because it may

If have to encompass changes stream typed item properties. public Transaction BeginTransaction(); public Transaction BeginTransaction( System.Data.lsolationLevel isolaationLevel );

Search Related Members

Jl Create an ltemSearcher that wil 1 search this item context for objects of the ! specified type. Throws ArgumenrtException if a type othem then an item, // relationship, or item extension is specified. public ltemSearcher GetSearcher( Type type );

Jl Find an item given its id. public Item FinditemByld( lteml« itemid ); // Find an item given its path. The path may be absolute or relative. If it is relative,

Il NotSupportedException will be thrown if multiple item domains were specified when

Il the temContext was opened. WF ili return null if no such item exists. Creates a

/I connection te the Wmachine\share part of the dorms ain to retrieve the item. The /f tem will be &assocaited with that domain. public 1tem FllnditemByPath( string path ); // Find an item given its path. The path is relative to tlhe specified item domain. // Creates a connaction to the specified domain to retrieve the item. The item will toe /l associated vwith that domain. Will return null if no sich item exists. public Item F#indltemByPath( string domain, stringy path ); // Find a set of ites given a path. The path is relatives to the item domains specifiead } // when the ltemmContext was opened. Will return an empty result if no such item exists. public FindRexsult FindAllitemsByPath( string path); // Find a relationship given its ids. public Relatiasinship FindRelationshipByld( temic] itemID,

Relationshipid relations hhipid ); // Find a item extension given its ids. public ItemEx_tenslon FinditemExtensionByid( Itemid ftemid,

ItemExtenslonld itemE>xx=tensionid ); /I Find all item, relationship, or item extensions of the specified type optionally // satisifing a g dven filter. Throws ArgumentException if a type other then one of // these is specified. public FindRe=sult FindAll{ Type type ); public FindRemsult FindAll( Type type, string filter O); // Find any iter, relationship, or item extensions of the specified type that satisfies // a given filter. Throws ArgumentException if a type Other then one of these is / specified. Re=turns null if no such object is found. public object ¥FindOne( Type type, string filter ); /l Find the iterrw, relationship, or item extensions of thea specified type that satisfies // a given filter. Throws ArgumentException if a type other then one of these is // specified. Th rows ObjectNotFoundException if no s uch abject was found. Throws=s // MultipleObjectsFoundException if more then one otoject was found. public object BFindOnly( Type type, string filter ); // Returns true if an item, relationship, or item extensisons of the specified type that ’ /l satisfies a givsen filter exists. Throws ArgumentExcemption if a type other then one // of these is spwecified. ’ public bool Exxists( Type type, string filter ); // Specifies hovv the objects returned by a search relate to the object identity map

J/ maintained by the ltemContext. public Search«CollisionMode SearchCollisionMode { get; set; } // Raised when PreserveModifiedObjects is specified —for ResultMapping. This evermt allows // the applicatio=n to selectivly update the modified object with data retrieved with thee // search. public event CChangeCollisionEventHandler SearchhCollision; / Incorporate a_n object from annother itemContext info this item context. If an obje ¢t /l representing the same itern, relationship or tem exteension does not already exist= / this this temContext’s identity map, a clone of the o-bject is created and added to - // the map. If ar object does exist, it is updated with trme state and content of the // specified objexct in a way concistant with the Search «CollisionMode. public item Inc orporateltem( item item );

public Relationship IncorporateRelationship( Relation ship relationship ); public ItemExte nsion IncorporateitemExtension( itemEExtension itemExtension ); } #/ Handler for Item -Context.UpdateCollision and Item Searchesr.SearchCollision events. public delegate void ChangeCollisionEventHandler( obJect source,

ChangeCollisionEventArg ss args );

I! Arguments for theme ChangeCollisionEventHandler delegates.

Public class ChargeCollisionEventArgs : EventArgs { /I Modified item, item extension, or relationship object. } public object Mc difiedObject { get; } _ // Properties from store. public IDictionar-y StoredProperties { get; } : } // Handler for ltemContext.UpdateProgress. ) public delegate void UpdateProgressEventHandler( itenraContext itemContext,

UpdateProgressEventArgs args );

II Arguments for thes UpdateProgressEventHandler delegate . public class Chan geCollislonEventArgs : EventArgs . { // The current updaate operation. oo public UpdateOp eration CurrentOperation { get; } // The object that i=s currently being updated. public object Cur-rentObject { get; } } ’ // Specifies how the objects retumed by a search relate to the objects identity map // maintained by the ItemGontext. public enum Searc=hCollisionMode { /l indicates that new objects should be created and returned. Objects representing the // same item, item wextension, or relationship in the identity map are ignored. If this // option is specifie«d the SearchColiision event will not be raised.

DoNotMapSearch Results, // Indicates that obj ects from the identity map should be retu med. If the content of // an object has beean modified by the application, the modifiead object's content is / preserved. If the «object has not been modified, its content is updated with the // data returned by the search. The Application may provide an handler for the // SearchCollision eavent and selectivly update the object as clesired.

PreservelModified«Objects, // Indicates that the abjects from the identity map should be returned. The content // of the object is updated with the data retumed by the search, even if the object

If has been modifiosd by the application. if this option is specified the Search-

J/ Collision event wi Il not be raised.

OverwriteModifiedi Objects } // The current updater operation. public enum UpdateOperation

{ // Provided wikhen Update is first called. CurrentObje=ct will be null.

OverallUpd a—teStarting, // Provided jusst before Update returns after a succe=ssful update. CurrentObject wil | be // null.

OverallUpd ateCompletedSucessfully, // Provided jusst before Update throws an exception. CurrentObject will be the exception // object.

OverallUpd aseCompletedUnsuccesstully, /1 Provided wien the update of an object is started. CurrentObject will reference th e // object that v=vill be used for the updated.

ObjectUpdateeStaring, // Provided wwimen a new connection is needed. CurreantObject will be a string that ceontains // the path idemntifying an item domain as passed to I-temContext.Open or retrieved #rom /1 the Locatiorm field of a relationship.

OpeningCo nmnection } } . [Remainder of Page Intentiomnally Left Blank]

APPENDIX B namespace System.Storagge { /l Executes a search across a specific type in an item context. public class ItemSearchem {

Constructors public temSearcher(); public ltemSearcher( Typoe targetType, itemContext context ); public ItemSearcher( Tygoe targetType, itemContext context, params SeamchExpression[] filters );

Properties /l The filters used to identi#¥y matching objects. public SearchExpressiomnCollection Filters {get;} // The temContext that sp ecifies the domains that will be searched public itemContext tem& ontext {get; set;} // The search parameter c«llsction. public ParameterCollect#on Parameters {get;} // The type the searcher wr ill operate against. For simple searches t his is the type of // the object that will be retumed. public Type TargetType {get; set;}

Search Methods // Find objects of TargetTy pe that satisfiy the conditions specified b y Filters. Retums ’ /f an empty FindResult if reo such objects exist. public FindResuilt Find A 11(); a public FindResult FindA 11{ FindOptions findOptions ); i public FindResult FIndA 11( params SortOption{] sortOptions ); // Find any one object of T argetType that satisifies the conditions sgoscified by Filters. // Returns null if no such object exists. public object FindOne(); public object FindOne( FindOptions findOptions ); public object FindOne( params SortOption[] sortOptions ); // Find the object of Targe¥ Type that satisfies the conditions specified by Filters. // Throws ObjectNotFounclException if no such object was found. Throws MultipleObjects- / FoundException if more then one object was found. public object FindOnly() 5 public object FIndOnly( FFindOptions findOptions ); /l Determine if an object off TargetType that satisfies the conditions specified by /! Filters exists. public bool Exists(); // Creates an object that can be used to more efficiently execute thea same search // repeatedly. public PreparedFind Pre pareFind();

public PreparedFind P®repareFind( FindOptions findOptions ); public PreparedFind P*repareFind( params SortOption[] sort@ptions ); // Retrieves the number of records that would be returned by FinadAll(). public int GetCount(); 1] Asynchronous versiors of various methods. public IAsyncResult B eginFindAll{ AsyncCallback callback, obeject state ); - public IAsyncResult B eginFindAll( FindOptions findOptions _,

AssyncCallback callback, obmject state ); public IAsyncResult B eginFindAll{ SortOption([] sortOptions ,

AssyncCallback callback, obwject state ); public FindResuilt End FindAll{ 1AsyncResult ar ); : public IAsyncResult B eginFindOne( AsyncCaliback callbacic, obmject state ); public IAsyncResult B eginFindOne( FindOptions findOption s,

AssyncCallback callback, obwject state ); - public IAsyncResult B eginFindOne( SortOption[] sortOption s,

AssyncCallback callback, obm]ect state ); public object EndFindeOne( [AsyncResult asyncResult ); public IAsyncResult B-eginFindOnly( AsyncCallback callback, oboject state ); public IAsyncResult B-eginFindOnly( FindOptions findOptiores,

AssyncCallback callback, object state ); public IAsyncResult B«eginFindOnly( SortOption[] sortOptiors,

AssyncCallback callback, oboject state ); public object EndFind®Only( 1AsyncResult asyncResult ); public lAsyncResult BeeginGetCount( AsyncCallback callbac K, olboject state ); public int EndGetCourmt( IAsyncResuit asyncResult ); public JAsyncResult BeeginExists( AsyncCallback callback, obj ect state ); public bool EndExists(C IAsyncResult asyncResult ); // Options used when exe-cuting a search. public class FindOptiorss

{ public FindOptions(); public FindOptions( params SortOption[] sortOptions ); / Specifies if delay loadable fields should be delay loaded.

. public bool DelaylLoad {get; set;} /! The number of matches that are returned.

public int MaxResults {get; sset;} /I A collection of sort options. public SortOptionCollectlon SortOptions {get;}

}

// Represents a parameter name and value.

public class Parameter

{

x // Initializes a Parameter object with a name and value. . public Parameter( string name, object value ); : // The parameter’s name.

public string Name {get;} // The parameter's value. public object Value {get; setz}

}

/! A collection of parameter nam ejvalue pairs.

public class ParameterCollectlon : ICollection

{ public ParameterCollection(p; public int Count {get;} public object this[string namee] {get; set;} public object SyncRoot {get; } public void Add( Parameter parameter ); public Parameter Add( strincg name, abject value ); public bool Contains( Param eter parameter ); public bool Contains( string name ); public void CopyTo( Parameter] array, int index ); void ICollection.CopyTo( Array array, int index ); {Enumerator IEnumerable.GestEnumerator(); public void Remove( Paramester parameter ); public void Remove( string rmame );

// Represents a search that #has been optimized for repeated exeacution. public class PreparedFindl { . public [temContext ItemContext {get;} public ParameterCollectieon Parameters {get;} public FindResult FindAl 1(); public object FindOne(); public object FindOnly(); public bool Exists(); ) // Specifies sorting options umsed in a search. public class SartOption { // Initialize a object with default values. public SortOption(); // Initializes a SortOptions Object with SearchExpression, order— public SortOption( SearcEhExpression searchExpression, SsortOrder order ); // A search SearchExpress don that identifies the property that will be sorted. . public SearchExpression Expression {get; set;} // Specifies ascending or deascending sort order. public SortOrder Order {cyet; set;} } // A collection of sort option objects. public class SortOptionCo=llection : IList { public SortOptionCollectilion(); public SortOption this[int: index] {get; set;} public int Add({ SortOptiomn value ); public int Add( SearchExpgpression expression, SortOrder o rder ); int List.

Add( object value= ); public void Clear(); public bool Contains( Sor-tOption value ); bool IList.Contains( object value ); public void CopyTo( SorteOption[] array, Int Index ); void ICollection.CopyTo( Array array, int index); public int Count {get;}

IEnumerator IEnumerable.GetEnumerator(); public void Insert( int Index, SortOption value ); void IList.Insert( int index, object value ); public int Index Of( SortOption value ); int IList.IndexOf( object value); public void Remove( SortOption value ); void IList.Remove( object va lue ); public void RemoveAt( int In dex ); . public object SyncRoot {get;} } // Specifies the sort order using #n a SortOption object. public enum SortOrder {

Ascending,

Descending

APPENDIX C namesspace System.Storage { publBc abstract class FindResult : IAsyncObjectReader { pubslic FindResult(); // Moves the FindResult to the next position in the result. pubslic bool Read(); pubrlic lAsync Result BeginRead( AsyncCallbauck callback, object state ); pub» lic bool EndRead( IAsyncResult asyncRessult ); // The current object. pubs lic object Current {get;} // Returns whether or not the FindResult contains any objects. pub» lic bool HasResults {get;} // Rexturns whether or not the FindResuilt is closed. pub lic bool IsClosed {get;} // Returns the type of items in this FindResult. pub lic Type ObjectType {get;}

Il Closes the FindResult pub Ric void Close(); voidll IDisposable.Dispose(); . // Retumns an enumerator over the FindResult, staarting at the current position. Advancing © // any enumerator on the FindResult advances al 1 enumerators as well as the FindllResult ll itself.

IEnumerator IEnumerable.GetEnumerator(); pubBic FindResultEnumerator GetEnumerator«); -} publi abstract class FindResultEnumerator : |EEnumerator, IDisposable { public abstract object Current { get; } - public abstract bool MoveNext(); public abstract void Reset(); pub®ic abstract void Close(); " vold IDisposable.Dispose(); } } names pace System { /l A common interface for iterating over objects. publics interface 10bjectReader : [Enumerable, Disposable

WOOP 2005/029313 PCT/US2€0D03/026144

Cd object Current {get;} bool ksClosed {get;} bool HasResults {get;}

Type ObjectType {get;} bool BRead(); void Close(); } // Adds asynchronous methods to IObjectReader public Interface IAsyncObjectReader : IObjectReader {

IAsyracResult BeginRead( AsyncCallback call ®ack, object state ); bool EEndRead(1AsyncResult result ); } } [Remainder of Page In®entionally Left Blank]

Claims (1)

1. What is Clai med:

   1. A data store comprising at least one of each of an Item, an Element, ard a Relationship, wherein: said I€em is a unit of data storable in a daata store and further comprisess said Element and said Relationship; said BE-lement is an instance of a type conprising one or more fields; arid said R-elationship is a link between at lea st two Items.

   2. The data store of claim 1 further comprising a plurality of Items, said plurality of Items comprising an Item Folder and at least one other Item that is a member =of said Item Folder.

   3. The daata store of claim 1 further comprising a plurality of Items, said plurality of Items comprising a Category and at least other one Item that is a member of said Category. 4, The daxta store of claim 1 wherein a Relat—jonship between two Items is established automatically by a hardware/software interface system.

   5. The data store of claim 1 wherein said EL ement is understandable by a hardware/soft~ware interface system.

   6. The data store of claim 1 further compris-ing a second Element, and wherein said Relationship comprises said second Element.

   7. The deata store of claim 1 further comprising a Core Schema to define = set of Core Items by whicsh a hardware/software interface sysstem understands and directly= processes said set of Core Items in a predetermined and predictzable way.

   a storage platform for managing said data store and for manipulating said pluralitsy of Items

   13. The= computer system of claim 12 wherein each Item from among said plurality of Items belongs to a.t least one Item Folder from among a plurality of Item Folders, and wherein e ach said Item may belong to more than one Item Folder from amon g said plurality of Item Folders.

   14. The= computer system of claim 13 wherein the deletion of said Item Folder does n-ot automatical ly result in the deletion of said Item.

   15. The= computer system of claim 13 wherein an Item is automatically deleted when it no longer belongs to amny Item Folder.

   16. The= computer system of claim 13 wherein said Item is automatically deleted whem itis a member of eonly one Item Folder and said Item Folder is deleted.

   17. The= computer system of claim 13 wherein an Item is automatically a member of a default Item Folder.

   18. The computer system of claim 13 wherein said Item, when it is a member of only” one Item Folder and ssaid Item Folder is deleted, automatically becomes a member of a default Iten= Folder.

   19. A method for organizing Items in a data store, said [tems comprising (a) a discret e unit of information that can be manipulated by an operating system, (b) at least one Element, andl (c) a Relationship with at least one other Item; wherein an Item can be a member of at least tweo Item Folders but is not owned by any of said Item Folders such that the deletion of any of said Item Folders does not automatically result in the deletion of said Item. Amended 22 March 2007

   8. The data store of claim 7 wher ein each Item from the set of Corre Items is derived (directly or indir-ectly) from a Common Single Basse Item. 9, The data store of claim 8 wher—ein said Common Single Base Iteem is a foundational Item. in a Base Schema.

   10. A computer-readable medium with computer-readable instructions for a data store compmrising at le ast two Items, said Items each comprising at least one Element, an said Items each sharing a Relationship with at least one other Ite m.

   11. The computer-readable medium of claim 10 further comprisings: instructions for said data store to store at least one of cach an Item, an Element, and a Relationship; instructions for said Item to fu rther comprises said Element anc said Relationship with said data store; instructions for said Element te comprise a type of one or more= fields; and instructions for forming a Relationship between two Items.

   12. A computer system, said computer system comprising: a plurality of Elements where ~each Element from among said pslurality of Elements consstitutes an instance of a type comprising one o- r more fields; a plurality of Items where eackm Item from among said plurality- of Items constitutes a discrete storable unit of information that can bes manipulated by a hardware/soft-ware interface system, arad whe rein each said Item comprises at le=ast one Element; a plurality of Relationships wi—ere each Relationship form among said plurality of Relationships is a link between at least: two Items; a data store, said data store comprising said plurality of Item, s=aid plurality of Elerments, and said plurality of Relatio nships; a4 Amended 22 March 2007

   20. The method of claim 19 wherein the Item is a member of an Item Folder but is not owned by said Item Folder, such that the deletion of said Item Folder does not automatically result in the deletion of said Item.

   21. The method of claim 20 wherein the Item is automaticak ly deleted when it no longer belongs to any Iterm Folder.

   22. The method ©f claim 20 wherein said Item, when it no leonger belongs to any Item Folder, automatically becomes a member of a default Item Folder.

   23. The method ©f claim 20 wherein the Item is automatically deleted when it is a member of only one Item Folder and said Item Folder is deletecl. 24, The method ©f claim 20 wherein said Item, when it is a member of only one Item Folder and said Iterra Folder is deleted, automatically becomes =a member of a default Item Folder.

   25. A computer system comprising: a plurality of Items comprising at least one Item, where each of said plurality of Items constitute a discrete storable unit of information that can be manipulated by a hardware/softwaure interface system; a plurality of Item Folders comprising at least one Item Folder, wherein said plurality of Item Folders constitute an organizational structure for sa-id Items; and

   . a hardware/s oftware interface system for manipulating said plurality of Items, wherein each of said plurality of Items belongs to at least ome of said plurality of Item Folders, and wherein each of said plurality of Jiems may be-long to more than one Item Folder of said plurality of Item Folders.

   26. The computer system of claim 25 wherein an Item iss a member of an Item Folde-r but : * 18 not owned by said Ite am Folder, such that the deletion of s=aid Item Folder does not . automatically result in t The deletion of said Item.

   27. The computer sy-sstem of claim 25 wherein an Item i=s automatically deleted wher it no “longer belongs to any It=em Folder.

   28. The computer sy~stem of claim 25 wherein said Item is automatically deleted whe=n it is a member of only one= Item Folder and said Item Folder iss deleted.

   29. The computer system of claim 25 wherein each Item is a member of at least one Item Folder but is not owned by said Item Folder, such that the d_eletion of said Item Folder d_oes not automatically result in the deletion of an Item.

   30. The computer sy~stem of claim 25 further comprising a plurality of Categories comprising at least one &Category, wherein said plurality of ® Categories constitute an organizational structure for said Items.

   31. The computer sy~stem of claim 30 wherein a Categor=y is defined by an Item propwerty.

   32. The computer sy~stem of claim 31 wherein one of said plurality of Categories is defined by an Item prop erty, and only an Item comprising tie Item property for a specifmic Category from among s=aid plurality of Categories can be a muember of said specific Cate=gory.

   33. The computer sy~stem of claim 30 wherein each of said plurality of Categories is defined by an Item prop erty.

   34. The computer sy~ stem of claim 33 wherein each of said plurality of Categories is defined by an Item prop -erty, and only Items comprising the Item property for a specific Category from among s =aid plurality of Categories can be me=mbers of said specific Category.

   . 35. A hardware/software interface systera capable of manipulating an Iteem, said Item comprising a discrete unit of information commprising a basic set of propertiees commonly supported across objects exposed by an oper—ating system shell.

   36. The hardware/software interface systzem of claim 35 wherein said Iteem is a fundamental unit of information manipulatecd by an operating system.

   37. The hardware/software interface system of claim 36 wherein said Item is a member of an Item Folder. :

   38. The hardware/software interface system of claim 37 wherein said Item is not owned by s aid Item Folder, such that the deletion off said Item Folder does not autaematically result in the deletion of said Item.

   39. The hardware/software interface syst_em of claim 38 wherein said Item is autosmatically deleted when it no longer belo-ngs to any Item Folder.

   40. The hardware/software interface syst em of claim 38 wherein said Item is autoematically deleted when it is a member oXf only one Item Folder and said_ Item Folder is “dele ted.

   41. A computer-readable medium comprising computer-readable instructions for an Item, said Item comprising a discrete unit of inforrnation that can be manipulated by a hard ware/software interface system.

   42. A computer-readable medium comprising computer-readable instructions for a hard ware/software interface system, said operating system comprising:

   means for manipulating a plurality of Items comprising at least one Item, where each of said plurality of Item s constitute a discrete unit of inforrrmation that can be manipulated by a hardware/software interface system; means for manipulating a plurality of Item Folders comprising at least one Item Folder, wherein said plurality of Item Folders constitute an organizational structure for said Items; and wherein each of said plurality of Items belongs to at lea=st one of said plurality of Item Folders, and wherein each of said plurality of Items may be long to more than one Item Folder of said plurality «f Item Folders.

   43. A computer-readabl € medium with computer-readable instructions for a hardware/software interfaces system for a computer system, wherein said hardware/software interface system manipulates a plurality of discrete units of infomation (“Items”), said Items interconnected by a plurality of Relationships managed by said hardware/software interface system.

   44. The computer-reada ble medium of claim 43 wherein a £iist Item has a Relationship from itself to a second Item .

   45. The computer-reada ble medium of claim 44 wherein said first Item is an Item Folder. . 46. The computer-reada ble medium of claim 45 wherein said second Item is an Item Folder.

   47. The computer-reada ble medium of claim 45 wherein samid second Item is a Category.

   48. The computer-reada ble medium of claim 45 wherein said second Item is an Item that is not an Item Folder or a C ategory.

   49. The computer-readable medium of claim 45 wherein each Ite=m from among said Itenms has a Relationship to at least osne other Item.

   50. The computer-readable medi um of claim 45 wherein a subset of Items comprise Item Folciers. ’

   51. The computer-readable medi um of claim 45 wherein a subset of Items comprise Catesgories.

   ‘52. The computer-readable medium of claim 45 wherein a subset= of Items comprise Items that are not Item Folders or Categori es.

   53. A computer-readable mediurm with computer-readable instructions for a harc3ware/software interface system or a computer system, wherein =said hardware/software interface system manipulates a plura lity of discrete units of informatmon having properties und-erstandable by said hardware/sof tware interface system (“Items”).

   54. The computer-readable medium of claim 53 wherein said har—dware/software interface syst-em comprises a base schema to clefine at least one of an Item anc at least one of a propoerty. )

   55. The computer-readable medium of claim 54 wherein at least -one of an Item in the bases schema is a foundational Item, «constituting a foundational Item type, from which all othezr Items manipulated in the hardvware/software interface system amre derived.

   56. The computer-readable medium of claim 55 wherein said foumndational Item type comprises a property for referencing at least zero Categories to which said Item is 2 member.

   57. The computer-readable medium of claim 55 wherein said feoundational Item type comprises a property for a unique identification of said Item in a h-ardware/software interface system.

   58. The computer-readable medium of claim 54 wherein at least one of a property in the base scherna is a foundational property, constituting a foundational property type, from which all other properties utilized in the hardware/software interface systeem are derived.

   59. The computer-readable medium of claim 54 wherein: at least one of an Item is the base schema is a foundational Item, constituting a foundational Item type, from which all other Items manipulatecd in the hardware/software interface system are derived; and oo at least one of a property in the base schema is a foundatioral property, constituting a foundational property type, from which all other properties util ized in the hardware/software interface system are derived.

   60. ' The computer-readable medium of claim 59 wherein the bamse schema further comprises a second Item deri ved from the foundational Item type ( the first Item), said second Item constituting the foundational type for an Item Folder, and sail second Item expressing a Relationship to said first Item.

   61. The computer-readable medium of claim 59 wherein the bawse schema further comprises a second property derived from the foundational property type (the first property), said second property constituting the foundational type for an idun=it key property.

   62. The computer-readable medium of claim 61 wherein the ba_se schema further comprises a third property derived from the second property, said third property constituting the foundational type for a Category reference.

   VYVO0 2005/029313 PCT/"US2003/026144 (eX) The computer-readable medium of claim 59 wherein the base schema further comprises a s econd property derived from the foundational property type (the first property), said s-econd property c onstituting the foundational type for Categories.

   64. The computer-readable medium of claim 53 wherein said hardware/software imnterface system Comprises a core schema to define a set of core Items which said hardware/software interface system uanderstands and can directly process in a predetermined and predictable way.

   6s. The computer-readable medium of claim 64 wherein each Item from the set off core Items 1s Aerived (directly or indirectly) from a common single base Item.

   66. The computer-readable medium of claim 65 wherein said common single base= Item 1s a foundational Item in a Base Schema.

   a. The computer-readable medium of claim 66 wherein said core schema further defines a set of core properties which said hardware/software interface system understands and can dimrectly process in a predetermined and predictable way.

   68. The computer-readable medium of claim 67 wherein each property from the seet of core Items i s derived (directly or indirectly) from at least one base property.

   69. The computer-readable medium of claim 68 wherein the core schema compris—es an Item for devices.

   70. The computer-readable medium of claim 68 wherein the core schema compris-es an Item for events. 71 The computer-readable medium of claim 68 wherein the core schema compris-es an Item for commodities. 72 The computer-readable medium of claim 68 wherein the core schema compris. es an Item for rIessages.

   73. The computer-readable medium of claim 68 wherein the core schema compris es an Item for principals. Amended 22 March 2007

   74. The computer-readable medium of claim 68 wherein the core schema ecomprises an Item for locat ions.

   75. The computer-readable medium of” claim 68 wherein the core schema -comprises an Item for docu. ments.

   76. The computer-readable medium of” claim 68 wherein the core schema comprises an Item for statements.

   77. The computer-readable medium of” claim 68 wherein the core schema comprises an Item for contacts.

   78. The computer-readable medium of” claim 68 wherein the core schema comprises a property for a certificate.

   79. The computer-readable medium of claim 68 wherein the core schema comprises a property for a primncipal idunit key.

   80. The computer-readable medium of claim 68 wherein the core schema comprises a property for a posstal address.

   81. The computer-readable medium of” claim 68 wherein the core schema comprises a property for arichh text element.

   82. The computer-readable medium of” claim 68 wherein the core schema comprises a property for an el ectronic address.

   83. The computer-readable medium of claim 68 wherein the core schema comprises a property for an id_unit security package.

   84. The computer-readable medium of claim 68 wherein the core schema comprises a Relastionship for occupying a role between two Contacts. Amended 22 March 2007

   85. The computer-readable medium of clairm 68 wherein the core schema comprisses a property for a basic presence.

   86. A method for manipulating a plurality of discrete units of information ("Items ") ina hardw zare/software interface system for a computer system, said method comprising inmterconnecting said Items with a plurality of Relationships and managing said Relationships at the hamrdwarc/software interface system level.

   87. The interface system of claim 86 wherein each Relationship from among said plurality of Relationships constitutes, at the hardware/softw are interface system level, a mapping Wbetween a pair of Items that said Relationship interconnects.

   88. The interface system of claim 87 wherein each Relationship has properties.

   89. The interface system of claim 88 wherein each Relationship comprises a prop erty (Target) for the identification of said Relationship's target Item.

   90. The interface system of claim 89 wherein each Relationship further comprises a property (IsOwmed) for the ownership of said Relationshiap's target Item. Amende. d 22 March 2007