Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/21—Design, administration or maintenance of databases
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/23—Updating
  • G06F16/2358—Change logging, detection, and notification

Definitions

• the present invention relates to the field of persistent data management of an entity, for example a company.
• the present invention relates to the monitoring of this persistent data in a database by means of a Database Management System. It is indeed difficult for a company to guarantee the monitoring of the evolution process of strategic persistent data because this monitoring presents some objective obstacles:
• the prior art also knows, from PCT patent application WO 02/27561 (Oracle), a system and a method for providing access to a temporal database.
• the invention described in this document relates to a system and method for selective visualization of data in temporary rows in a constant reading database.
• the saved transactions causing changes in the data in rows of a database are tracked and a change number of the stored system is assigned to each saved transaction.
• a requested selection of data values in rows from the database is performed as well as an interrogation time taking place before the save time of at least one saved transaction.
• the values of the data in ordered rows contained in the cancellation segments storing a transaction identifier for at least one saved transaction are recovered.
• access to a pending recording means that data relating to fields which differ from the corresponding fields of the up-to-date recording are superimposed on such up-to-date recording fields but with a video or video attribute. character different from the video attribute or default character.
• a plurality of historical or pending records can be composed so that all fields modified for a record set from the end of a defined period can be superimposed on an updated record at one time.
• European patent application EP 0 984 369 is a mechanism for storing dated versions of data.
• the data is stored as a plurality of records, each record comprising at least minus an attribute, a time marker indicating the duration for which the attribute is valid, an insertion time indicating when the record was created and a type field.
• the type field indicates whether the record is a concrete record, a delta record, or an archived record replacing one or more records which have been archived.
• Data is accessed to find an attribute value from the point of view of a "specified time", by extracting the records which have insertion times earlier than the "specified time” and constructing an attribute value from the extracted records.
• Data is updated only by adding concrete records or delta records, without changing attribute values in concrete records or delta records.
• the present invention intends to remedy the drawbacks of the prior art by proposing a method for monitoring the evolution of data in an architecture based on a DBMS, consisting of:
• the main object of the invention is to allow the exploitation of the data of the base according to successive versions, while limiting the needs for time and storage capacity and to authorize the restitution on the fly.
• a usual approach consists in recording successive versions of the databases, for example in the form of periodic storage on a medium such as a magnetic cartridge of the entire database, in the state corresponding to the current version. .
• the search for information requires restoration prior to the entire database, from the medium corresponding to the corresponding backup, then to interrogate the base thus restored.
• the volume corresponding to a state can exceed the terabyte, which volume should be multiplied by the number of saved states.
• the invention aims to respond to the technical problem of real-time operation of large volume databases.
• the invention relates in its most general sense to a method of organizing a digital database in a traceable form, comprising steps for modifying a main digital database by adding or deleting or modifying of a recording of the main database and of the steps of reading the main database, characterized in that the step of modifying the main database comprises an operation of creating at least one digital recording comprising at least : the unique digital identifiers of the records and attributes concerned in the main database, a digital identifier of the state of the main database corresponding to said modification of the main database, the elementary values of the attributes which are assigned to them affected through elementary operations, without storing attributes or records unmodified, and adding said record to an internal history database composed of at least one internal history table, and in that the reading step relating to any final or previous state of the main database consists in receiving (or intercept) an original request associated with the unique identifier of the targeted state, to carry out a transformation of said original request to construct a modified request for addressing the historical database comprising the criteria of the original request and the identifie
• said records from the historization database also contain references to other records from the internal database, with the aim of specifying the dynamic dependency links of source-destination type constituting the causal flow of interference between versions of data.
• said operation for modifying the main base is a logical operation and said operation for adding to the history database consists in adding: a record identifying the state of the base corresponding to the logical operation, as many 'records as parameters of the logical operation, a record for the possible result of the logical operation and to specify by a family link the groupings of operations from the elementary level of modification to the level of the transaction, passing the number of semantic levels necessary for the applications.
• the main database contains one or more table (s), organizing the evolution links between the identifiers of the successive and alternative states of the main database, intended to organize the records of the database. internal data.
• said one or more tables of evolution links between the states of the main database contain records specifying the rules for correspondence between the records of the internal historization database and the states of the main database.
• said read operation consists in determining said state of the main database by referring to said identifiers and to the tables of evolution links between the states of the main database.
• an application interrogating the main database can specify the state of the desired main database.
• the invention also relates to a database management architecture characterized in that said application can make modifications on any state of the main database and giving rise, in the case of the attempt to modify a state prior to, to the creation of new alternatives for base development main data whose data will be generated by the same internal historical database.
• the dependency links serve as criteria for questioning said operations already carried out.
• the updates carried out on different branches can be integrated or merged in the context of a new state "inheriting" from said branches.
• the cases of evolution of data structure of the main database are treated as particular cases of evolution of the data of said database, provided that the structure / schema of said main database be described as mentioned for the data, as a dictionary.
• the historization database is explored and interrogated by applications through the native mode of the DBMS in order to obtain information such as for example all the historical values of an attribute and all the incidences
• FIG. 1 illustrates a conventional architecture of communication between an application and a database
• FIG. 2 illustrates a communication architecture similar to that of FIG. 1 and comprising the elements necessary for the application of the invention
• FIG. 3 illustrates the different means of access to a traceable organized database provided with a system according to the invention.
• the management of persistent data of a company is generally entrusted to specific software also called DBMS.
• Computer applications offer users interactive ergonomic means capable of visualizing and changing the data in the company's database by communicating with the DBMS.
• Relational DBMS (but also object, network or hierarchical) on the market are good candidates for the role of persistence manager. This compatibility is also an advantage of our process which can also take advantage of the software base installed in the company.
• a Relational DBMS This allows the representation of data in the form of tables (or relationships).
• the columns indicate the attributes (or fields). Each column is characterized by a field (integer, character, date, float, etc.) and other possible information such as the maximum size (for character strings).
• Certain attributes constitute the key or the identifier of the record.
• each row of the same table represents a new record (or tuple) of uniform structure.
• Each cell represents the value of the attribute. For example, "aaa” is the value of the Attributl attribute of the first record whose key is 1001. Table
• the data is inserted, read, modified and deleted through a data manipulation language (SQL for example).
• SQL data manipulation language
• the persistence manager also allows the definition, the consultation and the evolution of the data structure, also called data schema.
• the tables can be defined, deleted or restructured. In the latter case, columns can be added or removed. Sometimes it is even useful to change the domain of an attribute, or other similar characteristics, which may involve implicit or explicit processing of the conversion of the data concerned.
• the table is the logical reference for data representation.
• applications generally "see” data in the form of tables. It is important to emphasize that our system wants to preserve this logical representation in order to ensure the greatest compatibility with existing applications. For example, after having requested the connection to a particular database, an application can address a persistence manager with a request of type "select * from client" and receive in exchange a set of data allowing the reconstruction of the data in tabular form.
• a database represents a coherent state of the real world represented.
• the data in the database evolves in jerks triggered by events through operations (insertion, update or deletion) generally grouped into transactions. These are characterized by special properties called ACID (Atomicity, Coherence, Insulation and Durability) which guarantee a certain level of quality.
• ACID Automaticity, Coherence, Insulation and Durability
• Ensuring the traceability of persistent data amounts to providing means allowing upstream and downstream monitoring of the data evolution process.
• the first level of traceability which can be described as elementary, is that of the representation and storage of data. It is therefore a question of describing the structure, then of storing and identifying the data, whether it is an order, an article or even a mechanical part, in order to be able to find it later. .
• This type of functionality is already provided by specialized software, called Database Management Systems (DBMS).
• DBMS Database Management Systems
• the process of evolution is manifested by the successive application of elementary operations such as reading, insertion, updating and deletion. These elementary operations are generally grouped into transactions in order to maintain the consistency of the data under conditions of concurrent use or even fault recovery.
• the updates have as a natural consequence the loss of existing values following their replacement by new values, since - by convention - to the same identifier can only correspond to a single datum (with its attributes).
• This first so-called elementary level of traceability is essential but largely insufficient.
• the second level of traceability authorizes data to have several versions at the same time (separate values). This improves traceability since it becomes possible to have at any time both the values preceding and those following the execution of an operation or a process, which makes it easier to understand the evolution.
• the version introduces a precious quality, since irreversibility is no longer inescapable (data evolution is allowed, without loss of current values). In addition to successive versions, there are alternative versions.
• a third level of traceability is that of operations. Tracing an operation is like leaving a persistent trace of the execution of said operation, allowing an even better understanding of how the data evolves. We can thus better explain the evolution of an order between two versions, if we know for example that there was a discount operation on the total price.
• Most DBMS have logging mechanisms that allow viewing of operations performed at the elementary level. To be understandable by users, this information must be correlated with high-level operations, but the basic problem is that the log entries do not have the same persistence cycle as the data. Thus, the log is generally located outside the database and is regularly purged by the administrator.
• PCT patent application WO 02/27561 provides an alternative solution to this problem, by proposing internal storage (in the database) of transactions and cancellation information of their effects (undo), which allows to recover any previous state of the database by executing in reverse order the reverse of the operations which have taken place since.
• this technique can be very expensive in terms of execution time because, to find a precise version of a data, it undoes all the operations that take place since, including those that do not concern it.
• it is also not suitable for obtaining the list of all the versions of a data item.
• the inventors have opted for a strategy opposite: upon receipt of a request, the present invention proceeds to the transformation thereof then to an execution on the versioned data.
• the need for higher level information for example provided by applications, in order to obtain an articulation between the semantics of the applications (application of a discount on an order) and that of the DBMS (update day of the attribute "amount" of the order).
• the most advanced level of traceability is that of causality. It aims to materialize information transport links at the most basic level (the finest grain).
• a particular case of evolution operation concerns the evolution of the schema which consists in changing the structure of the data without loss of information
• One of the objectives of the present invention is to propose a slightly intrusive and progressive method of organizing a digital database in a traceable form. We aim to ensure the successive levels of traceability described above, without imposing the redesign of existing applications.
• the objective pursued by the invention is to provide computer applications and their users with the ability to accurately track data throughout its evolution, by tracing their stories in a comprehensive manner, as well on the individual level (intermediate versions and links of succession) than on the collective level (triggering events and dynamic links of interdependence resulting from the interactions between the versions of the data), in the positioning it within the coherent framework of its original development.
• the method according to the invention uses a flexible architectural framework, as unconstraining and intrusive as possible in order to provide very broad applicability to the proposed method and as wide compatibility as possible with the methods of storing and manipulating current data.
• the method according to the invention makes it possible to ensure that it represents not only one but all the successive and / or alternative coherent states. necessary from the real world represented in its evolution, while preserving the ACID properties.
• journal organized in the form of an “internal historization database” consisting of a table or a set of tables dedicated to monitoring evolution and based on a universal storage mode with a stable schema (independent of the logical representation of application data) and particularly suitable for on-the-fly data reconstruction.
• a transaction and event monitor capable of detecting any request for changes in values and structure transmitted to the database, which gradually adds entries to the dedicated log characterizing the elementary evolution of the data (identity, attribute, value, triggering event and dynamic dependencies) a reconstitution module (R) on the fly of the state of the database according to a target event; the system is provided for this purpose with a cursor (C) dedicated to the selection of the desired state.
• R reconstitution module
• C dedicated to the selection of the desired state.
• special case in some cases, it may be useful to materialize the view of the so-called "current" or "main” base in the form of specialized structure tables, for example to allow high performance and total compatibility with existing applications (in particular to allow the use of stored procedures and other triggers or triggers that an application may require to function properly).
• the architecture also includes:
• the event log (J) (or the internal history database) is made up mainly from a table with a structure independent of that of the application data.
• the columns are:
• the role of the monitor (M) is to correctly detect and interpret each change request by adding the corresponding information to the event log (J).
• each event that tends to modify the logical database ends up creating one or more entries in the form of new lines (or records) in the log. This ensures that nothing is lost and that any logical deletion or update does not result in physical deletion. Thus, data from the past can be recovered.
• One of the advantages of this organization is the concurrent creation of views such as account books which generally block the update access of other users.
• the uniformity of the information storage structure the data is stored in identical fashion, whether it is the evolution of values or that of structures. This means that from a logical point of view, it is possible to reconstruct both logical tables and their structures, on the basis of the same mechanism.
• the fact of including the journal in the same database as the main database makes it possible to guarantee its relative coherence by the transaction mechanism provided by the DBMS.
• the reconstruction module (R) is in charge of restoring the data in logical format according to an event type parameter, from the event log (J).
• the journal can accommodate invocations of operations. This can be done by representing the operations under the form of logical tables, where each operation corresponds to a logical table name and each argument corresponds to a logical attribute.
• the application can send to the journal (for example, via an API: "Application Programming Interface") the information necessary for the traceability of calls to operation analogous to the manipulation of logical data (but this task can be automated and entrusted to a post-processor, to the compiler, to the processor or even to the virtual machine).
• transaction validation points can be plotted in the form of operations. Indeed, it is recommended that the cursor is positioned exactly on these points and not between two operations of the same transaction. The consistency of the result depends on it.
• applications such as design assistance tools may very well benefit from intermediate states, deemed to be incoherent, for explanatory purposes, and thus benefit from mechanisms of the “long transactions” type.
• the invention also relates to the materialization of causal links.
• attribute 4 was formed from attributes 2 and
• the implementation of the sources in the journal can very well be carried out by the intermediary of an additional journal (or sub-table), organized in a tabular manner, and this for reasons of performance optimization, according to the techniques in force. in the discipline of databases.
• the validity check of operations can be carried out against the data in force. For example, if the value of the Attr3 attribute of Client 110 changes after the execution of the "add" operation, the results sent by this operation can no longer be considered as compliant. It is said that there is “questioning”. In the case of an evolution without alternatives, this can be verified by a simple comparison of UEID between the sources of the arguments and the last values of the referenced sources.
• This characteristic of the process also makes it possible to set up an automatic optimization system, which - on the basis of the systematic verification of the validity of the sources - makes it possible to return the result calculated previously, without actually re-executing the operation.
• the implementation of such a solution involves the introduction of references to the calling operations (which can be done through additional arguments) and provided that the verification time is less than that of execution (statistics of performances can be maintained for information purposes and exploited effectively).
• Automatic notification of "questioning” may be implemented on the basis of the information concerning the validity of the versions of the data in relation to the flows. So, for an operation, a class operation, a target or a given source, flow consistency alerters will be able to notify the applications by synchronous or asynchronous messages.
• the re-execution consists of a new explicit invocation of a given operation on the model of a previous invocation, but on the basis of new values. In all cases, it will give rise to new values for the data, the operations and the traced sources.
• the method according to the invention is specially designed to manage operationally the historization over the water and the restoration on the fly.
• the management of storage volumes is facilitated and optimized by a set of factors:
• the necessary volumes of additional storage increase linearly with the number of attributes modified or deleted and do not depend on the volumes of data inserted in the database; this factor allows very advantageous use for a very wide spectrum of applications.
• the very relevant purges can be operated according to the data marked as called into question by the source-destination type traceability links, but this operation must be controlled by the applications according to the semantics of the called into question.
• Tree structure each event has a parent event. The value of a datum associated with an event can be obtained by a logical ascent of the parents to the nearest value.
• Graph oriented without circuit analogous to the tree structure, this organization allows a version to have several different parents. The ambiguities of resolution can be resolved by predefined rules, based on criteria of priority of the branches or on any other characteristic of the data (its type, etc.)
• Virtual versions are predefined event branches that allow the creation of parallel configurations that can simultaneously benefit from events applied to one or more so-called “reference” branches.
• the architecture implemented for the realization of the invention may also include the following modules • a system for monitoring compliance (SC) of the applications with the states of the database and its diagram.
• SC system for monitoring compliance
• the principle is based on the registration of a version identifier of the application in order to declare a level of compatibility with the state or states corresponding to the schema of the main database
• the invention can be implemented in several ways depending on the context in which it is integrated into an application.
• Figure 3 presents an architecture which allows three levels of integration of traceability, from bottom to top:
• Existing applications can continue to access the (so-called “main”) database in the same way.
• the database can either keep its original structure and redirect access to an associated log (called internal database), or evolve towards a physical log-type organization and offer views or a driver responsible for translating requests and results .
• Existing applications can be very easily provided with a "cursor" provided that access to the data is centralized (which is generally the case, for example through a single driver).
• the application may offer automatic means of access to the data in the database (now implemented in the form of a log) and allow users to activate a cursor which will position the readings on the desired event marker. Slight adaptations may take place in order to match the granularity of the events with the semantics of the application.
• branches offer the possibility of creating alternatives for upgrading the database. At the same time, this raises new problems with regard to traceability. Indeed, supposing that after the separation of the branches A and B, the data X is modified in the branch A through the operation 0. One can then wish to return its new value in the branch B, as if it had had this value at the time of the separation of the branches.
• This operation called refreshing, is very useful for many cases where institutional reference data are received at more or less regular intervals.
• Their integration can then pose problems of interference with the operations carried out in the meantime. For example, if no operation having as source or destination the data X in the branch B has not been carried out in the meantime, we can calmly consider that there is no impact. On the other hand, if it is the case, it will then be necessary to decide
• Virtual branches - which are by definition permanently refreshed by their “parent” branches - automatically benefit from data refresh in their parent branches, including disconnection operations (creation of new branches) which take place (virtually, of course ) at the same time on the virtual branches. For example, if branch B is virtual, then any operation carried out on branch A is automatically reflected on branch B. In addition, if we create a new branch A2 from A, this will have the effect creation of a similar B2 sub-branch from B. It is important to underline the virtual character of these refreshments. This means that in reality no treatment is actually carried out. The only effect is that a next request on branch B will have an enriched result (which takes into account the refreshed data). Finally, note that in the event of automatic propagation, there is no automatic conflict resolution, unless rules have been predefined. In some cases, it can be decided in advance that, by default, what has been explicitly modified in the virtual branch has priority over the data generated by refresh.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Databases & Information Systems (AREA)
• Data Mining & Analysis (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=31726006

Family Applications (1)

Country Status (5)

Families Citing this family (9)

Family Cites Families (4)

• 2002
  • 2002-09-11 FR FR0211250A patent/FR2844372B1/fr not_active Expired - Lifetime
• 2003
  • 2003-09-09 AU AU2003278287A patent/AU2003278287A1/en not_active Abandoned
  • 2003-09-09 WO PCT/FR2003/002675 patent/WO2004025507A2/fr not_active Application Discontinuation
  • 2003-09-09 EP EP03769596A patent/EP1537497A2/de not_active Withdrawn
  • 2003-09-09 US US10/527,516 patent/US20060123059A1/en not_active Abandoned

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events