Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/24—Querying
  • G06F16/245—Query processing
  • G06F16/2452—Query translation
  • G06F16/24524—Access plan code generation and invalidation; Reuse of access plans
• • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
  • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
  • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
  • Y10S707/00—Data processing: database and file management or data structures
  • Y10S707/99931—Database or file accessing
  • Y10S707/99933—Query processing, i.e. searching
  • Y10S707/99934—Query formulation, input preparation, or translation

Definitions

• the present invention relates to a method for processing a request.
• the invention relates more particularly to a request and result transformation method between a client application and a plurality of heterogeneous sources of data by an adapter device of given architecture, specific to each data source, and the given architecture of adapter allowing the implementation of said process.
• a lot of useful information is locked up in private files, and manual extraction of information, even partially usable, requires a lot of time.
• the integration of new servers or databases is laborious work and the result is generally inflexible and difficult to understand.
• the main problems are accessing information where it is, in its original form, transforming it into a usable format and delivering it to the applications that need it.
• the processing of a customer's request is done by an adapter.
• a specific adapter is associated with each data source.
• the present invention therefore aims to overcome the drawbacks of the prior art by proposing a method for processing a request, common to all the adapters.
• the method according to the invention provides a uniform access interface to information sources such as relational DBMSs (DataBase Management System), system files, "Lotus Notes” databases, document management systems and "Web” servers (WorldWideWeb), for example.
• the aim of the invention is to propose a method which makes it possible to extract information by working directly, for example, from a report, invitations to tender or similar documents in heterogeneous formats such as HTML (HyperText Markup Language) or ASCII (American Standard Code for Information Interchange), to obtain an object, a publication date or for example the place of a call for tenders published on a server having a determined protocol such as HTTP ( HyperText Transmission Protocol).
• the proposed method must allow information to be filtered to select useful information, instead of adding another specific function for each application which needs the information.
• This object is achieved by the fact that the process for processing requests and results between a client and a plurality of heterogeneous sources of data by an adapter device specific to each source, of given architecture, performs the following steps independently:
• the initialization of the filter chain is carried out from the internal representation of the request and of the filter object.
• the response object is produced from the filter chain.
• the response object is returned to the client application to allow it to retrieve the results.
• metadata concerning the schema of the data in the source, the operators supported by the source and the costs associated with these operators are provided to the client application.
• Another object of the invention is to propose a given architecture adapter allowing the implementation of said method.
• the adapter comprises four modules, a coordination module serving as an interface with the client application comprising a connection component and a "response" component, a request transformation module, comprising at at least one analysis component, at least one translation component and one processing construction component, a source communication module comprising a first "source” component and a second "source access filter” component and a transformation of the result comprising third filter components all having the same interface, each module comprises components each having a determined interface.
• the same application programming interface API
• the same query language and the same data model in the form of source metadata are used to process each source.
• the transformation module of the result consists of a set of components chosen from four kinds of thirds filter components, "operator” components applying additional constraints to the response, extraction filters, structure translation filters and response construction filters transforming the response into the data format of the client application.
• analysis components and / or translation components are stacked.
• FIG. 1 represents the architecture of an adapter according to the invention and the treatment method of a request.
• the adapters (1) are based on the transformation of requests and results between an application (6) and a data source (7) by a double process of rewriting the request and transforming the result .
• An adapter is associated with a source. The adapter is responsible for giving a relational view of the source to which it is attached.
• the requests coming from the client application (6) are analyzed, then transformed into requests supported by the data source manager (7), the parameters of the request are extracted and used to guide the transformation of the result.
• the results are transformed by information extraction and restructuring processes.
• Adapters can be used directly to access data sources from applications.
• the client application (6) can be any application needing to access a data source in a structured relational form.
• the architecture of the adapter (1) according to the invention can be broken down into four main modules: a coordination module (2), a request transformation module (3), a result transformation module (4) and a source communication module (5).
• the coordination module (2) serves as an interface with the client application (6).
• the coordination module (2) communicates with the client application (6) by exchanging requests and responses with the client application (6).
• the request transformation module (3) is responsible for analyzing the request received and translating of it in operations supported by the data source manager (7).
• the result transformation module (4) is responsible for restructuring the results, translating the data types from the source into the data types of the client application (6) in general, and elaboration of the response sent to the client application (6).
• the adapters offer query features and data models based on relational database technology to provide uniform access to different data sources. This involves translating a database query into a non-database query and translating unstructured results into relational structured results.
• the communication module (5) with the source is responsible for communication with the source (7) and encapsulates responses in data structures compatible with the source in data structures compatible with the adapter (1). Its role is on the one hand, to send the request to the source and on the other hand, to recover the results that the source (7) returns.
• Each module is made up of a set of components. The functionality of each module and the various components used to build them will now be described.
• the modules making up the architecture can be classified into three different layers: the "Client” layer (8), the internal layer
• the "Client” layer (8) comprises the coordination module (2) and the “Source” layer comprises the communication module (5) with the source.
• the internal layer (9) of the adapter deals with the content, that is to say with the requests received from the client application and sent to the source (7), and the data received from the source and sent to the client application.
• the internal layer (9) comprises the transformation module (3) of the request and the transformation module of the result (4).
• the client application using this configuration file determines the adapter used.
• the "Client” layer (8) groups the components that deal with the client application at the access level. Its role is to serve as an interface with the client application.
• the components of the client layer create an interface used by the client application (6).
• the coordination module (2) of this client layer has four main responsibilities. First, a connection component (21) accepts a request (71) from a client application and passes (72) the request to the request transformation module (3). Second, a "Response" component (22) is used to transfer the result, coming from the transformation module (4) of the result, to the client application (6). Third, it delivers metadata to the client application regarding the schema of the data in the source, the operators of the query with their costs when required. Metadata is data about the structure of the data itself. Fourth, it dynamically initializes and configures the other components of the adapter.
• the internal layer (9) of the adapter groups the components which deal with data transformation.
• Two modules are associated with the two main transformations carried out which are the query transformation module (3) and the results transformation module (4).
• the main functions of the request transformation module (3) are on the one hand, to rewrite the request coming from the coordination module into a request which can be submitted to the data source (7) and on the other hand, d 'initialize the results transformation module (4) which will process the results retrieved from the source (7). This is done in three phases. First, the incoming request is analyzed to identify the elements used, during the elaboration of a request accepted by the source and during the transformation results. These elements are the names of the relation, the attributes of the projection, and the selection or join predicates.
• This step consisting in establishing an internal representation of the request, is carried out by an analysis component (31).
• the information made up of these elements is used in a translation component (32) which rewrites the request in a form which can be accepted by the manager of the data source (5).
• the information is used by a "process builder" component (33) to initialize all the necessary filters.
• the query submitted to the data source provides a set of responses which is then filtered to produce a correct response to the client application. Passing this set of responses through the filters may be required to extend or increase the request processing capabilities of a data source manager.
• the analysis of the request, by the component (33) makes it possible to initialize filters according to the needs of the client application.
• the result transformation module (4) is responsible for extracting the relevant information from the responses obtained by the communication module with the source (5). This module (4) is also responsible for carrying out transformation on these, if necessary, and then for formatting the response to the data format of the client application (6).
• This module consists of a stack of components (42 to 45) called filters. Passing through this filter chain (46) corresponds to data extraction, verification against the initial request and conversions.
• the filters of the results transformation module (4) all have the same interface including "open”, “get-next", “close” methods and manipulate the data in the same internal format. Thus, the filters can be chained in any order according to the needs of extraction, verification and conversion. There are four classes of filters.
• the so-called "operator” filters (42) apply additional constraints to the response returned by the source, corresponding to selection or projection operators for example.
• So-called extraction filters (43) carry out a chain of information extraction and structuring methods.
• a structure translation filter (44) can be used to retrieve the data schema. These three types of components can be used several times in the chain with different initializations to carry out transformations which prove impossible by using only one filter.
• a response construction filter (45) ends the chain. This filter transforms the response from the internal data format of the adapter (1) to the data format of the client application.
• the “Source” layer (10) groups the components, which manage access from the source (7), in order to send a request to the source and retrieve the results.
• a first “Source” component (51) accepts requests in the format of the data source originating from the translation component of the request (32).
• a second component “Source access filter” (52) retrieves the results that the source returns according to the protocol of the source, these results being in the form, in the format and in the coding used by the source. This component (52) transforms these results to send them to the filter chain (46) in the internal format of the internal layer (9) of the adapter.
• the coordination module (5) with the source (7) has two functions. It sends a request to the data source, for example a SQL (Structured Query Language), or HTTP request. It uses the "filter” interface of the result transformation module (4) to return the next element of the response from the data source in the adapter's internal data format.
• FIG. 1 A description of the processing by the adapter of requests received from customers will now be made with reference to FIG. 1 in which the continuous arrows with a full head represent the path of a request, of a representation of a request or of 'particular information and the arrows with broken lines and with full head represent the data flow and the arrows with broken lines and with hollow head represent the operations of creation of an object of the process.
• a client (6) When a client (6) has requests to send to an adapter (1), it opens a connection to the adapter using a "Wapper" interface (annex), and submits the requests to the adapter.
• the connection component (21) receives the request (71) and transmits the request (72) via the "Analysis" interface to the module (3) for transforming the request.
• the adapter uses the analysis component (31) of the request forming part of the module (3) for transforming the request to establish an internal representation (73, 74) of the request which it transmits to the translation component (32) in a format compatible with the interface "Request Builder".
• This internal representation (73, 74) is also compatible with the "Process Builder” interface of the processing component (33).
• the representation (74) of the request is used by the request translation component (32) to translate the initial request (71) of the client application into a format compatible with the "Source” interface. that the source can understand.
• the translated request (75) is submitted to the source through a "Source” component (51) of the source layer, and a filter object corresponding to the "Source access filter” component (52) is produced.
• This filter object (52) realizes the "Filter” interface (represented in appendix 1 by the references 18 and 181 to 184) of the result transformation module.
• This filter object (52) is the first element of the filter chain which will filter and allow to retrieve the results from the source using the methods "open” (182), “next” (183), “close” (184 ).
• the adapter uses a component “processing constructor” or “filter initializer” (33), to initialize in the filter chain, the filter or filters necessary to complete the first filtering (52) in order to produce a response compatible with the format of the adapter.
• the adapter gives this processing constructor (33) on the one hand the internal representation (73) of the request by a function which uses as parameters the results of the "Analysis” interface and on the other hand share, by the arrow referenced (76), the filter object (52) produced by the source component (51.
• This processing constructor (33) renders a filter chain (46) made up of different filter components (42 to 45)
• the final phase is the construction of a response object, corresponding to the "Response" component (22).
• This response object is produced by the connection component (21) using the filter chain (46) that the processing constructor (33) provides it via the link (77).
• the connection component (21) gives the client application (6) the response object.
• the client application uses the "Answer” interface (12) of this component (22), comprising "open”, “next” methods, "close”, to retrieve the results in the particular data format of the interface.
• the following interfaces appearing in appendix 1 and 2 make it possible to submit a request, to retrieve results and to manage an adapter in operation.
• the notation used for the interfaces is the following "vector getSelectClause (), in which the content in parentheses expresses the parameters used by the function” getSelectClause "and” vector "expresses the form of the result provided by the function used by the interface.
• This result can be in the form of a chain (string), boolean (boolean), vector (vector) or an object (object)
• the "Client” layer defines the interfaces and components allowing the use of an adapter.
• the interfaces of the "Client” layer make it possible to use an adapter locally or remotely.
• the "Client” layer (8) comprises two specific interfaces, a first interface at the same time of connection (11, appendix 1), called “Wrapper” and adapter administration (13, appendix 1), produced by the connection component (21), and a second "Answer” interface (12) produced by the "response” component (22).
• first "Wrapper” interface provides the application cl have access to the basic services of an adapter (1).
• This interface includes a first "query” method (111) which uses an object as an input argument. This "query” method allows you to receive the request from the client application using as an input argument an object representing the request issued by the client application.
• This "query” method also makes it possible to return (78) to the client application the response object (22) which provides a certain number of functions defined in the "Answer (12) interface.
• a second method (GetCapability ) (112) of the first "Wrapper" interface does not take any input argument into account.
• This second method returns a string encoding the list of query operators supported by the adapter and their conditions of use. representing the capabilities of the adapter.
• the character string describes the operators that the adapter supports and how they can be combined.
• the operators that can be used are exploration, selection, projection, union and combination operators. Each adapter supports a subset of these operators.
• a third method (getSchema) (113) of the first "Wrapper" interface takes no input argument into account and returns a character string constituting a list of types.
• Each type describes the internal representation of a class in the source data schema.
• Types are constructed using, on the one hand, basic types such as integer, real, boolean, string and, on the other hand, constructors for sets and tuples or "tuples”.
• a tuple is a list of pairs (attribute-value). The types are defined when the adapter is created.
• This third method is used to transmit to the client application the structure of the data in the source and thus allow the client application to retrieve the results.
• a fourth method (getCost) (114) of the first "Wrapper" interface (11) does not take into account any argument and returns a string encoding the cost information associated with each operator. The cost information makes it possible to evaluate the time and the memory size of the result of a query.
• the client application maintains a cost model in order to select the best execution plan, that is to say the best filter chain.
• a default cost model is defined on the client application. In order to improve this model, costs are passed from adapters to the client application using the function of the fourth method (getcost) (114).
• the second "Answer” interface (12, appendix 1) provides the client application (6) with the services to retrieve the results of the request submitted.
• This interface includes different methods.
• a first method (getAII) (121) of the second "Answer” interface takes no input and returns a set of "tuples", list of pairs (attribute-value). This function returns a set of all the tuples returned by the source and filtered by the adapter.
• the second (open) method (122) of the second "Answer” interface takes no input and produces no output. It performs the necessary initialization procedure before the first "tuple” can be sent to the client application.
• a third method (getNext) (123) of the second "Answer” interface takes no arguments and returns a "Tuple” object.
• This object sets a "tuple” which is a part of the response to subqueries passed to the response object of the "Response" component (22). If there are no other tuples in the response, it returns a "null”.
• a fourth (close) method (124) of the second "Answer” interface takes no arguments. It performs cleaning operations. It does nothing if called before an opening call. This function can be called before retrieving the last "tuple”.
• the third method getnext
• the "Client” layer (8) includes a third "AdaptaterAdmin” interface (13, appendix 1) making it possible to initialize and configure the components of the adapter according to the access parameters of the source.
• This third interface provides administration services for managing an adapter during or after initialization.
• JDBC Java DataBase Connectivity
• "Java” is a new programming language, designed by the company “Sun”, based on C ++.
• JDBC is a utility for connecting "Java” to databases.
• This fourth JDBC interface (14, appendix 1) is a standard SQL database access interface, providing uniform access to a wide range of databases. As the query is an object, this JDBC interface (14) can be used with any type of database provided that there is an underlying translation component. A JDBC manager needs to handle SQL queries, which cannot be done at the interface level.
• the adapter programmer for the analysis component (31) in the module request transformation (3) is a SQL grammar analyzer
• the adapter will be fully JDBC compliant.
• the interfaces are not the same, features found in the interfaces (11, 12, 13) of the "Client" layer described above can be found in the fourth JDBC interface.
• the various JDBC interfaces listed below, "java. Sql. Driver”, “java. Sql. Connection””java.sql.ResultSet” and “java.sql.DatabaseMetadata” are part of the JDBC standard.
• the creation of the "java. Sql. Driver” interface (141) replaces the third interface (AdaptaterAdmin) described above. The configuration must be done before connecting the database.
• the realization of the interface "java. Sql. Connection” (142) plays the same role as the realization of the interface “Wrapper” for the recovery of metadata and the submission of the request.
• the realization of "java.sql.ResultSet” (143) plays the same role as the realization of the interface "Answer” (12) to retrieve results.
• the methods to retrieve the adapter capacities and the cost information correspond to methods of the "java.sql.DatabaseMetadata" interface.
• the request transformation module (9) must contain three elements for each adapter: an analysis component (31), a request translation component (32) and a processing construction component (33) which can each be stackable. So for another client language the adapter will use another analysis component (31 '), another query translation component (32') and another processing construction component (33 '), each adapted to this other language, but uses the same methods that make up interfaces for a given component. Each has its own interface used by the coordination module (2) to finally generate an execution plan, that is to say the filter chain (46), is defined by the "ProcessBuilder” components from the file configuration and information extracted from the request by the analysis component for a received request.
• a first "Analysis" interface (15) contains the methods that a request analyzer (31) must perform.
• the query is considered something similar to SQL queries.
• the request is given to the analyzer (31) at the time of creation or with a first method “setRequest” (151).
• the format of the request depends on the interface made.
• the parser must extract different elements from the request and uses the different methods below.
• a second method “getSelectClause” (152) returns a list (name, alias), of type string, of the projected attributes ("selectclause”).
• a third method “getFromClause” (153) returns a list (name, alias) of type chain of relations ("from clause”).
• a fourth “getWhereClause” method (154) returns a predicate describing the selection condition ("where clause”). The format of the predicate depends on the implementation.
• a fifth method “getGroupByClause” (155) returns a list (name, alias), of type string, of attributes (groupby clause).
• a sixth method “getHavingClause” (156) returns a predicate using all of the operators used to select the groups (having clause). The format of the predicate depends on the implementation.
• the analysis of the request can be carried out in phases by stacking different analyzers. A stack (31, 31 ', ...) of analysis components may be necessary to solve conversion problems for example.
• a second "Request Builder” interface (16) contains the methods that a request translation component (32) must perform.
• the incoming request is defined as the output of the previously described parser ("select clause", “from clause”, “where clause”, “groupby clause” and “having clause”).
• the request is given to the translation component at the time of creation or with the "setRequest” (161) method. This method returns the value “true” if all goes well, “false” otherwise.
• the format of the translated request depends on the type of format that the source can accept.
• Another method “getTranslatedQuery” (162) returns the translation of the previously given query. In the same way as the analysis components, different translation components (32, 32 ', ...) can be stacked.
• a third interface "Process builder” (17) contains the signatures of the method that a processing constructor (33) must carry out.
• the processing constructor (33) needs to produce the execution plan.
• the processing manufacturer supplies the filter chain (46), which constitutes the main component of the execution plan corresponding to the given request.
• a first method (171) makes it possible to give the processing constructor (33) the necessary information, extracted by the analysis component (31), to generate the execution plan.
• a second method (172) makes it possible to give the processing constructor (33) the first filter element of the execution plan. This first element is the filter object corresponding to the "Source access filter” component (52), supplied by the source component (51) after the submission of the translated request, and making it possible to retrieve the results from the source.
• a third method (173) returns to the connection component (21) the execution plan (77), that is to say the chain (46) of filters calculated for the request.
• the result transformation module (4) contains interfaces and classes for building a filter chain that processes data from the source before sending it to the client application.
• the following interfaces and classes are used by each component of this module in order to reduce the dependence of the components and to promote the reuse of these components.
• Each filter component (42 to 45) must provide a "Filter” interface (18). This interface makes it possible to compose a filter chain (46) by inserting any filter component at any position.
• a first method "initialize” (181) initializes the filter with the given list of parameters contained in a direct access table (hashtable).
• a second "open” method (182) opens the stream of data delivered by the chain.
• a third method “next” (183) returns an object of the class "DataFormat” representing a tuple. This method allows you to take the tuples of the same request one after the other.
• a fourth "close” method (184) closes the data stream delivered by the chain.
• a class "DataFormat” defines the structure for processing common data in the adapter. In the case of filters, it corresponds to a list of attribute-value pairs or "tuples" which is transmitted from one filter in the chain to the next filter.
• a set of methods (61), given by way of example in appendix 2, makes it possible to handle a tuple containing semi-structured data defined by the "DataFormat” class.
• the result transformation module can include additional interfaces to allow, for example, to perform processing in parallel.
• the components of the source layer (10) must provide a "Source” interface (62, appendix 2) in order to be used by the connection component (21) of the coordination module to retrieve the information in the form of metadata and to send the request at the source.
• This interface includes a first method “checkMetadata” (621) which checks the availability of metadata.
• This interface includes a second method “getMetaData” (622) retrieves the metadata delivered by the source and translates it into something understandable by the upper client layer.
• a third "sendQuery” method (623) receives a request understandable by the source, produces and returns (76) to the processing constructor (33) a filter object (52) in order to retrieve the results.
• the source component (51) If the source component (51) is configurable during the operating period, it must provide another interface (63), called the "Admin" interface.
• This interface includes a first method “setConfig” (631) to give the configuration parameter to the source component.
• the source component programmer must provide the format and form of this or these parameters.
• the interface uses a second “getConfig” method (632) to retrieve the configuration parameters available in the adapter.
• the adapters according to the invention have many advantages. They offer a uniform access interface to heterogeneous sources of information, each interface relying on a minimum number of carefully selected methods.
• the same application programming interface (API) the same query language and the same data model are used to process each source, even if these sources are completely different in terms of access protocol, query language and the data model. Only the objects to which the methods apply are modified according to the source languages and the languages of the client applications.
• the definition of a generic architecture with common interfaces for the components included in the adapter modules makes it possible to produce components, for example, connection (21), analyzer (31), translator (32), filter (42 45), processing (33) source (5), reusable. These components can be reused in several adapters without modifications.
• the composition of reusable components in component chains allows programmers to easily make adapters.
• the evolution of data sources and the evolution of components can be easily supported by plugging components into chains of components already existing in the adapter or by replacing one component with another.
• the rapid creation of robust adapters relies on these reusable components.
• the common architecture according to the invention can be used with libraries of reusable components to create adapter families characterized by a common data manager, for example "Lotus Notes", or to create specialized adapters made for a source of specific data.
• Three types of components can be distinguished according to their dependence on the data manager of the associated source. This dependency dictates how the components can be reused from one adapter to another. Some components are specific to an adapter and cannot be reused.
• query building for a WWW depends on the language that this particular source accepts.
• Some components are common to an adapter family.
• the HTTP communication module can be reused for all adapters associated with a website-based data source.
• Some components are common to all adapters.
• the projection and selection filters applied to data obtained from the source do not depend on the source, and can potentially be reused to build each adapter.
• the method according to the invention makes it possible to provide "metadata" to the client application to optimize the requests and allow better use of the source.
• the adapter provides metadata about the source of data, allowing a programmer to build applications that dynamically use this metadata, thus making the applications reusable from completely different sources. This makes building and managing these applications easier.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Computational Linguistics (AREA)
• Data Mining & Analysis (AREA)
• Databases & Information Systems (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Computer And Data Communications (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=9534805

Family Applications (1)

Country Status (6)

Families Citing this family (61)

Family Cites Families (10)

• 1998
  • 1998-12-28 FR FR9816780A patent/FR2787957B1/fr not_active Expired - Fee Related
• 1999
  • 1999-12-27 JP JP2000591536A patent/JP2002533842A/ja active Pending
  • 1999-12-27 CA CA002320891A patent/CA2320891A1/fr not_active Abandoned
  • 1999-12-27 US US09/622,357 patent/US6704726B1/en not_active Expired - Lifetime
  • 1999-12-27 WO PCT/FR1999/003294 patent/WO2000039709A1/fr active Application Filing
  • 1999-12-27 EP EP99964723A patent/EP1057128A1/de not_active Ceased

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events