WO2016203479A1

WO2016203479A1 - A system and method providing database manipulation operations and management

Info

Publication number: WO2016203479A1
Application number: PCT/IL2016/050640
Authority: WO
Inventors: Koby COHEN
Original assignee: E.K. Coya Holdings
Priority date: 2015-06-17
Filing date: 2016-06-16
Publication date: 2016-12-22

Abstract

The present invention provides a method for providing database manipulation operations and management including at least the operation of database partitioning. The method comprising the steps of: analyzing an original database schema and data for identifying key parameters for the database partitioning, applying an automatic partition process, in reference to at least one identified key parameter, thus obtaining a plurality of partitioned databases, each pertaining to a predefined portion of the schema and data of the original database and facilitating a single, global point of access to interface said plurality of partitioned databases, enabling users to query or edit database content or modify database schemas, through an at least one designated communication server;

Description

A SYSTEM AND METHOD PROVIDING DATABASE MANIPULATION

OPERATIONS AND MANAGEMENT

FIELD OF THE INVENTION

[0001] The presented invention generally relates to the field ofDatabase

administration. More specifically, this invention relates to methods of centralized maintenance and management of complex database structures.

SUMMARY OF THE INVENTION

According to some embodiments the steps of analyzing , applying, facilitating are performed by at least one processor.

According to some embodiment of the present invention the method further comprising the step of: maintaining multiple copies of database functions jobs or stored procedures, to accommodate access and application of said functions and stored procedures from both partitioned database and said designated communication sever.

According to some embodiment of the present invention the method the step of creating a common database having content which is common to more than one partitioned database, wherein said common database is accessible to more than one partitioned database, hence any modification of the common database affects more than one partitioned database. According to some embodiment of the present invention the method creating a global database incorporating a data structure relating to at least two partitioned databases.

According to some embodiment of the present invention the analysis of the original database schema and data is based on values of predefined profiling parameters of the original database.

According to some embodiment of the present invention the determination of partition key parameters is based on categorized profiling of the predefined profiling parameters , wherein the categorization is based on iterative machine learning of plurality of databases partitioning processes to provide optimal partition.

According to some embodiment of the present invention the predefined profiling parameters include database usage characteristics.

According to some embodiment of the present invention the manipulation process includes database scaling, dividing or merging operations on the partitioned databases.

According to some embodiment of the present invention the database manipulation process includes database migration operations to migrate databases from one database type to another.

The present invention provides system for providing database manipulation operations and management including at least the operation of partitioning. The system comprised of:

Data base partitioning module for analyzing an original database schema and data for Identifying key parameters for the manipulation : partitioning and applying an automatic partition process, in reference to at least one identified key parameter, thus obtaining a plurality of partitioned databases, each pertaining to a

predefined portion of the schema and data of the original database; and database interface for facilitating a single, global point of access to interface said plurality of partitioned databases, enabling users to query or edit database content or modify database schemas, through an at least one designated communication sever.

[0002] database type to another, DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 presents a block diagram depicting a general conceptual overview of the system according to some embodiment of the present invention.

[0004] Figures 2 and 3 jointly present a flow chart diagram, depicting the deployment of the system, and the basic operation scenario according to some embodiment of the present invention.

[0005] Figures 4, 5 and 6 jointly present a flow chart diagram, depicting the process of applying the current invention to perform database partitioning, according to some embodiment of the present invention, i.e. to disperse the original data stored on a single database entity to form an aggregate of databases.

[0006] Figure 7 presents a block diagram, depicting an example of database partitioning, according to some embodiment of the present invention.

[0007] Figure 8 presents a flow chart diagram, depicting the process of applying the current invention to perform database scaling according to some embodiment of the present invention, i.e. to increment the number of databases produced by a database partitioning process to include additional, empty schemas of identical structure.

[0008] Figures 9 and 10 jointly present a flow chart diagram, depicting the process of applying the current invention to perform database type migration according to some embodiment of the present invention, i.e. modify a selected database type, either as a stand-alone process or as part of a parent database manipulation process.

[0009] Figure 11 and 12 jointly present a flow chart diagram, depicting the process of creation of a database profile according to some embodiment of the present invention, i.e. acquiring characteristics of the organization's original database in order to suggest optimal database partitioning and scaling processes.

[0010] Figure 13 presents a flow chart diagram, depicting the normal work mode against the aggregate of databases formed in the process of database partitioning according to some embodiment of the present invention.

DETAILED DESCRIPTION OF SOME EMBODIMENTS OF THE INVENTION

[0011] Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. The invention is applicable to other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.

[0012] Following is a table of definitions of the terms used throughout this application.

database process.

Global databases differ from Partitioned databases in the sense that they gather all non-specific partition keys.

Pertaining to the example brought in Figure 6 and discussed below, The database partition key was selected to be the Car ID (which represents specific automobile manufacturers). The Global database contains the data respective to two automobile manufacturers: Mazda and Renault.

Common Common databases contain data that is common to all Partitioned Database databases. Consequently, the data residing in Common databases is accessible to the administrator of each Partitioned database.

Database Database Scaling is the process of applying the current invention to Scaling increment the number of database entities within an aggregate of

"Partitioned" databases.

The basic form of database scaling involves creation of new "Scaled database" entities, which incorporate empty schemas, identical in structure and relations to the said Partitioned databases.

According to one embodiment of the current invention, the "Scaled database" empty schemas may be modified during the process of their creation, to differ in structure from the said Partitioned databases (e.g. include new tables or omit tables that originally reside in the "Partitioned" databases)

Scaled A database entityformed as an outcome of the Database Scaling database process.

The Scaled database incorporates an empty schema which is either identical in structure and relations to that of a referenced Partitioned database, or is a modified version thereof.

Database A database manipulation process involving the migration and migration adaptation of a database from one instance to another. Database migration may involve operations such as:

• Transferring the database to a different physical location

• Changing the database type from one type to another (e.g.: from MySQL to MS-SQL).

• Adaptation of the database schema, its content and relations to comply with a specific format

This process may be employed either as a "stand alone" procedure, or as part of a parent database manipulation process (e.g. Database Partitioning)

Migrated A database entity formed as an outcome of the Database Migration database process.

With the exclusion of the database type (e.g. MS-SQL, PostGRE), the Migrated database is equivalent in all aspects to the original database on which the Database Migration process had been applied.

Restore_DB The Restore_DB initiated as a back-up of the original

organizational database. Prior to applying any process of database manipulation, the original database content is uploaded to the Restore_DB database.

In addition, the Restore_DB database

• contains all the information relevant to the processes of database manipulation as set up on the Client application, and

• is updated to reflect changes made to the partitioned

databases, thus qualified as a backup to the entire database aggregate. jobs Scheduled tasks on the databases, such as validation tasks

[0013] Figure 1 displays a block diagram depicting a general conceptual overview of the system as presented in the current invention. The system is deployed by installing the components of the installation package, which include at least one of the following:

• A Master, and an optional Slave Gateways: Server based applications or

services that provide a single point of access to all the databases in the system

• Logic modules that facilitate different aspects of the system functionality.

These include:

o Accessing all database instances (Database Interface Module 2100) o Setting up database partitioning and scaling (Database partitioning module 2200 and Scaling module 2300)

o Database profiling and machine learning (Database Profiling Module

2600)

o Maintenance of a failover database (Database failover module 2500) o Migration between different database types (Database Migration Module 2400)

o Synchronization of the Restore_DB database (Database

synchronization module 2700) for Running in order to synchronize the gate against the partition databases,

o A Client module, providing a user interface to the said Logic modules, databases and Gateway modules,

o Core engine 2800 analyzes the source database and reviews the

database target candidates. It manages distribution and resolves conflicts within the said target databases.

[0014] According to some embodiments of the present invention the following logical procedure are available: Data Sync process between all databases ,

maintaining database partition linkage with gateway, Logical Load balancer for managing read/write operations, jobs schemas updating etc., Slave synchronization, maintain Match rules configurations or monitors data exchange(e.g. preventing Data Leakage)

[0015] Figures 2 and 3 jointly present a flow chart diagram, depicting the deployment of the system, and the basic operation scenario.

[0016] The Client application presents the various available processes of database manipulation, embedded within the logic modules. These processes are presented to authorized database administrators, according to predefined roles and permissions.

[0017] The said database manipulation process types include at least one of the following:

• Single or Multiple Database access

• Database partitioning

• Database scaling, dividing , merging or

• Database backup and

• Migration between different database types

Each database manipulation processes may be configured either manually, or through a dedicated Wizard program (e.g. the database partitioning wizard). [0018] The dedicated Wizard program offers the following benefits:

1. A structured, orderly flow of configuration

2. Suggestions for the database manipulation process setup, based upon database profiling and machine learning algorithms.

[0019] Prior to applying any process of database manipulation, the original database content is uploaded to the Restore_DB database. The original database remains untouched from that point onward. In addition, the Restore_DB database contains all the information relevant to the processes of database manipulation as set up on the Client application.

[0020] The database manipulation process is carried out as configured by the administrator on the Client application. It is executed by a respective logic module and the active (Master / Slave) Gateway. The implementation of each database manipulation process is further elaborated below.

[0021] The Restore_DB database is updated both periodically and in real-time (according to predefined triggers) during worktime, to reflect the changes made on each separate database instance. This qualifies Restore_DB as a backup for the entire system, enabling redeployment of databases and data restoration in case of system failure. The action of data synchronization is governed by the database

synchronization module 2700..

[0022] The Failover module 2500 continuously monitors the system's activity to detect locks or failures in specific databases or other system modules. When such a failure is detected, the Failover module 2500 is deployed to restore database integrity according to the information maintained in the Restore_DB database.

[0023] All database instances are centrally accessed through a Gateway. By default, the Master Gateway will handle the access to, and manipulation of all database instances.

[0024] The Master Gateway constantly updates the Slave Gateway regarding its configuration and actions, qualifying it to take over in case the Master Gateway fails, so that all databases will be centrally accessed through the Slave Gateway.

[0025] The Slave Gateway may also be configured to work adjacent the Master gateway as a load balancer unit, so as to share computational resources with the Master gateway, enabling to execute any operation or task of the master gateway.

[0026] After applying various database manipulation processes, a structure of multiple inter-related database entities is normally formed. The Database Interface Module 2100 is designed to facilitate access to these databases (Read, Write and modification of database schemas) during normal work mode. The properties and capabilities of the Database Interface Module 2100 are elaborated below.

[0027] Figures 4 ,5 and 6 jointly present a flow chart diagram, depicting the process of applying the current invention to perform database partitioning, i.e. to disperse the original data stored on a single database entity, according to predefined keys or configurations, to form an aggregate of databases.

[0028] The Partition interface module 2200 presents (via the Client application 1000) the following basic partitioning -related selections:

• Selection of the database partitioning option from a set of available database manipulation processes.

• Selection of the relevant original database for partitioning.

• Selection of a work mode; i.e. whether the configuration is to be applied manually, or automatically - with the assistance of a dedicated Database Partition Wizard.

[0029] The Database Partition interface 2200 interfaces the copy of the original organizational database on the Restore_DB database, through the active (Master / Slave) Gateway, to obtain the content and structure of the original database.

[0030] The Database Partition Wizard 2201 obtains the content and structure of the original database from the Database Partition Module.

[0031] The Database Partition Wizard 2201 interfaces the Database Profiling Module 2600 in order to analyze the contents of the original database and suggest partition parameters based upon:

• 1. Preset database profiles, reflecting characteristic database implementations (e.g. for industry, banking, retail commerce)

• 2. Database keys (e.g. department, plant, location, process)

• 3. Database constraints (e.g. size of specific database tables)

[0032] The Partition interface module 2200 presents (via the Client application 1000) the option to either accept or reject all or some of the partition parameters suggested by the Wizard. The database administrator may also manually edit the partition parameters (e.g. select additional partition keys).

[0033] The Partition interface module 2200 presents (via the Client application 1000) the option to define which schemas would be manifested in a dedicated partition and which would be aggregated in the Global database partition. Pertaining to the car manufacturers' database example presented on Figure 7 (and discussed below), some car manufacturers would have a dedicated database, whereas other car manufacturers would be aggregated in the Global database partition.

[0034] The Partition interface module 2200 presents (via the Client application 1000) the option to apply disintegration of the original database's schema into partitions that embody partial schemas, containing a partial set of tables, or database shards.. The said partial schemas may reside within:

• 1. The Global database partition or dedicated database partitions; In this case, tables that reside in different partitions are not related, and are inaccessible between partitions.

• 2. A Common database: In this case, tables that reside within the partial

schema are accessible to all or some of the partition databases.

[0035] The Partition interface module 2200 presents (via the Client application 1000) the option to configure different database partitions' metadata. This includes at least one of the following:

• Database type (e.g. MS-SQL, MySQL, PostGRE)

• Database physical location (Hostname, IP)

• Administrative credentials, access roles and permissions

• Catalogs, views, synonyms

• Indexes, Triggers, Sequences, Constraints

[0036] The Partition interface module 2200 presents (via the Client application 1000) the administrator with the option to apply changes to the database schema (e.g. omit table columns) and metadata (e.g. change database type). Selecting this option will invoke the Data migration interface 2400 in order to configure the required changes. The configuration of the database migration interface and the application of consequent database schema and metadata changes is discussed below, in the description of figures 9 and 10 ("Applying database migration").

[0037] The Partition interface module 2200 presents (via the Client application 1000) the administrator with the option to apply the partition configuration, thus invoking the action of database partition according to the described configuration. Creation and access to the new partition database host machines is physically facilitated by the active Gateway.

[0038] The Core Engine 2800 analyzes the source database and reviews the database target candidates. It manages distribution to partition databases and resolves conflicts within the said target databases.

[0039] The Core engine Module 2800 creates new partition databases, according to the partition database metadata (e.g. database type, location, views, permissions etc.) as defined in the said configuration.

[0040] The Core engine Module 2800 creates new database schemas;

• The data content of the schemas is based on that of the original database. It is partitioned by the said partition keys, and applied database disintegration as mentioned above.

• Relations, triggers, functions, stored procedures are also modified according to the said configuration.

• The format and logic of the new schema is modified to support the new

database type.

[0041] The Core engine Module 2800 populates the newly created partition databases with the said schemas.

[0042] The Core engine Module 2800 duplicates stored procedures, functions, jobs and triggers on the newly created databases: partition databases, Global database and Common database.

[0043] The core engine module 2800 duplicate jobs which include scheduled task operations on the databases such as daily operations of data validation, e.g. bank database require validation on financial transactions. The jobs function can be accessible directly from the local partition database server or through the gateway.

[0044] Database partition- specific duplicates of stored procedures are renamed according to the partition database they reside on, e.g.:

StoredProcedureName_PartitionDB l. Such duplicates of stored procedures will be accessible by either one of these three methods:

• Locally, from the local partition database server. This method promotes a distributed, encapsulated programming approach. • Globally from the Gateway. This is a convenient approach for implementing global procedures that access multiple partition databases.

• Locally, from the local partition database server directed through the gateway

[0045] The Core engine Module 2800 operates built-in tests to ensure the new database structure's integrity and operability

[0046] The Partition interface module 2200 presents (via the Client application 1000) the option to move, divide, merge, or rebalance partitioned databases. These actions are applied to the partitioned databases by the Core engine Module 2800 in a similar manner to the partitioning process as described above.

[0047] According to some embodiments of the present invention the partition

manipulation module support the operations of moving , dividing , merging or rebalancing the partitions database. These operations can be applied by the user using a wizard which provide the him with suggestions which operation is relevant based on analyzing changes in partitions database schemas, usage characteristics of the database and updating of the databases.

[0048] In addition, The Partition interface module 2200 presents (via the Client application 1000) the option to apply any type of database migration operation on the said partitioned databases. This type of operation will invoke the database migration interface 2400, and employ the same actions as discussed below, in figures 9,10 ("Applying database migration")

[0049] Figure 7 presents a block diagram, depicting an example of database partitioning In this example, the original database is consisted of data pertaining to different car manufacturers, labeled by name and a Car ID field as follows:

• Car ID 1: Ford

• Car ID 2: Citroen

• Car ID 3: Mazda

• Car ID 4: Renault

The original data relating to each car manufacturer is presented in the following tables:

• The Products table, which presents data that is common to all car

manufacturers • The Employee, Department and Users tables, which present data that is unique to each car manufacturer.

[0050] The first stage of the database partition process is the transfer of the original database to a new entity: the Restore_DB database.

[0051] The partition is configured as follows:

• Car ID was selected as the partition key for applying the database partition process. Consequently, the Global database and Partitioned databases are differentiated by the manufacturer's Car ID (Global DB: Mazda and Renault, Partitioned DB 1: Ford, Partitioned DB2 and Partitioned DB3: Citroen).

• The 'Products' table as set as common data for all Car IDs (since all

manufacturers produce a steering system and an engine). Consequently, the 'Products' data resides in the Common Database and is accessible to all Partitioned databases.

• Mazda and Renault (Car ID 1 and 2 respectively) are set to share the same database, and hence their data is kept in the Global database.

• The data relating to Citroen is internally partitioned into two separate

databases: Partitioned DB 2 and Partitioned DB 3. Each of these two databases contains any subset of the original data relating to Citroen in the original database.

[0052] Figure 8 presents a flow chart diagram, depicting the process of applying the current invention to perform database scaling, i.e. to increment the number of databases produced by a database partitioning process to include additional, empty schemas of identical structure.

[0053] The Client application presents the following scaling-related selections:

• Selection of the database scaling option from a set of available database

manipulation processes.

• Selection of the relevant partition database for scaling.

• Scaled database metadata, (e.g.: The designated database type, physical

location, administrative credentials, permissions and views.)

[0054] The Database Scaling Module 2300 presents the administrator with the option to apply the scaling configuration, thus invoking the action of database scaling according to the described configuration. Creation and access to the new scaled database host machines is physically facilitated by the active Gateway. [0055] The Database Scaling Module 2300 creates new empty schemas that own the same structure and properties as the partition databases created in the preceding database partition process.

[0056] The Client module presents the option to modify the newly created schemas. Such modification include (but are not limited to):

• Add or remove tables

• Add or remove columns from specific tables

• Assign new Stored procedures, functions or triggers or omit them from the schema

[0057] The Database Scaling Module 2300 creates new scaled databases, according to the scaled database metadata (e.g. database type, location, views, permissions etc.). It populates the newly created Scaled databases with the said newly created empty schemas.

[0058] The Database Scaling Module 2300 duplicates stored procedures, functions and triggers on the newly created scaled databases

[0059] Database- specific duplicates of stored procedures will be renamed according to the scaled database they reside on, e.g.: StoredProcedureName_ScaledDB l. Such duplicates of stored procedures will be accessible by either one of these two methods:

• Locally, from the local scaled database server. This method promotes a

distributed, encapsulated programming approach.

• Globally from the Gateway. This is a convenient approach for implementing global procedures that access multiple scales databases.

[0060] Figures 9 and 10 jointly present a flow chart diagram, depicting the process of applying the current invention to perform database type migration, i.e. modify a selected database type, either as a stand-alone process or as part of a parent database manipulation process.

[0061] The Migration interface module 2400 presents (through the Client application) the following selection options:

1. Selection of the database migration option from a set of available database manipulation processes.

2. Selection of the relevant source database for migration.

3. Selection of the designated target database metadata, including at least one of the following: • Database type (e.g. MS-SQL, MySQL, PostGRE)

• Database physical location (e.g. Hostname, IP)

• Administrative credentials, access roles and permissions

• Catalogs, views, synonyms

• Indexes, Triggers, Sequences, Constraints

[0062] The Migration interface module 2400 further presents options for changes in the database schema and metadata, such as:

• Addition or omission of tables or columns

• Changes in relations within the schema (e.g. unique and foreign keys)

• Changes in functions, Triggers and stored procedures

[0063] Database migration could be implemented as part of another, parent database manipulation process (e.g. database partition or database scaling). In this case, the Database Migration Module 2400 will provide its services to the respective parent database manipulation process.

[0064] In a complementary manner, database migration could be implemented as a "Stand alone" process, unrelated to any other database manipulation process, and targeting a specific database entity. In this case, the Database Migration Module 2400 will act independently from any other database manipulation module.

[0065] The Core Engine 2800 analyzes the source database and reviews the database target candidates. It manages distribution and resolves conflicts within the said target databases.

[0066] The Database Migration Interface 2400 presents (via the Client module 1000) the option to apply the database migration configuration, thus invoking the action of database migration according to the described configuration. Creation and access to the new migrated database host machines is physically facilitated by the active Gateway.

[0067] The Core engine Module 2800 creates a new, empty database, according to the configured destination database metadata (e.g. database type, location, views, permissions etc.) as defined in the said configuration.

[0068] The Core engine Module 2800 creates new database schemas;

• The data content of the schemas is based on that of the original database, with changes made according to the said configuration. • Relations, triggers, functions, stored procedures are also modified according to the said configuration.

• The format and logic of the new schema is modified to support the new database type.

[0069] The Core engine Module 2800 populates the newly created migration databases with the said schemas.

[0070] The Core engine Module 2800 operates built-in tests to ensure the new database's integrity and operability

[0071] Figures 11 and 12 jointly present a flow chart diagram, depicting the process of creation of a database profile, i.e. acquiring characteristics of the organization's original database in order to suggest optimal database partitioning and scaling processes.

[0072] The Database Profiling Module 2600 extracts specific parameters from the original organizational database. These parameters include at least one of the following:

• Organization's business (e.g. Banking, Industry, Engineering, Retail

commerce, etc.)

• Organization's size (e.g. number of employees)

• Organizational structure (e.g. Departments, projects, business units, cost centers, etc.)

• Structural properties of the database (e.g.: Indexed fields, triggers, stored procedures and functions, relations between tables etc.)

• Database size (Consumed memory space, rate of database growth, Number of entries in selected fields, etc.)

• Patterns of database access (e.g.: rate of access, timing of access, access to specific tables etc.)

[0073] The Database Profiling module analyzes the said extracted organizational database parameters, and compares them to predefined thresholds. This analysis serves to produce a prediction of data manipulation operations that may need to be applied to the organizational database [1610]. For example: According to the rate of growth of database size, the Database Profiling module may predict the need to perform database migration to a different database type or allocate additional resources to the existing database.

[0074] The Database Profiling Module 2600 analyzes the said extracted parameters, and produces suggested keys for database partitioning. Such keys may rely on the most common business units, e.g. organizational hierarchic levels, white labels, time period, or end-user range.

[0075] The Database Profiling module accumulates the database manipulation processes that are applied, and the subsequent structural changes that take place on the organizational database by database administrators.

[0076] The said information (database parameters and structural changes applied to the organizational database) are gathered from multiple organizations upon consent from their owners. They are sent to the Database Iterative Analysis module, which is a computational module, external to the installed system.

[0077] The said Database Iterative Analysis module applies the following algorithm:

1. It categorizes different original organizational databases according to the said parameters.

2. It matches an optimal database structure to each original database category, according to the accumulated data regarding database manipulation processes conducted by database administrators.

3. The analysis module employs an iterative machine learning algorithm to

cluster database categories according to the optimal database structure.

The said algorithm is an iterative one, meaning that the clustering of original organizational database categories and the matching of an optimal database structure to each cluster is constantly refined, as additional data is presented to the Database Iterative Analysis module.

[0078] The Client application provides an administrative interface to submit the original organizational database parameters that have been extracted by the Database Profiling module to the external Database Analysis module.

[0079] The Database Profiling module consequently obtains an optimal database structure from the external Database Analysis module, and generates suggested database manipulation processes. These suggestions are presented in the database manipulation processes' wizard, as part of the database manipulation configuration process. [0080] Figure 13 presents a flow chart diagram, depicting the normal work mode against the aggregate of databases formed in the process of database partitioning. After applying various database manipulation processes, a structure of multiple interrelated database entities is normally formed. The Database Interface Module 2100 is designed to facilitate access to these databases, and assist database administrators during normal work mode.

[0081] The Database Interface Module 2100 provides the following capabilities:

• Apply specific database queries (Read)

• Change the database contents (Write)

• Apply structural changes to the database schema (e.g. add or remove tables or columns)

• Call stored procedures and functions

[0082] The Client module presents the different options for applying the said Database Interface Module 2100 capabilities:

[0083] Database access can be applied either singularly (to a specific databases) or globally (to multiple database entities simultaneously).

[0084] Database entities may be accessed for read and write operations through at least one of the following:

• A dedicated selection form

• A scripting interface, or

• Through executing a saved script file (e.g. T-SQL).

[0085] Duplicates of stored procedures are named according to the partition database or scaled database they reside on, e.g.: StoredProcedureName_PartitionDB l and StoredProcedureName_ScaledDB 1 respectively (See 1365, 1450). The Database Interface Module 2100 facilitates calling such duplicates of stored procedures by either one of these two methods:

• Locally, from the local database server. This method promotes a distributed, encapsulated programming approach.

• Globally, from the Gateway. This is a convenient approach for implementing global procedures that access multiple database entities.

[0086] The physical access to target database entities is facilitated by the active (Master / Slave) Gateway. [0001] The system of the present invention may include, according to certain embodiments of the invention, machine readable memory containing or otherwise storing a program of instructions which, when executed by the machine, implements some or all of the apparatus, methods, features and functionalities of the invention shown and described herein. Alternatively or in addition, the apparatus of the present invention may include, according to certain embodiments of the invention, a program as above which may be written in any conventional programming language, and optionally a machine for executing the program such as but not limited to a general purpose computer which may optionally be configured or activated in accordance with the teachings of the present invention. Any of the teachings incorporated herein may wherever suitable operate on signals representative of physical objects or substances.

[0002] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions, utilizing terms such as, "processing", "computing", "estimating", "selecting", "ranking", "grading", "calculating", "determining", "generating", "reassessing", "classifying", "generating", "producing", "stereo-matching", "registering", "detecting", "associating", "superimposing", "obtaining" or the like, refer to the action and/or processes of a computer or computing system, or processor or similar electronic computing device, that manipulate and/or transform data represented as physical, such as electronic, quantities within the computing system's registers and/or memories, into other data similarly represented as physical quantities within the computing system's memories, registers or other such information storage, transmission or display devices. The term "computer" should be broadly construed to cover any kind of electronic device with data processing capabilities, including, by way of non-limiting example, personal computers, servers, computing system, communication devices, processors (e.g. digital signal processor (DSP), microcontrollers, field programmable gate array (FPGA), application specific integrated circuit (ASIC), etc.) and other electronic computing devices.

[0003] The present invention may be described, merely for clarity, in terms of terminology specific to particular programming languages, operating systems, browsers, system versions, individual products, and the like. It will be appreciated that this terminology is intended to convey general principles of operation clearly and briefly, by way of example, and is not intended to limit the scope of the invention to any particular programming language, operating system, browser, system version, or individual product.

[0004] It is appreciated that software components of the present invention including programs and data may, if desired, be implemented in ROM (read only memory) form including CD-ROMs, EPROMs and EEPROMs, or may be stored in any other suitable typically non-transitory computer-readable medium such as but not limited to disks of various kinds, cards of various kinds and RAMs. Components described herein as software may, alternatively, be implemented wholly or partly in hardware, if desired, using conventional techniques. Conversely, components described herein as hardware may, alternatively, be implemented wholly or partly in software, if desired, using conventional techniques.

[0005] Included in the scope of the present invention, inter alia, are electromagnetic signals carrying computer-readable instructions for performing any or all of the steps of any of the methods shown and described herein, in any suitable order; machine- readable instructions for performing any or all of the steps of any of the methods shown and described herein, in any suitable order; program storage devices readable by machine, tangibly embodying a program of instructions executable by the machine to perform any or all of the steps of any of the methods shown and described herein, in any suitable order; a computer program product comprising a computer useable medium having computer readable program code, such as executable code, having embodied therein, and/or including computer readable program code for performing, any or all of the steps of any of the methods shown and described herein, in any suitable order; any technical effects brought about by any or all of the steps of any of the methods shown and described herein, when performed in any suitable order; any suitable apparatus or device or combination of such, programmed to perform, alone or in combination, any or all of the steps of any of the methods shown and described herein, in any suitable order; electronic devices each including a processor and a cooperating input device and/or output device and operative to perform in software any steps shown and described herein; information storage devices or physical records, such as disks or hard drives, causing a computer or other device to be configured so as to carry out any or all of the steps of any of the methods shown and described herein, in any suitable order; a program pre-stored e.g. in memory or on an information network such as the Internet, before or after being downloaded, which embodies any or all of the steps of any of the methods shown and described herein, in any suitable order, and the method of uploading or downloading such, and a system including server/s and/or client/s for using such; and hardware which performs any or all of the steps of any of the methods shown and described herein, in any suitable order, either alone or in conjunction with software. Any computer-readable or machine-readable media described herein is intended to include non-transitory computer- or machine-readable media.

[0006] Any computations or other forms of analysis described herein may be performed by a suitable computerized method. Any step described herein may be computer-implemented. The invention shown and described herein may include (a) using a computerized method to identify a solution to any of the problems or for any of the objectives described herein, the solution optionally include at least one of a decision, an action, a product, a service or any other information described herein that impacts, in a positive manner, a problem or objectives described herein; and (b) outputting the solution.

[0007] The scope of the present invention is not limited to structures and functions specifically described herein and is also intended to include devices which have the capacity to yield a structure, or perform a function, described herein, such that even though users of the device may not use the capacity, they are, if they so desire, able to modify the device to obtain the structure or function.

[0008] Features of the present invention which are described in the context of separate embodiments may also be provided in combination in a single embodiment.

[0009] For example, a system embodiment is intended to include a corresponding process embodiment. Also, each system embodiment is intended to include a server- centered "view" or client centered "view", or "view" from any other node of the system, of the entire functionality of the system, computer-readable medium, apparatus, including only those functionalities performed at that server or client or node.

Claims

1. A method for providing database manipulation operations and management including at least the operation of database partitioning, said method comprising the steps of: analyzing an original database schema and data for identifying key parameters for the database partitioning; applying an automatic partition process, in reference to at least one identified key parameter, thus obtaining a plurality of partitioned databases, each pertaining to a predefined portion of the schema and data of the original database; facilitating a single, global point of access to interface said plurality of partitioned databases, enabling users to query or edit database content or modify database schemas, through an at least one designated communication server; wherein the steps of analyzing , applying, facilitating are performed by at least one processor.

2. The method of claim 1 further comprising the step of:

maintaining multiple copies of database functions jobs or stored procedures, to accommodate access and application of said functions and stored procedures from both partitioned database and said designated communication sever.

3. The method of claim 1 further comprising the step of creating a common database having content which is common to more than one partitioned database, wherein said common database is accessible to more than one partitioned database, hence any modification of the common database affects more than one partitioned database.

4. The method of claim 1 further comprising the step of creating a global database incorporating a data structure relating to at least two partitioned databases.

5. The method of claim 1 wherein the analysis of the original database schema and data is based on values of predefined profiling parameters of the original database.

6. The method of claim 5 wherein the determination of partition key parameters is based on categorized profiling of the predefined profiling parameters , wherein the categorization is based on iterative machine learning of plurality of databases partitioning processes to provide optimal partition.

7. The method of claim 5 wherein the predefined profiling parameters include database usage characteristics.

8. The method of claim 1 wherein the manipulation process includes database scaling, dividing or merging operations on the partitioned databases.

9. The method of claim 1 wherein the database manipulation process includes database migration operations to migrate databases from one database type to another.

10. A system for providing database manipulation operations and management including at least the operation of partitioning, said system comprising the step of:

11. The system of claim 10 further comprising the step of:

maintaining multiple copies of database functions and stored procedures, to accommodate access and application of said functions and stored procedures from both partitioned database and said designated communication sever.

12. The system of claim 10 further wherein the partitioning module further included the step of creating a common database having content which is common to more than one partitioned database, wherein said common database is accessible to more than one partitioned database, hence any modification of the common database affects more than one partitioned database.

13. The system of claim 10 further wherein the partitioning module comprising the step of creating a global database incorporating a data structure relating to at least two partitioned databases,.

14. The system of claim 10 wherein the analysis of the original database schema and data is based on values of predefined profiling parameters of the original database.

15. The system of claim 10 wherein the determination of partition key parameters is based on categorized profiling of the predefined profiling parameters , wherein the categorization is based on iterative machine learning of plurality of databases partitioning processes to provide optimal partition.

16. The system of claim 10 wherein the predefined profiling parameters include database usage characteristics.

17. The system of claim 10 wherein the manipulation process includes database scaling operations to increase the number of partitioned databases.

18. The system of claim 10 wherein the database manipulation process includes database migration operations to migrate databases from one database type to another,