JP6511394B2 - Method and system for data storage and retrieval - Google Patents

Description

  The present invention relates generally to the type of data management system used by the leading providers of goods and services to manage the level of availability and availability of their entire product. More specifically, the present invention can respond to high-level queries from remote users of data storage without delay or with only a slight delay, and constantly updates the content as a result of data storage management Systems that do not affect the completion of the transaction.

  In all the interconnected worlds, the leading providers of all goods and services have built large database systems that retain the characteristics, specifications and costs of providing products and services. Operated under Database Management System (DBMS) control, a large number of online customers will be able to simultaneously access content, perhaps from around the world. Online customers are thus provided the opportunity to query the database and complete a commercial transaction using a particular online software application. Through specific online software applications, online customers reserve and purchase various products and services.

  In the aviation industry, an example of such a very large database is a database listing airline stocks. Such a database is used to track, in real time, the configuration of owned aircraft, as well as the actual number of seats that a given airline operates and the current status of reservations.

  More specifically, the inventory of the airline usually contains all the flights with available seats, and generally the service class (eg fast, business or economy class) and different prices and reservation conditions apply It is divided into many booking classes. One of the central functions of inventory management is inventory management. Inventory management manipulates the number of available seats in the various booking classes by starting and ending sales of individual booking classes. The fares stored in the fare estimation system are combined with the reservation conditions to determine the charge for each sales seat. In most cases, inventory management interfaces with the airline's revenue management system and supports permanent optimization of the offered booking classes as demand changes. The user accesses the inventory of the airline through an availability indicating application having a display and a graphical user interface. The application indicating availability contains all the flights provided in combination with the available seats in various booking classes up to the specific city.

  The airline inventory database is usually managed by the airline. The airline inventory database is also set up by companies that provide travel services to many parties in the travel industry. The travel industry also includes airlines, traditional travel agencies and other types of online travel service providers. Such companies include, for example, AMADEUS, a European travel service provider with a head office in Madrid, Spain. Some inventory controls are directly managed by the airline and connected with the Global Distribution System (GDS) or Central Reservation System (CRS).

  In this environment, the use of these databases is characterized by a significant increase in the level of queries or reading queries over the years. In fact, of transactions, the online reservation rate that the database has to process is very high. Thus, the travel service provider places the necessary computerized resources, and an increasing number of online customers can efficiently query the database and get quick answers, while at the same time completing the reservation, the airline's The situation must be addressed if updating the database can proceed simultaneously as a result of the sale of seats to the passenger.

  Large database systems that are available and widely used to implement such databases are provided by several specialized companies, such as Oracle. Oracle, headquartered in Redwood Shore, California, USA, specializes in the development of database management systems. However, standard DBMSs alone can not handle the level of acceptance that leading service providers of goods and services may have to serve tens of thousands of potential customers simultaneously. In order to achieve this purpose, the database must be protected in some way so as not to directly receive countless queries from the user.

  Therefore, many solutions have been developed to cache the contents of the database. The cache may be an application cache located in the application hierarchy, which basically reuses data previously acquired from the database by the application. This quickly raises the issue of the quality of the data provided in response to another query of the user, as the contents of the database may be updated during that time. This has been found to be a significant problem for some applications where the database is constantly updated and that requires high quality data. For example, in the case of an application related to airline inventory, the quality of the data becomes an issue as the up-to-date data directly affects the availability and fares of the seat sales offered to the customer.

  Thus, unless the quality of the data provided by this type of cache is not very important, and unless it is considered to be more useful than anything else, this type of application cache can be used between database and cache Requires implementation of a sophisticated mechanism. Thereby, when updated in the database, invalidation and / or exchange of previously acquired data is possible, thus keeping the contents of the application cache and database practically consistent. The cache is always first queried by the application, as the cache is often inserted into the path between the database and the application. If the queried data does not exist in the cache, it is retrieved from the database and brought into the cache before being provided to the application. All these solutions have in common that caches and databases are required to be tightly coupled and need to be aware of each other. As a result, when the service provider has to deploy more computer resources to handle the increase in traffic and maintain the performance of the system, these solutions should be serviced to more customers. It is not easily extensible.

  However, certain solutions allow fairly good extensibility and allow for some independence between cache and database, as described in "System and Method of Routing Database Requests to Database and Cache". It is described in 1. In the disclosed solution, the database and cache are to some extent independent by being activated individually under control of the application. However, caching is only used to answer read queries, and updates are only implemented in the database by the application. Therefore, in order to reflect changes to the database in the cache, Patent Document 1 describes that the duplicate parts for updating the cache are contained in the database.

  While all the caching solutions described above add significant workload to the database, caches and databases are not always guaranteed to be consistent, and the databases must be aware of the various caches. This makes it necessary to perform certain operations on the database when adding a new cache, and thus scalability is not easily realized. As mentioned above, Patent Document 1 requires the database management system to incorporate external parts. This is not really compatible with the use of a standard DBMS.

  Thus, for the purpose of the present invention, we describe a computerized data system with a database that can provide users with adequate data quality while allowing high traffic and high scalability.

  Other objects, features and advantages of the present invention will become apparent to those skilled in the art by examining the following description with reference to the attached drawings. It is intended that any additional advantages be incorporated herein.

U.S. Patent No. 6,609,126

  According to embodiments of the present invention, the aforementioned and other problems are overcome and other advantages are realized.

A first aspect of the present invention provides a method of storing data in a data storage system and acquiring data from the data storage system. The data storage system comprises a software application, one or more database systems, and a plurality of cache nodes. The software application is configured to receive the user request, the user request requests reading of at least one data or writing of one of the data, and the software application further transmits a read query or a write query to the data storage system , Configured to process user requests. The method is characterized in that the software application uniquely connects with one or more database systems and a plurality of cache nodes, and the method comprises the following steps performed by the software application comprising at least one data processor. .
That is, upon receiving a user request requesting to read at least one data, the software application sends only read queries to multiple cache nodes. Preferably, if the software application receives queried data (i.e. acquired data) from at least one cache node in response to a read query, the software application uses the queried data to make a user request. Process Preferably, if the software application receives a loss from all cache nodes in response to a read query, this means that the data was not found at the cache node, and the software application has one or more database systems Take out. When queried data is present in the database, upon obtaining the queried data from one or more database systems, the software application uses the queried data to process the user request to at least one cache node Send the queried data and send an indication to add the queried data to the at least one cache node.

  According to a preferred embodiment, upon receiving a user request requesting writing of at least one data, the software application sends a writing indication to the one or more database systems, and simultaneously a writing indication to the plurality of cache nodes. Send, thereby injecting multiple cache nodes into each lost read query and each write query of the data storage system. Each piece of data is thus stored identically in at least one cache node and one or more database systems of the plurality of cache nodes, whereby the database system and the plurality of cache nodes are always completely synchronized. To guarantee.

  Thus, the invention makes it possible to have a database completely independent of multiple caches, as opposed to the known solutions involving replication components integrated in the database to update the caches, It comprises a plurality of cache nodes. In known solutions, the database and cache are not completely independent, limiting the scalability of the entire storage system and requiring a specific database.

  By computerized data systems with databases and caches that are completely independent and not aware of each other, thus only introducing more computer and storage capacity when it is necessary to handle increased traffic, A data system with unlimited extensibility is possible.

  Furthermore, high extensibility can be achieved while reducing the cost of the device. In particular, the invention can be implemented with standard databases and DBMSs. The invention can also reduce maintenance costs. Specifically, the increase in storage resources does not have to do anything on the database.

  The software application is responsible for updating and caching data in the database. This is done by reflecting database writes or by adding queried data that exists in the database but is not yet in the cache. The end user is thus provided with high quality data or up-to-date data. In addition, because caches are injected quickly, throughput is increased as soon as new cache nodes are added to the system.

  Furthermore, the present invention can provide the user with an accurate and customer-friendly response.

  According to a non-limiting embodiment, the write query comprises at least one of adding, updating and deleting data of the database system.

  Optionally, the method according to the invention may comprise any one of the following conditional features and steps:

  The cache and database data models may be identical, but they do not have to be strictly identical. The only requirement is that it be consistent, so that exactly the same processing key can be obtained to access the cache and database records. The key can also lock database records for consistency of write operations. Thus, the records of the data, when present, are stored identically in the database and cache, or in such a way as to guarantee consistency in the processing of the same data in the cache and database. For example, a cache data model may be adapted to a database model to speed up data acquisition. Thereby, cache access time is improved while processing is completely consistent between the two entities.

  According to a non-limiting embodiment, the data model of the cache node is identical to the data model of the same database or databases. Each data of each cache node is stored identically in the database system. Each data of the database system is stored identically in each cache node.

  Write instructions to the one or more database systems are sent from the software application to the one or more database systems.

  An instruction to simultaneously write to a plurality of cache nodes is transmitted from the software application to the plurality of cache nodes.

  One single software application accesses the database system and cache node.

  The data storage system comprises one single database system.

  The cache comprises cache nodes and comprises non-persistent data storage means.

  The software application receives a positive acknowledgment when successfully adding the queried data to the at least one cache node.

  When writing of data occurs, the same queried data is retrieved from one or more databases at the same time, and subsequent addition of the queried data to the at least one cache node is aborted and a negative response ( negative acknowledgments are returned to the software application. Thereby, the software application can use the written data instead.

When sending an instruction to write to one or more database systems and sending an instruction to write simultaneously to multiple cache nodes, the following steps are performed.
Obtaining from the one or more database systems currently stored data to which the write is applied and locking it to the one or more database systems;
Processing the new data to be stored in the software application and writing it to one or more database systems;
Writing a cache buffer to the software application and holding new data to be temporarily stored;
Transferring new data to be stored to at least one cache node and committing the transaction to one or more database systems.

  In the present invention, cache nodes or caches are different from cache buffers. The cache buffer temporarily stores data during writing. No data is retrieved from the cache buffer in response to user requests. The cache buffer is dedicated to write processing.

  If the commit fails, then the application software sends an indication to cause at least one cache node to delete any previously set new data.

  At least one cache node containing the new data deletes the new data from its contents. If multiple cache nodes contain new data, all multiple cache nodes delete the new data.

  The software application determines which cache node or cache node among a plurality of cache nodes to issue an instruction to add data or to update or delete data to be sent.

  The decision considers load balancing.

If the queried data is not in either one or more database systems or at least one cache node:
After retrieving one or more database systems, the loss is returned to the software application instead of the queried data,
The software application sends the absent data to at least one cache node, the absent data is added for the queried data corresponding to the at least one cache node, and the absent data is for all subsequent read queries. Immediately available,
Thereby, the software application is prevented from further retrieving one or more databases in order to obtain the lost queried data.

  Data requested by the end user and ultimately not found in the database is then cached as "lost data". Thereby, it is possible to immediately return information that the data requested by the user is neither in the cache nor in the database for the next query after the cache. This prevents further queries to the database from slowing down the database system.

  According to a non-limiting embodiment, each data is associated with a header to form a record. The header indicates whether the content is lost in the database system. Therefore, it is possible to know whether it is worth extracting the database system just by reading the header of the record.

  According to another embodiment, the cache node stores specific values associated with data. This particular value indicates that the data does not exist in the database.

  The software application uniquely connects with one or more database systems via a first dedicated interface and connects with at least one cache node via a second dedicated interface.

  Data models are chosen to be directly mappable to databases and caches.

  Each set of data is grouped by functional entity and indexed by key. This allows immediate access to a set of data thanks to the keys of both the database and cache node, even if the content is somewhat different.

  The data is grouped by flight-date and identified by the flight-date key.

  The software application is a travel provider inventory software application.

  Software applications, database systems and cache nodes are included in the travel provider's inventory.

  Usually, the travel provider is an airline.

  The user request received by the software application is sent from at least one of a travel agent, an online travel agent, and an online customer.

  The cache node's data model and database are consistent, thereby obtaining exactly the same processing key and accessing cache node and database data.

  If the data is present, the data is stored identically in the database and at least one cache node, or in such a way as to ensure consistency in the processing of the same data in the cache and database.

  In another aspect of the invention, a computer-program product or non-transitory computer- is a readable medium and contains software program instructions. Execution of the software program instructions by the at least one data processor leads to operating performance which includes the execution of the aforementioned method.

The exemplary embodiment also includes a method of storing stored data in a data storage system and obtaining data from the data storage system. The data storage system comprises a software application, one or more database systems and a plurality of cache nodes. The software application is configured to receive a user request requesting reading of at least one data or writing of one of the data. The software application is further configured to send read and write queries to the data storage system to process user requests. The method is characterized in that the software application uniquely connects with one or more database systems and a plurality of cache nodes, and the method comprises the following steps performed by the software application comprising at least one data processor. I assume.
Upon receiving a user request requesting to read at least one data, the software application sends only read queries to multiple cache nodes.
If the software application receives queried data (ie, acquired data) from at least one cache node, the software application processes the user request with the queried data.
If the software application receives losses from all the cache nodes in response to a read query, the software application retrieves one or more database systems, and is queried if the queried data is present in the database system After obtaining the data from the one or more database systems, the software application processes the user request with the queried data, sends the queried data to at least one cache node, and at least the queried data. Send an instruction to add to one cache node. If not found in the database, add information to the cache indicating that the data does not exist.
Each data is stored identically in at least one cache node and one or more database systems of multiple cache nodes, or stored in such a way as to guarantee consistency of processing of the same data in cache and database Be done.

  Optionally, but advantageously, upon receiving a user request requesting writing of at least one data, the software application sends an indication of writing to one or more database systems and sends only queries to multiple cache nodes . The one or more database systems also send instructions to populate each of the lost read queries of the data storage system and multiple cache nodes of each write query.

In another aspect of the invention, a method of storing data in an airline inventory data storage system and a method of acquiring data from the data storage system are provided. The data storage system comprises a software application, one or more database systems, and a plurality of cache nodes. The software application is configured to receive a user request requesting at least one of the following: Reading data to know availability for at least one flight and writing data to correct availability for at least one flight. The software application is further configured to send read and write queries to the data storage system to process user requests. The method is characterized in that the software application uniquely connects with one or more database systems and cache nodes, and the method comprises the following steps performed by the software application comprising at least one data processor. .
Upon receiving a user request requesting to read data to know availability for at least one flight, the software application sends only read queries to multiple cache nodes.
If the software application receives queried data (ie, retrieved data) from at least one cache node, then the queried data is used to process the user request.
If the software application receives losses from all cache nodes, the software application reads one or more database systems. If the queried data is in a database system, after obtaining the queried data from one or more database systems, the software application uses the queried data to process the user request and at least the queried data. Send to one cache node and send an indication to add the queried data to at least one cache node.
Each piece of data is stored identically in at least one cache node and one or more database systems among the plurality of cache nodes.

  Optionally, but preferably, the user request for at least writing is a user request for seat purchase, seat cancellation, or seat change, in order to modify the availability for at least one flight. .

  In another aspect of the present invention, the present invention provides a data storage system. The data storage system comprises one or more database systems, at least one cache node, at least one data processor, and a software application. Execution of the software application on at least one data processor leads to operating capabilities including execution of any one of the aforementioned methods. The one or more database systems and the at least one cache node are independently configured to be driven by the software application.

  Advantageously, the number of cache nodes and the processing power of the computerized means for executing the software application are adapted to match the aggregated peak traffic created by all end users of the software application.

  Optionally, the data storage system according to the invention may comprise any one of the following optional features and steps:

  Adapt the number of cache nodes and storage resources and keep all database system contents.

  Some data of the database system are stored in two or more cache nodes.

  If the system content of all the databases has been transferred by the software application to at least one cache node, the hit ratio query of at least one cache node will eventually reach 100%.

  Another aspect of the invention is to provide an inventory of travel providers comprising the data storage system of the invention.

1 shows a data storage system according to the invention. It illustrates the process of an end-user requesting and eventually acquiring in the application data not yet present in the cache. Describe the process of writing from the application to the database and cache simultaneously. In the specific case where simultaneous write operations occur, we illustrate the process of acquiring cache data from a database. It further shows details of the timing of the data being written to the database and cache simultaneously by the application. This illustrates the case where the requested data does not exist in either the cache or the database. Illustrate the case where database writes and caches are deleted.

  The following detailed description of the invention refers to the accompanying drawings. While the description includes representative embodiments, other embodiments are possible, and modifications may be made to the described embodiments without departing from the spirit and scope of the present invention.

  FIG. 1 illustrates a data storage system 100 according to the present invention. The software application 10 uniquely connects to the database system 20 on the one hand and to a cache system, also referred to as a cache, on the other hand, and comprises one or more cache nodes 30.

  It is worth noting that the inventive database cache system described below is specific. This is mainly because the entire contents of the database are eventually transferred to a set of cache nodes. This cache node acts as a front end processing layer that protects all read traffic. If the cache node does not protect the traffic being read, then the traffic being read will reach the database system 20, thus significantly improving the performance of the data storage system 100. A sufficient number of cache nodes are then deployed to support the entire traffic and process the entire contents of the database together. Thus, if the system has been up and running for a significant period of time, all data entities contained in the back end database will eventually be transferred to a set of cache nodes or reside in cache nodes. As a result, all read queries are handled by the cache node so that no cache loss occurs anymore. Database writes are systematically implemented in caches and databases. As such, cache and database content are always consistent. Even though the data storage system described below uses such a high speed front end storage and processing system as a repository of data, the term cache is used in the following description of the invention.

  Data storage system 100 takes on the traditional tree hierarchy architecture commonly used in data processing systems. The middle hierarchy 120 is a software application 10 hierarchy, and from the software application 10 hierarchy, the service provider's proprietary software application 10 is executed. In the example used previously in the GDS, this is usually the inventory management application of any airline, whose purpose is to manage the reservations of all airline holdings and reservations of seats.

  The client hierarchy 130 comprises all of the remote users 40 of the application 10. In the case of travel applications set up by service providers, such as the aforementioned airline inventory, end users are typically travel agents of traditional travel agencies. End users also have individuals. The individual uses any of the many available travel websites or online travel agencies, from which travel requests are made and in some cases online travel bookings using airplanes.

  The lower tier is a storage tier 110, which comprises a database system 20. The present invention does not consider the database system used by the service provider. Most of them are based on commercially available standard database management systems (DBMS), but they may be proprietary database systems. Which database system is used by the service provider, implemented from a sufficient amount of hardware and software resources, maintains and processes all data of the service provider. In FIG. 1, all hardware resources required to implement data storage system 100 are shown as machines such as the individual computers generally referred to at 101. Persistent, non-volatile storage is considered available from each individual computer and, if necessary, another data disk 102, for example, permanently retaining the contents of the database.

  The data storage system of the present invention comprises a storage tier 110 and an intermediate tier 120.

  In the present invention, the terms "user request" or "request" refer to a request from the user 40 and reach the application 10. The user may be an individual, such as a traveler or a travel agent, or may be a computerized system that sends a request.

  In the present invention, the terms "data query" or "query" refer to a request that is sent from the application 10 to the cache node 30 and / or to the database system 20. The query may be a read query or a write query.

  The read query includes at least an instruction obtained from the cache node or an instruction to read data from the database system. Usually, the act of acquiring data from a database system is indicated as "read" and the act of acquiring data from a cache node is indicated as "get". The queried data is, at least in part, data that must be obtained or read to fulfill the user request.

  The write query includes instructions to add, update / set, or delete data. Usually, the action to modify data from the database system is referred to as "updating" and the action to modify data from the cache node is referred to as "setting up".

  Thus, in the following invention, the application 10 receives a user request and sends a data query. These queries are read or write queries.

  Whichever system is actually used, the present invention deduces that the database 20 is the service provider's ultimate data repository. The database 20 then preferably observes the complete set of ACID (atomicity, consistency, independence and permanence) properties, thus ensuring that database transactions are processed with respect to atomicity, consistency, independence and permanence. Guarantee that.

  With respect to the aforementioned database system known in the prior art, the software application 10 of the present invention thus remains directly connected to the database 20 through the dedicated interface 12. Thus, the operation of the database system is not affected at all by one or more cache nodes 30, which have their own dedicated interface 14 with the software application 10. As discussed further in the following description of the invention, it is up to the software application 10 to send only the required transactions that the database has to process to the database. That is, the contents of the database may be permanently updated as a result of having to change some aspect of the reservation, for example, because a new reservation has been completed and, for example, a cancellation occurs.

  Therefore, none of the cache nodes 30 connect with the database system 20. No messages, instructions or data are exchanged between the database system and the cache node 30.

  In the data storage system 100, all traffic that the software application 10 processes is then supported through the interface 14 of the dedicated cache software application 10. As shown in FIG. 1, caches are functionally arranged in storage tiers such as databases. The interface 14 and one or more cache nodes 30 are hard enough to meet the expected throughput in the hierarchy 120 of the software application 10 and in the storage hierarchy 110 for the cache nodes, whatever throughput is the goal It is assumed that all traffic of data storage system 100 can be processed by only providing and deploying hardware and software resources. Thus, processing a large amount of data can only be achieved by adding many computational and storage resources to the resource. In this way, the scalability of the system is not limited by structural considerations other than the number of computer platforms that need to be deployed to achieve the targeted throughput, ie its cost, power loss and footprint.

In order to make the aforementioned extensibility efficient, the data storage system 100 is based on a global key / value data model that is consistent in cache and database. Therefore, both can be retrieved using the same key. The data model is thus chosen to be directly mappable to the database and cache. In particular, each set of data is grouped by functional entity and indexed by a common unique key. This allows overall immediate access from both the database and cache unique keys, even if the content is somewhat different. The only requirements of the data model to operate as above are:
Pre-update ability to lock the superset of data to be updated in the cache.
-Possibility of guessing to update all cache keys in the database that are affected by a given update.

  Typical examples in the travel industry are as follows:

式中、
（＊）Ｏ＆Ｄ＝出発地と目的地
（＊＊）区間はフライトの一部である。たとえば、ニース（ＮＣＥ）からニューヨーク（ＮＹＣ）まで、パリ（ＣＤＧ）での途中降機があるフライトを例に取る。このフライトには２つの区間がある。ＮＣＥ−ＣＤＧおよびＣＤＧ−ＮＹＣまでである（フライトが３つのＯ＆Ｄを有することに注意されたい。Ｏ＆Ｄ：ＮＣＥ−ＣＤＧ、ＮＣＥ−ＮＹＣおよびＣＤＧ−ＮＹＣである。）

During the ceremony
(*) O & D = departure and destination (**) sections are part of the flight. Take, for example, a flight with a stopover in Paris (CDG) from Nice (NCE) to New York (NYC). This flight has two sections. NCE-CDG and CDG-NYC (note that the flight has 3 O & D. O & D: NCE-CDG, NCE-NYC and CDG-NYC)

  In the above example, the schedule information is stored in the associated database. The "Mother" table has flight-date primary keys. One of the "child" tables has an interval-date primary key. Some writes (e.g., updates) occur at the flight level and other writes occur at the interval level. Locking at the flight level is done in both cases. This is used to ensure that no corrections are made on the flight and all sections of the flight. Locks can not be set at the interval-date level. This is because a flight update can update all the segments next and lead to a simultaneous update.

  Thus, the database and cache data models must be consistent if not identical. Thereby, the same symbolic key can be derived to access the cache and database records while allowing the database records to be locked.

  The architecture shown in FIG. 1 uses a cache configured as a single-tier client side as a distributed cache that supports overall throughput and also greatly simplifies the management of cache data consistency. Having a client-side distributed cache means that the data distribution among the various cache nodes 30 that make up the cache is known and calculated on the client side in the software application 10 hierarchy. As a result, all cache nodes 30 are therefore completely independent and the scalability of the system is practically unlimited. However, in practice, adding a new cache node 30 to the storage tier to further obtain processing power can be realized only when distribution of data among the nodes is also balanced. Data is distributed based on the distribution key properties so that the distribution is actually balanced. For example, data on flights may be distributed based on flight numbers. A modification that results in a change in distribution due to changes in the number of available cache nodes or distribution parameters is also supported, for example, by a normal redistribution procedure. In this procedure, the entire cache system is kept online and operates at nominal conditions while redistribution takes place. To this end, temporary duplex provisioning for two cache configurations will be described later in the following description of the invention.

  The data storage system 100 of the present invention does not require any synchronization mechanism between cache and database. The cache is explicitly used by the software application 10. That is, for example, when a database or cache must be written, it is decided during the same user request whether to use two data sources, one or both of the database or cache Is a software application 10 hierarchy. As a direct consequence of this approach, the database is not aware of the existence of cache at all and is not affected by the presence or absence of the cache of the data structure of the present invention. The reverse is also obviously true. The cache is completely disconnected from the database. Thus, both structures completely evolve independently, as needed.

  It should be noted that data writes in the cache do not use an invalid policy. Every write results in the immediate exchange of data with the cache. The contents of the entire database are eventually mapped to caches and distributed to all available cache nodes 30, and even if very high levels of simultaneous writes occur, the hit ratio is 100%.

  Cached data can always be considered valid and does not require an additional process to check its validity. In fact, all caches miss the trigger to add lost values from the database to the cache. This is ultimately done, thus, to minimize the load on the database retrieved only once per data entity. This often occurs after maintenance operations when the cache becomes operational, for example, after booting the system after adding cache node 30, and after a failure or cache node 30. The present invention assumes that there is sufficient room in the distributed cache node 30 to accept the entire contents of the database.

  The cache also records that the data requested by the end user is not in the database. If the data requested by the end user can not be found in the cache and can not be retrieved from the database, the data absence will be recorded in the cache and the next time the cache is queried, the corresponding data will be limited to limit the database load. No attempt is made to retrieve from the database.

  The architecture described in FIG. 1 can be extended to any kind of data that can be key-value based. Also, it is applicable to any process that can take a key-value scheme. In particular, it is applicable to any process designed to confirm the availability of a flight.

  The following diagram describes the operations that the software application 10 performs to obtain its cache, between the database and the cache. The cache ultimately supports the entire traffic generated by the software application 10 and handles all user requests.

As mentioned above, the cache portion of the system is fairly simple and consists of one or more independent computers that provide the basic remote key / value protocol. Three basic operations on the cache are defined as follows. That is, having the software application 10 update the cache, having the cache input from the database, and having the data be obtained from the cache.
Setting (key, value): Update the value associated with the key unconditionally.
Add (key, value): Add the value associated with the key if it is not already in the cache.
Get (key): Return the value associated with the key from the cache.
The present invention makes no assumptions about the method implemented by the software application 10 as long as the expected level of performance is realized. Advantageously, bulk operation is defined. Bulk operations allow multiple basic operations to be sent together and processed.

The main part of the system is in the software application 10 hierarchy, which controls data distribution across cache nodes 30. Key / value data is distributed among the nodes that make up the cache. The properties of the key are extracted and the corresponding cache node 30 is calculated according to the following formula so that it can be distributed as uniformly as possible across all nodes.
Node_number = key as number_property_MODULO node number

  Data about flight uses properties. Consecutive flight numbers are usually used for flights having the same property. In this case, the flight number is used directly as the basis of the variance.

  The hash value is calculated with a single O & D key, for data about the flight based on the flight origin and destination (O & D).

  As mentioned above, balancing data distribution on all available nodes is really important to achieve unlimited scalability.

  FIGS. 2 and 3 illustrate how the cache is consistent with the contents of the database with only the way data is entered into the cache and the control of the software application 10.

  FIG. 2 describes the process. The process eventually allows the end user to request in the software application 10 to obtain data that is not yet present in the cache. This situation often occurs, for example, when data is entered into the cache after system startup, or a new node has been inserted or removed, and it is in progress to rebalance the contents of cache node 30 And so on.

  When the software application 10 needs to respond to a user request, the cache is first read through a "get" operation 210. In the example of an airline inventory database, this is, for example, to answer one of a number of user requests issued by end users of the database. A user request is made to determine if a particular flight seat is available for a particular date, a particular class, etc. If the corresponding data is not in the cache, that is, generally, if the corresponding data has not yet been incorporated into the cache by a previous read, then the cache then returns a "loss" 220 to the software application 10. If this is not the case, it is clear that the information is simply returned from the cache to the software application 10 sending the "get" operation. The software application 10 can thus execute the end user's user request. Finally, software application 10 aggregates the queried data with additional data and returns it in response to requests from end users. The additional data is usually another data that may or may not be obtained to fulfill the user request. For example, some data may be obtained from a cache node, but other data needed to perform the same user request may also be obtained from other cache nodes and / or read from database system 20.

  Upon receiving information that the queried data is not in the cache, the software application 10 queries the database in a "read" operation 230. The loss information is then returned 240 to the software application 10. Reading of data from the database is performed by the database dedicated interface 12 as described above. This is done by issuing the corresponding query from the software application 10 to a database management system (DBMS) that the data storage system 100 of the present invention uses.

Upon receiving non-cached data from the database, software application 10 then performs an "add" operation 250 to cache the data. From this point on, data is in the cache 270, unless the cache is operational and reconfigured. When this operation is completed, an acknowledgment (OK) 260 is sent back to the software application 10.

  It should be noted that this process is performed only once while the cache is updated and operating for any given data that is stored identically or consistent with the database and cache node 30. This first occurs when data has been requested by the software application 10 but is not yet present in the cache. After this, for example, if the airline seat is sold and the contents of the database need to be changed, the corresponding data may be updated. In this case, as described below, the software application 10 does not have to re-execute the process of FIG. 2 in order to update both the cache and the database.

  FIG. 3 describes the process of updating the database and cache from software application 10 simultaneously.

  To maintain consistent database and cache content, software application 10 constantly updates both cache and database. The cache update is then performed in the "set" operation 310 as described above. At the same time, database "update" 305 is implemented using the query language of the DBMS being specified. Update 320 is efficient after the application commits the operation to the database.

  More specifically, settings are not performed when the update is performed in the database, but are performed when the commit is performed. Depending on the application, the data remains in memory until it is committed. There may be multiple steps between update 305 and configuration 310. However, configuration 310 and commit 320 are intended to be implemented subsequently.

  In steady state, i.e. after the system has been started up and running for a long time, all the contents of the database are eventually taken over and distributed across all cache nodes 30. Next, update operations, ie content updates, inserts and deletes, are the only operations that need to be performed on the database interface. As a result, the database load is significantly reduced. In the case of a delete operation, invalidation of corresponding data is triggered in the cache described in FIG.

  Also note that the cache of the present invention is input from both read and write operations, as the process of FIG. 3 does not assume any specific conditions to be performed to write to the cache. It does not. This allows the input of the cache node 30 after startup to be significantly faster than in a system that uses read only for cache input. This is possible and is done simply as such. As mentioned above, the data entities stored by the database and cache both continue to be updated. Updates do not continue to be made in other caching solutions where the contents of the database and cache generally differ widely. This is to try to minimize cache storage requirements, or because the cache data entities provided to the software application 10 are constructed by disassembling data extracted from various parts of the database.

  FIG. 4 describes the process of FIG. Specifically, the case is the simultaneous writing (e.g. updating) of the database is requested from the software application 10 and thus interferes with its execution.

  Similar to the scheme described in FIG. 2, the process starts at 210 "getting" data from the cache. This is followed by a "loss" 220, triggering retrieval 230 of the data being lost from the database. However, the lost data is typically returned 240 to the software application 10. Write queries 410 for the same data are also received from software application 10. Writing is performed as described in FIG. In the cache, writes are done using the "set" operation 310 and the "update" operation 305 in the database. When a "set" is issued to the cache and a "commit" 320 is sent in the database, the corresponding data becomes immediately available 420. The application holds the data in memory ("set in memory") before the configuration is triggered.

  Second, in this particular case, the cache content should not be further updated by the "Add" 250 below. This is the result of the act of retrieving 230 the lost data from the database, as the database has been updated in the meantime. "Add" 252 is then actually interrupted. A negative acknowledgment (KO) 262 is returned, notifying software application 10 that a cache update is not actually being performed by the "add" operation.

  Thus, in order to update the cache with data read into the database, the present invention uses additional commands. Thereby, the data can be sent to the cache without locking the data to the database. In fact, when trying to add data, it is efficiently added if the data is not yet in the cache. If it has been updated by the update process, the addition will fail but is expected. The update process has a lock on the database and has a priority to update on this key. Therefore, it is normal for this to remain in the cache.

  These aspects of the invention allow, among other things, the database system and the cache to integrate very smoothly with the update process, as they can neither lock nor affect the performance of each other. . At the same time, the load on the database is maximally reduced, as the data is read in the database only once.

  FIG. 5 describes in more detail the timing of data updates simultaneously implemented in the database and cache by the software application 10.

  Software application 10 initiates an update transaction by issuing a corresponding query 510 to the database to obtain the currently stored values. At the same time, the database management system (DBMS) locks the currently stored values so that simultaneous updates are not made from another software application 10. Within the software application hierarchy, data is processed by software application 10. Once the data is ready to be updated by the DBMS 530, an update of the buffer cache 540 is also implemented in the software application 10 to hold the new data to be transferred and stored in the cache.

  Software application 10 may then commit change 550, which is immediately implemented in a “set” operation at cache 552 and committed to database 554. Thus, it will be noted that the new data is actually committed to the database and becomes available to the cache 556 shortly before it becomes available 558. The reference 556 indicates the time frame that will be updated and available to the end user in the cache when it is not yet available to the database system 20.

  For example, if you commit for some reason, such as a hardware and / or software failure, but do not finish normally, the previous cache write operation, the "set" operation 552, is rolled back, so the cache contents Will not change. Thus, if the commit fails, a "commit KO" 560 is sent to the application, which then issues a delete 562 to the cache to delete the added data. As a result, during that period 564, an incorrect value will be present in the cache.

  Thus, the highest performance, high impact data quality is provided to the cache, regardless of database. Updates are communicated to the cache with "commit request" data using write-ahead commit. If different constraints are barred by cached data, then the data in the cache will be "prepared" to be "pre-formed" in comparison to the database, but in particular the very expensive normal 2 Without the staged commit architecture, there is no additional cost incurred. Such quality meets the data quality requirements for available requests and can also be considered as an advantage from the perspective of the final client.

  FIG. 6 describes the case where the queried data is neither in cache nor in the database. This deals with the case where the end user requests information that the database does not hold.

  When such a situation occurs, the corresponding data absences are also cached to avoid further queries to the database. The next time the cache is queried from the software application 10, the information that the queried data is absent in the database is provided directly by the cache itself, thus further reducing the database load.

The process is similar to that described in FIG. When the "get" operation 210 issued to the cache returns a "loss" 220, the reading 230 of the corresponding data in the database also returns the database "loss" 640 to the software application 10. Next, data absences are added to the cache 650. Similar to the data, the data absence is immediately available 270 in the cache, which returns a response 260 to the software application 10.

  According to a non-limiting embodiment, each data is associated with a header to form a record. The header indicates whether or not the content is lost in the database system 20. Therefore, it is possible to know whether it is worth extracting the database system just by reading the header of the record. According to an alternative embodiment, the cache node stores specific values associated with the data. This particular value indicates that the data does not exist in the database. Therefore, it is possible to know whether it is worth retrieving the database system simply by reading the value of the record.

  FIG. 7 illustrates the case described above in FIG. In this case, a particular update operation from the application removes data 705 from the database. This operation is generally performed as described in FIG. 3, but the deleted data is not actually deleted from the cache, but is indicated by displaying as "data absent". As the deletion is database committed 320 by the application, the corresponding information is cached in a particular "set" operation 310. "Data not available" is immediately available 330. Thus, as mentioned above, if the cache is later queried, it can provide information directly that the requested data is no longer in the cache or in the database.

  In the following, we will discuss, for example, the cases where it is necessary to modify the configuration of the database system of the invention in order to cope with traffic increase. In order to extend the system configuration shown in FIG. 1, additional cache nodes have to be added to provide more cache storage capacity and to distribute increasing traffic to more processing nodes. However, with a large number of nodes, and generally speaking, whenever you have to change the number of active nodes, recalculate the key to work on the data uniquely at the nodes and in fact all traffic is equally new It should be able to spread over the complete set of nodes.

  The present invention does not envisage any specific way of calculating keys from data entities stored and retrieved identically from databases and caches. In most cases, depending on the type of data being processed by a particular application, a key that has been hashed node processing by using some hashing functions and then taking modulo the number of nodes and further calculating Simply pull it out. Thus, when the number of nodes is changed, different results are obtained to obtain a particular data entity. Specific data entities may need to be located at different nodes in the new configuration. The problem arises from the fact that the configuration is not updated and the database system has to be implemented transparently while fully operational. Not all cache clients are simultaneously aware of the new configuration. This means that writing of part of the data may be done based on the new configuration, while other writing may use the old configuration. As a result, data sets may be inconsistent between the cache and the database.

  The present invention addresses this problem by enabling a procedure called "dual delivery". Dual delivery is referred to as "double delivery" in order to maintain one additional configuration in addition to the configuration typically used for caches. The additional configuration is not the default, but can be launched on configuration change. Once activated, all write operations are sent to both standard and duplex configurations. Time to live (TTL) is a property associated with each item in the cache. As the name implies, the survival time corresponds to the time the item is valid. When the lifetime expires, the item can no longer be retrieved from the cache, leading to a cache loss as if data had been lost. This can be set by the configuration such as one in the standard configuration or one in the multiple configuration. The item does not expire unless the lifetime is set.

The activation of duplex configurations is also not atomic, so it must first be activated with a short survival time. Once the dual feed configuration is fully activated, the survival time can be deleted. Standard and duplex configurations can be exchanged only once the lifetime expires. Once the configuration change is complete, the dual supply can be stopped. During the step in which the configuration is being propagated (dual start / stop), some "invalid" data is written but only at unread locations. Thus, the procedure is as follows.
-Creation of dual supply configurations with short TTL.
-Start of duplex configuration, wait for its propagation.
Removed a short TTL from a dual supply configuration.
-Exchange of standard and dual delivery configurations. Wait for propagation.
-Stop of compounding.

  The complete set of procedures that can make any changes on the system online is described below.

The proposed architecture provides extensibility so that not all systems later get into a position not to work properly without a cache. To address such situations, according to embodiments of the present invention, all maintenance operations are intended to be performed online, and at most one node (or equal proportion of traffic) at the same time (e.g. cache) Upgrade nodes or replace them one by one, global cache changes implemented using a dual-supply mechanism, and reduce the impact on the database.
Cache node upgrades or exchanges occur one by one. The system preferably uses a database to obtain the data that this node was to host.
-Global cache modification is usually the addition or deletion or modification of multiple cache nodes, causing a significant change of global distribution, and implemented using a dual supply mechanism as described in the previous paragraph.

From the foregoing, it is apparent that the present invention can maintain data consistency between cache and database. It does not strictly adhere to ACID, but is highly extensible, does not affect the database, allows 100% hit rate, and above all, a mechanism that fully meets the data quality needs Thanks to Furthermore, the present invention allows caches to be highly dynamic data, ie, typically dozens of writes per second per unified data, while benefiting from the benefit of reduced cache load.

Claims (14)

A method of storing data in a data storage system (100) and acquiring data from said data storage system (100), said data storage system (100) comprising a software application (10) and one or more databases. A system (20) and a plurality of cache nodes (30), the software application being configured to receive a user request, the user request requesting at least one read of data or a write of one of the data The software application is further configured to send a read query or a write query to the plurality of cache nodes (30) to process the user request, the method comprising: That the connection uniquely connects with the one or more database systems and the plurality of cache nodes (30), and the method comprises the following steps performed by the software application comprising at least one data processor: Characterized in that said step is
Said software application (10) transmitting only a read query (210) to said plurality of cache nodes (30) upon receipt of a user request requesting read of at least one data;
If the software application (10) receives queried data from at least one cache node (30) in response to the read query, the software application (10) uses the queried data Processing the user request;
If the software application (10) receives losses (220) from all cache nodes (30) in response to the read query, the software application (10) may be the one or more database systems (20) And retrieve the data from the database 230 and obtain the queried data from the one or more database systems (20), the software application (10) processes the user request using the queried data. Sending to the at least one cache node (30) the said queried data and an indication to add (250) the said queried data to the at least one cache node (30),
Upon receiving a user request requesting writing of at least one data, the software application (10) sends a writing indication to the one or more database systems (20) and, simultaneously, the plurality of writing indications. Transmitting to the cache node (30), thereby adding data to the plurality of cache nodes (30) in each lost read query and each write query of the data storage system (100);
The number of cache nodes and storage resources are adapted to maintain the content of all the database systems
Method. The method according to claim 1, wherein the write query comprises at least one of addition, update and deletion of data of the database system (20).   The software application (10) receives an acknowledgment (260) as to the successful completion of adding the queried data to the at least one cache node. The method described in.   If the writing of data (410) occurs while simultaneously obtaining the same queried data from the one or more database systems, the queried data may be added later to the at least one cache node (252) ) Is interrupted and a negative acknowledgment (262) is returned to the software application (10), whereby the software application can use the written data instead (420). The method according to any one of 3. Sending an indication to write data to the one or more database systems (20) and sending an indication to simultaneously write the plurality of cache nodes (30),
Obtaining (510) currently stored data to which the writing is applied from the one or more database systems, and locking the currently stored data;
Writing 530 new data to be stored in the one or more database systems;
Writing (540) a cache buffer to the software application (10) and temporarily holding the new data to be stored;
Forwarding 552 the set of new data to be stored in the at least one cache node;
Committing a transaction to the one or more database systems (554);
The method according to any one of the preceding claims, wherein is performed.   The method according to claim 5, wherein if the commit fails, the software application (10) sends an indication to the at least one cache node to delete the previously set new data. If the queried data is not in either the one or more database systems or the plurality of cache nodes (30):
After retrieving (30) data from the one or more database systems (20), a loss (640) is returned to the software application in place of the queried data.
The software application (10) transmits absent data (650) to at least one cache node (30), and the absent data (650) is transmitted to the at least one cache node (30) And the absent data is immediately available (270) for all subsequent read queries,
A method according to claim 1, wherein the software application (10) avoids further retrieving data from the one or more database systems (20) in order to obtain the lost queried data. The method according to any one of 6.   The software application (10) is uniquely connected to the one or more database systems (20) by the first dedicated interface (12), and the plurality of cache nodes (30) and the second dedicated interface (14) The method according to any one of the preceding claims, wherein the connection is made. The data model of the cache node and the database system is consistent, whereby exactly the same processing key is obtained to access data of the cache node and the database system. Or the method described in paragraph 1.   When data is present, the same data is stored in the database system and at least one cache node, or in such a way as to guarantee the consistency of the processing of the same data of cache and database system The method according to any one of claims 1 to 9.   A computer program product containing software program instructions, wherein said software program instructions are executed by at least one data processor, comprising performing said method according to any one of claims 1 to 10. , Leading to the implementation of the operation, the program.   A data storage system (100) comprising one or more database systems (20), at least one cache node (30), at least one data processor, and a software application (10), said software application 11. Execution by the at least one data processor leads to the performance of operations including performing the method according to any one of claims 1 to 10, wherein the one or more database systems (20) and The system, wherein the at least one cache node (30) is independently launched by the software application (10).   The data storage system (100) according to claim 12, wherein part of data of the database system (20) is stored in two or more cache nodes (30). A travel provider inventory management system comprising a data storage system according to claim 12 or 13.

Images