FR3077146A1 - RECALCULATION OF PRECALCULATED INTERVIEW RESULTS - Google Patents

Abstract

Precalculated query results stored on a database server are recalculated by a computing device. The precalculated query results are composed of a plurality of parts. Each share includes a number of precalculated query results. The calculation means of the computing device, necessary for recalculating a precalculated query result of a first part, depends on recalculating, or not, other pre-calculated results of the first part which are recalculated during an interval. given time by forming a current computation cycle. The computing device receives a request to recalculate precalculated query results for at least a first portion, determines a number of pre-computed query results in the first portion to be recalculated during the current recalculation cycle based on a reinforcement learning algorithm, and recalculates the determined number of precalculated query results in the first part.

Description

RECALCULATING PRE-CALCULATED QUERY RESULTS

FIELD OF THE INVENTION

The present invention relates generally to database technology. More specifically, it aims to recalculate pre-calculated search results stored as database records.

CONTEXT

A common problem in database technology is to ensure short response times to database queries or queries that require complex, time consuming processing and significant resources, eg. ex. due to large volumes of data. This computing power consumer processing must for example be implemented in response to so-called "open queries" which contain very little information entered (eg only one or two parameters among a dozen possible parameters are specified and / or the specified value of the value ranges is wide) and, therefore, generally return a large number of results. The possibilities for speeding up data processing by increasing hardware performance are limited. Attention is therefore focused on improving the mechanisms underlying complex, long and resource-intensive processing in order to answer these questions.

A general approach to shortening response times is to prepare results to be returned in response to expected queries (eg by calculating or collecting these results in advance) and to keep the corresponding query results in a reserve of pre-calculated interrogation results. It follows that the requests are not currently processed and the responses based on the large base of original data, but are directed towards the pool of precomputed query results.

Recalculating a large number of precomputed query results is a task that requires a substantial amount of computing resources. Thus, recalculation mechanisms are sought to effect the recalculation of the precalculated query results.

Patent application n ° 18 50581 A5013307FRPOOMm

EP 2913764 relates to the recalculation of precalculated search results where the computing resources necessary to recalculate a precalculated search result i generally depend on the fact of recalculating, or not, other precalculated search results associated with the precalculated search results i which are recalculated during the same calculation cycle. Therefore, the computing resources for recalculating the pre-calculated search results are therefore not static, but vary depending on the selection of a set of pre-calculated search results to be recalculated during a calculation cycle. EP 2913764 proposes an iterative estimation process according to which a recalculation controller iteratively selects a number of precalculated search results to be recalculated by the calculation platform during a given time interval forming a recalculation cycle. The number of precomputed search results is limited by the computing resources available during the given time interval. After the initialization of the computing resources Ci necessary to recalculate each precalculated search result i with a value assuming the calculation of i independently of the calculation of other precalculated search results selected for recalculation during the given time interval, a first portion of the pre-calculated search results is selected for recalculation in a first selection iteration, in which this first selected portion requires only a percentage below 100% of the global computing resources available for recalculation in the time interval given. The computing resources Ci necessary to recalculate each pre-calculated search result i are then re-evaluated taking into account the pre-calculated search results associated with the pre-calculated search result i which have been selected for a recalculation. Other iterations to select other portions of the pre-calculated search results that have not yet been selected and to reassess the computing resources Ci necessary to recalculate each pre-calculated search result i are then performed if less than 100% of the global resources computations available for recalculation in the given time interval have been exhausted.

ABSTRACT

According to a first aspect, a calculation device is provided for recalculating precalculated query results stored on a database server. The precalculated query results are composed of a plurality of parts. Each part includes a number of pre-computed query results. The computing resources of the computing device, necessary to recalculate a precalculated result

Patent application n ° 18 50581 A50133O7FRP0OMm of interrogation on the first hand, depend on the fact of recalculating, or not, other precalculated results of the first part which are recalculated during a given time interval by forming a current calculation cycle. The given time interval forms a current recalculation cycle. The computing device is arranged to receive a request for recalculation of the precalculated query results for at least a first share. The calculation device is also arranged to determine a number of precalculated query results in the first part to be recalculated in the current recalculation cycle on the basis of a reinforcement learning algorithm. The determined number of precomputed search results is limited by the available computing resources of the computing device during the given time interval. The calculation device is also arranged to recalculate the determined number of precalculated query results in the first part.

According to a second aspect, a corresponding method executed by a computing device is provided.

In a third aspect, a computer program stored on a storage medium is provided which, when executed by a computing device, equips the computing device with the corresponding functionality.

BRIEF DESCRIPTION OF THE GRAPHICS

The present invention will be described with reference to the accompanying graphics. Similar reference numbers generally refer to functionally identical or similar elements.

FIG. 1 schematically describes an example of a database system storing precomputed query results.

FIG. 2 is a visual representation of an effect of decreasing the probability of validity of the pre-calculated interrogation results over time.

FIG. 3 illustrates alternative computing resources, necessary for recalculating different exemplary parts of precalculated query results.

FIG. 4 is a more detailed diagram of a recalculation system as described herein.

FIG. 5 shows the basic elements of the reinforcement learning system.

FIG. 6 illustrates two examples of an aggregated calculation of key performance indicators.

FIG. 7 shows an example of the updating of a value function.

FIG. 8 is a schematic sequential flow with high-level processes implemented by the computing device and its subsystem described herein.

FIG. 9 illustrates an iterative approach based on reinforcement learning to determine an optimal number of precalculated query results to be recalculated during a calculation cycle.

FIG. 10 is a schematic exemplary view of the internal architecture of the database table and / or of a client request.

DETAILED DESCRIPTION

In order to be able to manage database queries or batches of calculation queries that require calculations based on large volumes of underlying data, the search results corresponding to expected queries are generally precomputed and stored as records from database to database. This database is queried by requesting entities (such as customers, applications, browsers installed on user terminals, etc.) during a search and pre-calculated search results that meet the criteria indicated by the query is returned to the client in response to the query. Thereafter, the term "interrogation" is used as a general term including any type of information retrieval queries such as transactional queries, queries for batch calculations and other forms.

FIG. 1 illustrates such a database environment 1 at an abstract level. Basic data, also referred to below as "calculation data", is stored in and / or accessed by a calculation device 3 which is connected to a recalculation controller 2. The latter issues recalculation orders to the calculation device 3 which , in turn, transmits the corresponding results to the search platform 4 and, moreover, to the recalculation controller 2 which also stores the pre-calculated search results for reasons of recalculation control. End users 5 such as applications on user terminals access the pre-calculated search results from the search platform 4. As shown in FIG. 1, one or more search platforms 4 can be present in the environment 1. If a plurality of platforms 4 is present, the precomputed search results can be stored in a distributed manner between several of the search platforms 4 and the controller Recalculation 2 can control the recalculations of all the pre-calculated search results distributed between the search platforms 4. The search platform 4 can also offer a heterogeneous set of pre-calculated search queries, p. ex. search platforms 4 store pre-calculated search requests relating to air transport, other search platforms 4 store pre-calculated search requests linked to insurance and other search platforms 4 store pre-calculated (or pre-extracted) requests linked to websites on the Internet. Such a heterogeneous environment can be controlled by a single recalculation controller 2 or by a plurality of recalculation controllers 2. Otherwise, the plurality of research platforms 4 can be used to reflect the same precomputed research results, for example, for reasons for redundancy.

The approach of precalculating search results and their storage on the search platform accessible to requesting customers leads to the general situation that the calculation data can change over time and, therefore, the precalculated search results become obsolete or invalid (both terms are used as synonyms herein). Pre-computed research results that are still relevant, i.e. which correspond to corresponding equivalent calculations in real time (results which would currently be calculated on demand without having pre-calculated search results) are hereinafter referred to as “exact” pre-calculated search results. Therefore, when the search platform that stores the precalculated search results correctly represents the current underlying state of the cached query results data domain, i.e. that the calculation data, the pre-calculated search results stored on the search platform are - in general - correct.

Generally, to allow the search platform to return correct results, one will want to maintain a high level of correlation between the pre-calculated search results that are provided by the query entity in response to queries from the database. and their real-time computing equivalents. At the same time, it is nevertheless desirable to minimize the consumption of computing resources caused by the recalculations, i.e. to avoid any unnecessary recalculation such as recalculation of cached query results which are still current. Computing resources are limited and generally there are not enough computing resources to recalculate all pre-computed search results at any time. Therefore, a compromise must be found between the accuracy of the pre-computed query results and the use of the available computing power.

In order to make the recalculation more efficient, metrics are defined to assess the "necessity" or "the absence of necessity" of a recalculation. For example, relaunching a global, massive recalculation every day may not be worth it, if less than half of the calculated query results turn out to be out of date. On the other hand, if categories of query results are known to change frequently, recalculating them several times a day may prove to be beneficial in terms of accuracy. Therefore, an effective way to assess or estimate the accuracy of research results is necessary, generally taking into account both attributable earnings, accuracy and the cost of recalculation. To this end, mechanisms model the decreasing validity of the pre-calculated interrogation results over time. Two of these exemplary functions of probable accuracy decreasing over time are illustrated in FIG. 2. Function 10 represents a precalculated search result which remains potentially more exact (or more precisely, remains at a high probability of being valid over time) than another precalculated search result associated with function 11. For example , the pre-calculated search result represented by function 10 has a 70% probability of being still valid 35 hours after its last recalculation while the validity of the other pre-calculated search result characterized by function 11 does not go beyond by 50% approximately 35 hours after its last recalculation. Functions 10 and 11 can also represent entire sets of pre-calculated search results and then indicate proportions of the pre-calculated search result sets likely valid at some point in the past since the last recalculation of the set.

Often, considerations of recalculating pre-computed query results assume that the recalculation of any pre-calculated search result by the computing device 3 requires substantially the same amount of computing resources regardless of whether the pre-calculated search results are calculated. separately, together with associated precalculated query results (eg from the same origin, destination and adjacent dates) or any other non-associated query results. However, in practice this assumption cannot generally be made, since certain precalculated query results and / or their corresponding underlying calculation data are, for example, interdependent. Recalculating these pre-computed interdependent query results together (i.e. during the same recalculation cycle) could include synergistic effects and could therefore be more effective than recalculating them separately. Therefore, when pre-calculated interdependent query results are recalculated together the calculation costs per pre-calculated individual query result are generally less than the cost of calculation per pre-calculated individual query result required if they are recalculated separately (c. i.e. during different recalculation cycles.

To facilitate understanding of the pre-calculated interdependent research results, a specific example will now be presented. This example is from a particular application of the distributed database environment 1 in the travel industry (i.e., the database 4 stores and computes pre-calculated travel related search results availability of end users. However, the following example is not intended to limit the pre-computed interdependent search results to such a travel-related data application. Instead, similar or analogous conditions for synergistic recalculation, and therefore more efficient from pre-computed interdependent search results, are present in database systems independent of the content of pre-calculated data sets. In general, we can say that any process of recalculating pre-calculated search results aims at a pooling of sub - recalculation tasks which must be performed fluently for any precalculated research result he of a set of pre-calculated search results. Thus, the group recalculation of the pre-calculated search results which have such recalculation sub-tasks in common is generally more favorable than the group recalculation of pre-calculated search queries which do not share similar recalculation sub-tasks.

In a particular embodiment, the pre-calculated search queries are for round trip flight data records, each specifying a travel origin and a destination, and a departure and arrival date (or a duration of stay relative to the departure date as an alternative to the arrival date). In this example, database 4 contains 5 pre-calculated round trip recommendations for any origin-destination pair and any departure-arrival date pair to be covered. Table 1 shows a small extract from pre-calculated travel recommendations kept in database 4, the extract being travel recommendations for the city pair Nice-Boston (“NCE-BOS”) and for departure dates from July 1 to July 5 with a maximum stay of five 10 days, the abbreviation "pc-fr x" means "number x of pre-calculated travel recommendations".

NCE-BOS Dep. 1 JUL. Dep. JUL 2. Dep. JUL 3. Dep. JUL 4. Dep. JUL 5. Ret. JUL 2. pc-fr 0 - - - - Ret. JUL 3. pc-en 1 pc-fr 5 - - - Ret. JUL 4. pc-en 2 pc-en 6 pc-en 10 - - Ret. JUL 5. pc-en 3 pc-en 7 pc-en 11 pc-fr 15 - Ret. JUL 6. pc-en 4 pc-fr 8 pc-en 12 pc-fr 16 pc-fr 20 Ret. JUL 7. - pc-fr 9 pc-fr 13 pc-en 17 + pc-en 21 Ret. 8 JUL. - - pc-en 14 pc-en 18 pc-en 22 Ret. JUL 9. - - - pc-en 19 pc-en 23 Ret. JUL 10. - - - pc-en 24

Table 1

The recalculation of a specific pre-calculated travel recommendation, for example pc-fr 9 15 relating to a departure date on July 2 and a return date on July 7, is more or less carried out by the calculation device 3 by the sub - following tasks:

1: Recovery of possible geographic routes between Nice and Boston including intermediate stops / connections, for example: Nice-Paris-Boston, Nice-London Boston, Nice-New York-Boston, Nice-Paris-New York-Boston, etc.

2: The identification of all possible flights on all routes recovered by activity 1: for example, Air France 123 and Emirates 7123 for the Nice-Paris segment, Air France 456 and American Airlines 564 and United Airlines 125 for the ParisNew York segment, etc.

at. This activity is performed for the departure portion of the trip on the departure date July 2 (subtask 2a) and

b. for the return portion of the trip on the return date July 7 (subtask 2b) because flights are generally not available daily.

3: Checking the availability of all flights by activities 2a and 2b, i.e. July 2 for the departure portion of the trip (= subtask 3a) and July 7 for the return portion of the trip (- subtask 3b).

4: The recovery of the tariffs which are applied to these flights on these dates. A typical fare is a rule that produces a price for the entire trip. However, the rates may also be valid for the departure and return portions of the trip on their respective dates in which case they can be recovered once for the departure portion (= sub-task 4a) and once for the return portion of the trip (= subtask 4b). Fares have restrictions on departure dates, return dates, applicable flights, and many others. Tariffs can be combined, reduced in specific cases and so on.

5: The best solution, i.e. the correspondence at the best price is retained, returned by the calculation device 3 and stored by the database 4 under pc-ff 9.

These subtasks are performed in a similar manner for any pre-calculated travel recommendations. By performing the recalculation for the complete set of pre-calculated travel recommendations (pc-fr 0 to pc-fr 24), several recalculation subtasks can be pooled. Execution of sub-task 1 is necessary only once for the complete set of travel recommendations pc-fr 0 to pc-fr 24 because all these pre-calculated travel recommendations belong to the same pair of cities of origindestination, Nice -Boston. Thus, subtask 1 does not need to be re-performed for each precalculated search result pc-fr 0 to pc-fr 24.

Sub-tasks 2a, 3a and sub-task 4 are, on the other hand, specific to a start date. They can therefore be reused for all pre-calculated travel recommendations relating to the same departure date. This is indicated in Table 2 for the 5 pre-calculated travel recommendations pc-fr 5 to pc-fr 9 relating to the departure date of July 2:

NCE-BOS Dep. JUL 1. Dep. JUL 2. Dep. JUL 3. Dep. JUL 4. Dep. JUL 5. Ret. JUL 2. pc-fr 0 - - - - Ret. JUL 3. pc-en 1 Common subtasks: 2a, 3a, 4a - - - Ret. JUL 4. pc-en 2 pc-en 10 - - Ret. JUL 5. pc-en 3 pc-en 11 pc-fr 15 - Ret. JUL 6. pc-en 4 pc-en 12 pc-fr 16 pc-fr 20 Ret. JUL 7. - pc-fr 13 pc-en 17 pc-en 21 Ret. 8 JUL. - - pc-en 14 pc-en 18 pc-en 22 Ret 9 JUL. - - - pc-en 19 pc-en 23 Ret. JUL 10. - - - - pc-en 24

Table 2

Similarly, subtasks 2b, 3b and 4b are specific to a return date, therefore, they are commonly performed for pre-calculated travel recommendations for one and the same return date. This is illustrated in Table 3 for the pre-calculated travel recommendations pc-fr 9, pc-fr 13, pc-fr 17 and pc-fr 21, all relating to the return date of July 7.

NCE-BOS Dep. JUL 1. Dep. JUL 2. Dep. JUL 3. Dep. JUL 4. Dep. JUL 5. Ret. JUL 2. pc-fr 0 - - - - Ret. JUL 3. pc-ff 1 pc-ff 5 - - - Ret. JUL 4. pc-en 2 pc-en 6 pc-en 10 - - Ret. JUL 5. pc-en 3 pc-en 7 pc-en 11 pc-fr 15 - Ret. JUL 6. pc-en 4 pc-fr 8 pc-en 12 pc-fr 16 pc-fr 20 Ret. JUL 7. - Common subtasks: 2b, 3b, 4b Ret. 8 JUL. - - pc-en 14 pc-en 18 pc-en 22 Ret. JUL 9. - - - pc-en 19 pc-en 23 Ret. JUL 10. - - - - pc-en 24

Table 3

To summarize, only part of sub-task 4, in particular the recovery of fares which are not valid for the entire departure portion of the trip and for the entire return portion of the trip, but which are specific to the sub- sets or specific travel recommendations, must be carried out separately for each pre-calculated travel recommendation while other subtasks can be carried out jointly for all pre-calculated travel recommendations relating to the same pair of cities of origin destination (true for sub-task 1) or at least for travel recommendations relating to the same departure date (sub-tasks 2a, 3 a and 4a) or to the same return date (sub-tasks 2b, 3b and 4b). Consequently, the more the pre-calculated recommendations are linked to a pair of origin-destination cities and the more the pre-calculated travel recommendations linked to a departure date and a return date respectively, the more the calculation resources can be saved by pooling these subtasks. between the respective precomputed flight requests.

This relationship between the growing synergistic effects and an increasing number of pre-calculated travel recommendations relating to a pair of origin-destination cities and a departure or return date is also visually represented by FIG. 3. FIG. 3 shows six graphs of sets of pre-calculated travel recommendations, each set belonging to a pair of origin-destination cities and forming an example of a part of pre-calculated query results as used herein. In decreasing number of pre-calculated travel recommendations associated with a city pair, Chart 36 relates to the New York-Buffalo city pair, Chart 37 in New YorkMinsk, Chart 38 in New York-Hilo, Hawaii, the chart 39 in New York-Bilbao, graph 40 in New York -Male and finally, graph 41 in New York-Mauritius. The x-axis of the diagram in FIG. 3 denotes the number of pre-calculated travel recommendations while the axis plots a measure of the recalculation resources necessary to perform a recalculation of the pre-calculated travel recommendations, in particular machine time (CPU).

On the one hand, FIG. 3 shows that a recalculation of certain sets of precalculated recommendations requires more computing resources than others. For example, recalculating pre-calculated travel recommendations from set 41 including pre-calculated travel recommendations for flights between New York and Mauritius is generally more costly than recalculating pre-calculated travel recommendations from set 36 including pre-calculated recommendations for flights between New York and Buffalo. This general difference in the amount of computing resources required to recalculate pre-calculated travel recommendations is taken into account by the inclusion of recalculation costs C in the recalculation indicator, as explained in detail above. However, the graphs in FIG. 3 also indicate that the recalculation costs vary with the number of pre-calculated recommendations for the travel of a set being recalculated. In general, the more recalculated travel recommendations for a set are recalculated together, the less computational resources are required to recalculate each pre-calculated travel recommendation. For example, Graph 36 indicates that recalculating pre-calculated travel recommendations for the New York-Buffalo set to 32 requires approximately 1000 ms of CPU, i.e. roughly 31.25 ms of CPU per pre-calculated trip recommendation, while recalculation to 170 of pre-calculated travel recommendations requires about 5000 ms of CPU, i.e. only about 29.4 ms of CPU per pre-calculated trip recommendation. Compared to graph 38 relating to the New York-Hilo city pair, recalculating to 35 the precalculated travel recommendations of this set requires approximately 10 000 ms of CPU, i.e. 286 ms of CPU per pre-calculated travel recommendation whereas the recalculation to 140 of the pre-calculated travel recommendations of this set requires approximately 25,000 ms of CPU, i.e. only 178.5 ms of CPU per pre-calculated trip recommendation. Therefore, it is clear that the pre-calculated travel recommendations belonging to a pair of origin-destination cities (the present example on the one hand) are interdependent in terms of IT resources required for their recalculation as explained above.

In other words, the computing resources necessary to recalculate a precalculated search result i generally depend on whether or not to recalculate other precalculated search results relating to the precalculated search result i during the same calculation cycle. . Thus, the computing resources for recalculating the pre-calculated search results are not static, but vary according to the selection of the set of pre-calculated search results to be recalculated during the calculation cycle.

Therefore, a mechanism is sought to dynamically determine an optimal number of pre-calculated query results on the one hand to be recalculated during a calculation cycle (hereinafter referred to as "number of recalculations") to optimize the efficient use of resources. recalculation. At the same time, the mechanism should minimize the waste of computational resources due to the recalculation of precalculated query results that are still valid and need not be recalculated. Furthermore, the mechanism should not compromise the stability of the recalculation device, but ensure its robustness.

It is proposed to supplement the system 1 of FIG.l with an additional module, designated below as the self-adjusting subsystem of calculation 6 (FIG. 4). The function of the calculation auto-tuning subsystem 6 is to automatically and autonomously determine as well as refine the number of interdependent precalculated query results which must be calculated in order to recalculate more precalculated search results with the same computing resources. To fulfill this function, the self-adjusting calculation subsystem 6 incorporates a database of expert knowledge and is coupled to two other subsystems, in particular

- to the calculation device 3 to which the calculation self-tuning subsystem 6 provides an optimal number (a set) of pre-calculated search results to be recalculated for a given recalculation request coming from the recalculation controller 2. In terms of functionalities, the calculation self-adjustment subsystem 6 can be considered as forming part of the calculation device 3.

- A statistics server 7 which acts as a statistics provider to refine the internal knowledge base of the self-adjusting calculation subsystem 6.

The recalculation controller 2 regularly triggers the recalculation of precalculated interrogation results by sending recalculation requests (potentially based on recalculation schedules 12) to 16 to the calculation device 3. For example, at a point in time, the recalculation controller 2 decides to recalculate one or more particular parts of the pre-calculated interrogation results (also known as said “at least a first part”) and triggers a recalculation of said one or more parts. Each time that the recalculation controller 2 requests 16 to recalculate certain precalculated interrogation results during a new calculation cycle, the calculation device 3 retrieves relevant data from the self-adjusting subsystem 6 to decide on the policy for determining the number of pre-computed query results to be recalculated in the respective parts of the pre-computed query results. A back-up mechanism process 22 provides some quality of decision on the number of pre-computed query results to be recalculated which can lead to potential dynamic realignment of the decision in the current case and over time. An asynchronous process 24 collects, from the statistics server subsystem 7, the statistical data of previous calculations (which have been posted to 20 on the statistics server subsystem 7) to refine the decisions over time. The recalculations of the calculation device 3 itself are based on other calculation data, in the example of FIG. 4 flight or travel schedules, availability and fare data, stored in the database system 8. Note that this is not a limiting example of the distributed database system. The mechanism presented herein is applicable to all cases of the use of precalculated, or cached, query results, in which precalculating or caching involves synergies between the interdependent query results in the exemplary manner defined in the details above compared to FIGS. 2 and 3.

There is no model that could be immediately used to assign an optimal number of recalculations to a given share of pre-computed query results in order to recalculate the pre-computed query results on the part during a calculation cycle. On the one hand, the synergistic effects of recalculating a number of associated precalculated interrogation results depend on the specific use of the database system and on the characteristics of the precalculated interrogation results in terms of their interdependence and validity. decreasing over time. On the other hand, the optimal number of associated pre-computed query results to be recalculated also depends on the recalculation strategy employed (e.g. always recalculate pre-computed query results with the least probability of being up to date) . In addition, options can be included in the triggering of calculation 16 from the recalculation controller 2. which have an influence - a priori unknown - on the precalculated results of interrogation which can be more important to be valid / exact than other.

In order to address these obstacles, it is therefore generally proposed herein to use a learning algorithm which dynamically optimizes the decision on the number of precalculated query results during a calculation cycle over time, independently of said a priori unknown influences mentioned previously. However, it does not seem possible to use standard supervised learning algorithms, i.e. an algorithm based solely on the knowledge of an expert such as a neural network since there is no knowledge of the optimal number of recalculations for a given share of pre-calculated interrogation results. These algorithms would require training on a database associating the calculation triggers 16 from the recalculation controller 2 with their respective best numbers of recalculations for each part to be recalculated. However, the required expert knowledge is generally not available.

Another way to build this database would be to use a brute force approach to permute all the possible numbers of recalculations for a given calculation trigger 16 coming from the recalculation controller 2. This could lead to determining what number of recalculations is most suitable for triggering computation 16. However, this approach presents several problems, since it would require a great deal of time and material to collect all these statistics. Furthermore, it only establishes numbers of static recalculations and there could therefore be inconsistencies between the current status of this database and the time when it is used by the calculation device 3. This would expose system 1 as a whole (FIG.l) to a potential elevation of technical rejections, i.e. failure states caused by calculation triggers 16 coming from the recalculation controller 2, in the event of changes in the pre-calculated interrogation results.

Consequently, it is proposed to train the learning algorithm continuously during the processing of each triggering of calculation 16 coming from the recalculation controller 2. An algorithm shaped by tests and errors seems suitable for noting the result of a decision on the number of recalculations in response to the triggering of calculation 16. The rating could for example use thresholds of a number of key performance indicators to judge the quality of the decision. The reinforcement learning family algorithms seem to be the most suitable for this task.

Consequently, the calculation device 3 is improved by the self-adjustment module 6 to implement a reinforcement learning algorithm in order to determine an optimal number of pre-calculated interrogation results in a part to be recalculated during a calculation cycle. More specifically, the calculation device 3 receives a request to 16 to recalculate precalculated query results for at least a first share. The calculation device 3, more specifically the self-adjusting subsystem 6 determines a number of pre-calculated interrogation results in the first part to be recalculated in the current recalculation cycle on the basis of a learning algorithm by enhancement. The determined number of precomputed results is limited by the computing resources of the computing device which are available during the given time interval. The calculation device 3 then recalculates the determined number of precalculated query results in the first part during the calculation cycle.

In some embodiments, the RL learning algorithm described herein takes into account the key performance indicators (KPIs) and examines the effects of the respective decisions of the number of precalculated query results to be recalculated according to the KPIs during a calculation cycle. Expert knowledge can be used to determine, for example, the calculation trigger options 16 of the recalculation controller 2 which have no influence on the KPIs - this can be used to reduce the dimensionality of the space d 'learning.

In some embodiments, KPIs include one or more of the following factors: the sum of the numbers of the pre-computed query results, the time elapsed to recalculate the pre-computed query results to be recalculated in the given time interval, the maximum resources computation to recalculate pre-calculated interrogation results indicated in the request and the maximum memory required to recalculate pre-calculated interrogation results which are indicated in the request. In some embodiments, the values of the KPIs are aggregated into an aggregated value of KPIs, which is explained in detail below. In some embodiments, the KPI can be determined according to a hierarchy, with more general KPIs composed of a number of more specific KPIs. This allows a fine-grained evaluation of the technical effect, i.e. efficient use of computing resources due to an optimal number of pre-calculated query results selected for recalculation. KPI aggregation is then at each hierarchy level, where the number of more specific KPIs is aggregated to form the more general KPI and more general KPIs are aggregated to establish a common reward value for a certain action as will be explained in more detail below.

Before turning to more detailed design options for an RL algorithm in order to solve the technical problem mentioned above, an overview of some RL concepts is first presented. Reinforcement learning mechanisms are also described, for example in the book “Reinforcement Leaming” by Richard S. Sutton and Andrew G. Barto, published by MIT Press in 1998. RL mechanisms use advanced terms with established meanings. and are used herein with this meaning established to describe the algorithm for determining an optimal number of recalculations, including (see FIG. 5):

- the agent: the module which learns and makes decisions, the calculation device 3 with the self-adjusting subsystem 6 which define the number of recalculations.

- the environment: all the facts external to the agent with which the agent interacts at each of the discrete points of a sequence in time. The environment influences the agent's decisions and is influenced by the agent's decisions.

- the task: a complete specification of an environment, a case of reinforcement learning problem.

- an observation: the determination of a state of the environment at a discrete point in time. In the present report, the report includes various parameters such as the content of a calculation triggering request 16 (an indication of one or more parts to be recalculated and potentially other recalculation options), the report recalculation (has the calculation been successfully completed?), the recalculation KPIs (how well did the calculation finish? More specifically: CPU costs, memory consumption, etc. ).

- action: a decision taken by the agent from a set of actions available in the current state of the environment. Herein: it is a question of determining the number of precalculated results of interrogation of one or more parts to be recalculated during the current recalculation cycle.

- Politics: the mapping of the states of the environment to the probabilities of selecting each possible action.

- the reward function: the function which determines a reward for each action selected by the agent. Herein, the reward function uses KPI thresholds. For example, an action that causes technical rejection of the environment will produce a bad reward. In some embodiments, the reward function assigns a value to an action taken by the self-tuning subsystem 6 based on the aggregated KPIs retrieved from the statistics server 7.

- the function value: a table which associates a set of actions with their estimated rewards.

The agent’s goal is to maximize rewards not immediately, but over the long term.

A general characteristic of RL is the compromise between exploration and exploitation:

- In exploration mode, the agent tries new kinds of action to see how effective they are. The effectiveness of an action is immediately provided by the reward returned to the agent in response to the selected action.

- In exploitation mode, the agent makes use of actions which are known to produce a high reward using the history of rewards derived from the value function. More specifically during each operating phase, the number of recalculations is determined, that is to say, according to the database of expert knowledge currently known to produce the most rewards. The goal is to maximize rewards over the long term (mathematically, this means that the sum of all rewards over an infinite lifetime is maximized). Generally speaking, in exploitation mode, the algorithm tries to take advantage of what it has learned while exploration mode can be seen as an "investment" to seek other opportunities in order to further optimize the determination of the number recalculations.

In exploration and exploitation mode, the agent continually learns from his environment. However, exploration and exploitation must be balanced. Too much concentration on exploration or exploitation will produce poor results. The agent can explore to discover new strategies that could potentially yield a very high reward, but exploring too much can be dangerous because the agent never uses his knowledge of the environment. A strategy for balancing these two modes in order to resolve a particular technical problem discussed herein will be presented below.

Certain features of the concept of the reward function for implementing the computing device 3 with the subsystem 6 will now be described. The main problem to be avoided is that of technical rejections, i.e. the states of failure of the calculation device 3 caused by an inappropriate number of recalculations which produce a rejection message for a triggering of calculation 16. Consequently, a high penalty is attributed to the agent in this case.

On the other hand, the agent should not overreact. Instabilities of the calculation device 3 which would generate technical rejections and which would not be due to the actions taken by the agent are possible. For this purpose, in certain embodiments, the numbers of recalculations which have resulted in technical rejections (RcNbTR) are associated with numbers of recalculations which have not resulted in a technical rejection (RcNbOK). Therefore, in some embodiments, the first term of the reward function is defined by the following formula:

PENALTY ™

RcNbTR \

RcNbOK / with PENALTY (X) = exp (X), i.e. an exponential function.

Furthermore, a set of KPI thresholds is used in certain embodiments. These KPI thresholds are limits that the agent's actions cannot exceed. Consequently, these KPI thresholds must be determined judiciously, e.g. with the help of experts in the field of system 1 (FIG.l), depending on the particular use of system 1.

In order to increase the secure actions of the agent, an additional level of security is implemented in certain embodiments. The threshold is taken as a given percentage of the current threshold from which a technical rejection would currently be generated ("hard threshold"). This produces three possible states: acceptable (below the threshold), warning (above the threshold, but always below the hard threshold) and technical rejection (beyond the hard threshold). In this way, a safety margin is introduced below the hard threshold:

THRESHOLD _KPlx = MARGINhresholdCkpix ') * HARD THRESHOLD _K pi _x with 0 <MARGIN _THRESHOLD ( _{KP! X)} <1 constant

In response to the determination that all KPIs are below their corresponding thresholds after the agent has taken a certain action, the reward value is positive. The closer a KPI is to its threshold, the higher the reward awarded. But if a KPI exceeds its threshold ("warning" state), the system is rewarded with a negative reward. A negative reward is also called a penalty.

Optionally, the KPIs are normalized by their threshold to allow faster comparisons. So in certain embodiments, a second term of the reward function is given by:

where, for each KPI, the reward is positive (multiplied by a positive factor "positive reward" PR) if the threshold is respected, but negative (multiplied by a negative factor "negative reward" NR) if the threshold is exceeded:

AWARD (X)

The MARGIN _SEU parameters i ^ _KPIx ^, with which the variables PR and NR must be adjusted on the basis of an empirical approach to obtain the desired behavior.

In summary, in some embodiments, the overall reward function is defined as:

An example is given by FIG. 6. Two radars are plotted with a vector of three KPIs which are determined after a recalculation of a number of precalculated interrogation results in accordance with the decision of the agent implemented in the auto-tuning subsystem 6. As explained above, the KPIs are normalized by their respective permitted threshold indicated by the inner circle in solid line, with the hard threshold indicated by the dotted outer circle. When each KPI is below the threshold as illustrated in FIG 6A, the vector resulting from the three KPIs (full triangle inside the solid inner circle) is considered acceptable. However if only one KPI exceeds the threshold and the resulting vector enters the corridor between the solid circle and the dotted circle as illustrated in FIG. 6 B, the RL algorithm considers this as a warning and penalizes the individual reward for this KPI (KPI2 in the example of FIG. 6B).

Particulars of the concept of the value function for implementing the calculation device 3 with the self-adjusting subsystem 6 are described next. As mentioned above, the value function is an array which associates a set N of possible actions, in particular the number of precalculated query results on the one hand which must be recalculated during the calculation cycle, with their estimated rewards . In some embodiments, the estimated rewards associated with the function value are specified by a reward function of the reinforcement learning algorithm, such as the exemplary reward function described above. As just described, the reward function can assign an aggregated value of a plurality of key performance indicators to a corresponding selection of precalculated query results of at least said first share.

In certain embodiments, the value function is calculated iteratively from a history of the rewards previously received (indexed by k which represents the k ^th iteration). The value function V of the number of recalculations 1¾ is therefore in certain embodiments

V _{k +} i (WÙ = ^ (^) + “(k ₊₁ ) (WJ * (AWARD ^) - VM) where <x _k (Wj) is the learning speed parameter (step size parameter ) used to process the reward after the k ¹¹¹ selection of the number of recalculations 1¾.

Therefore, two reinforcement learning formulas are used to ensure convergence of the RL algorithm. The first formula: provides that the learning stages are wide enough to possibly surpass all the initial conditions or random fluctuations:

"K (W) = ⁰⁰ k

The second is given by:

a _k (147) <oo k

Since System 1 is not stationary, i.e. as the behavior of the system will vary over time so that the optimal recalculation numbers will change, this second condition cannot be met. Indeed, the system never converges completely because it needs to be continuously adapted to modify what may happen.

In some embodiments, the learning speed parameter oc _fc (1¾) is assigned a higher influence to the last rewards and is adapted to the situation of the system to provide another learning optimization including the state of the environment and mode of the RL algorithm:

- current mode: exploration or exploitation. As decisions must be conservative in order to avoid failures, learning should be done less actively in exploration mode, i.e. that the agent learns, but in a secure manner. Conversely, in operating mode, a change in behavior must be taken into account more quickly.

- code load indicator indicating a recent software update for the calculation device 3: If it is set to true, the system must learn more quickly, because a code load can induce a rapid and real variation of the KPIs which do not indicate any system instability.

The step size parameter is then the function of a system situation S (/ c) and a constant oc:

oc _k (W) = ° c * S (fc) where S (k) =

1, ER Operating Mode <1, CL Exploration Mode> 1, Code Load

ER, CL constant the variables oc, ER and CL are adjusted with an empirical approach (eg using the knowledge of a domain expert) to obtain the desired behavior.

The learning speed parameters must ensure that, in the case of the integration of a negative reward, the value function is not too degraded, as it may be due to a certain instability of the system:

V _{k +} i (Wd <V _k (Wd

After SN successive negative rewards, the RL algorithm must force the system to react quickly and to redetermine a new policy as a starting point:

v _{k +} s _N (Wi) <0

At this point, the accuracy of the value function is questionable. It is likely that the value function for other numbers is incorrect. So the first approach could be to use a "raw approach" to arbitrarily set the new starting point to number of recalculations New = DF number of recalculations old where 0 <DF <1 is a degradation factor.

FIG. 7 shows an example of reward integration by the value function of the weight W4. The reward value for W4 increased between k + 1 and k selections.

In certain embodiments, the request to recalculate precalculated results of interrogation of at least a first part (= triggering of calculation 16 in the example of FIG. 4) indicates precalculated results of interrogation of said at least a first part which must be recalculated. By reusing the nonlimiting example of FIG. 3, described above, the request may include one or more pairs of origin and destination, each pair of origin and destination indicating a part to be recalculated (as in the example of FIG. 3, each pair origin and destination defines a share). In addition, the request may indicate a departure date, multiple departure dates or a range of departure dates, as well as a length of stay, multiple lengths of stay or a range of lengths of stay. To give a particular example, the recalculation controller 2 can ask the calculation device 3 to recalculate the lowest prices of all the travel recommendations with Nice as origin and Boston as destination (identifying the Nice-Boston share) with a fork. 30 day departure date starting November 1, 2017 and a range of length of stay between 7 and 14 days (= a range of eight days), requiring a total recalculation of 30-8 = 240 of the pre-calculated results d 'interrogation forming a portion of the Nice-Boston part to be recalculated.

In these embodiments, the calculation device 3 is optionally arranged to determine the number of pre-calculated interrogation results in the first part to be recalculated on the basis of the pre-calculated interrogation results indicated in said at least one first part. For example, the computing resources of the computing device 3 can make it possible to recalculate 100 precalculated query results during the next three computing cycles. In this situation, the calculation platform is arranged to select an optimal number of precalculated query results to be recalculated during the next calculation cycle by dividing the portion of the first part defined by the number indicated in query 16 into three slices, where the division results in slices which use the available computing resources in an optimal way. Each of the three tranches is recalculated during the next three (or three of the next) calculation cycles. Another more specific example of such a division is given below.

More particulars of the concept of the reinforcement learning algorithm for implementing the computing device 3 with the self-adjusting subsystem 6 will now be described with reference to FIG. 8. In some embodiments, the algorithm for determining the number of recalculations is composed of three main activities:

- In response to the reception of a calculation trigger 16 by the calculation device 3 (FIG.8: "Start of activity 1"), the agent located at the self-adjusting subsystem 6 (after having been notified via the interface 18) retrieves on 24 from the statistics server 7 the number of recalculations of each part of the pre-calculated results of the interrogation concerned which were determined at the last recalculation of the respective part.

- The agent then determines the RL mode (FIG. 8: "use and save"), ie. he chooses between exploration mode and exploitation mode. The determination of the mode is based on the value function for the number (s) recovered from the last recalculation as will be described in more detail below.

- The agent also saves 20 of the calculation reports on the statistics server 7 (FIG. 8: "use and save"). Consequently, in a certain embodiment, the calculation device 3 is arranged to store the determined number of precalculated interrogation results in the first part on the statistics server 7 for a subsequent determination of the number of precalculated interrogation results. to be recalculated in a subsequent calculation cycle.

- In addition to these activities (FIG. 8: “use and save”), the agent can use a safeguard mechanism (FIG. 8: “safeguard mechanism”) in order to exclude situations in which agents' determinations could cause a state of failure of the calculation device 3 or of the system 1 as a whole which would jeopardize all the calculations). For example, if such a technical problem with the self-tuning subsystem 6 is detected, the backup mechanism can blacklist the self-tuning subsystem 6 for one or more specific parts to be recalculated and use a default policy (such as a given recalculation or a predetermined number of precalculated results of querying said one or more shares) known to work correctly.

- The agent also runs an asynchronous process (FIG. 8 "start of activity 2") to collect historical data from the statistics server 7, analyze this data and refine the database for decisions concerning recalculation numbers.

More details on the RL determination mode will now be described. As explained in the introduction to reinforcement learning above, a balanced compromise is sought between these two modes.

Two known balancing methods are the Epsilon-Greedy strategy and the Softmax strategy. The Epsilon-Greedy strategy explores new strategies for a given percentage of time and exploits the action with the highest reward the rest of the time (greedy action) For example, the system could explore 1% of the time (by choosing an action random among all the possible actions) and use the rest (99%) of the time (use of the action with the highest score). The Softmax strategy aims to resolve a disadvantage of Epsilon, including exploration by making an equal choice between actions. This means that it is as likely to choose the worst action apparently as it is to choose an action close to the best. For tasks where the worst actions are harmful, this can be disadvantageous. The Softmax strategy therefore refines the Epsilon-Greedy strategy by varying the probabilities of action as a graduated function of the estimated value.

The balancing strategy used in some embodiments herein is different from these two known strategies. At a general level, the computing device 3 is arranged to retrieve an indication of the precalculated results of interrogation of said at least one first part which has been recalculated in a previous time interval, i.e. the last number of recalculations as explained above. The calculation device 3 is also determined to apply a mode of exploitation of the reinforcement learning algorithm or to apply a mode of exploration of the reinforcement learning algorithm. In certain embodiments, the computing device 3 determines to apply either the operating mode of the reinforcement learning algorithm or to apply the exploration mode of the reinforcement learning algorithm by determining the reward given by the value function of the precalculated query results indicated in the indication retrieved and by determining a number of recalculation cycles to apply the operating mode of the reinforcement learning algorithm on the basis a comparison of the determined reward with a reward threshold. In some embodiments, the reward threshold is defined by given percentages of threshold values respectively for each of the plurality of key performance indicators for which recalculation for a given time interval would result in an error state of the computing device. As described below, an additional safety net is introduced, avoiding the technical failure of the computing device 3 caused by the RL algorithm. The calculation device 3 applies the operating mode of the reinforcement learning algorithm during the current recalculation cycle and the subsequent recalculation cycles as provided by the determined number of recalculation cycles. A more detailed example will be described below.

In response to the determination to apply the operating mode, the calculation device 3 of certain embodiments is arranged to determine the number of precalculated query results in the first part to be recalculated during the current calculation cycle based on a function value of the reinforcement learning algorithm for the number of precalculated interrogation results indicated by the indication retrieved, the value function associating possible selections of precalculated interrogation results in said at least a first part for a recalculation with respective estimated rewards.

In certain embodiments, the calculation device iteratively adapts the number of precalculated interrogation results indicated by the indication retrieved based at least on the number of precalculated interrogation results indicated by the retrieved indication, the function of value of the reinforcement learning algorithm for the number of precalculated query results indicated by the indication retrieved and the reward threshold. A more specific example, illustrating how the computing device 3 can determine a new number of recalculations during the exploration mode in order to further optimize its actions, is given below.

The balancing mechanism for switching from operating mode to exploration mode and vice versa is justified by the fact that it is possible to "classify" the numbers of recalculations. In other words, it uses the fact that up to a certain point, the higher the number of recalculations, the greater the reward (and beyond this point, there is a failure). This helps to build an exploration policy: assuming an acceptable number of recalculations (KPIs are below the threshold), it can be decided to explore with a higher number of recalculations (since it is known that a weight lower will bring no improvement in reward).

It is also possible to decide how far the system can explore using the difference between the value function of the current number of recalculations and the maximum expected reward. For this purpose, we can estimate how far the number of recalculations is from the KPI threshold and use this estimate to decide to explore with a large or small step. This makes it possible to reach a compromise between the speed of convergence of this optimal number of recalculations to apply it to a given share and the risk of collecting a low reward (in particular the avoidance of technical rejections).

More specifically, the following case-based RL mode determination algorithm is used in some embodiments to find the optimal number of recalculations while keeping the system 1 stable.

1. 0 <a value function for the last number of recalculations <50% of the maximum aggregate reward resulting from all the KPIs pointing to their respective threshold value (Max Reward): Exploit Cl times and then explore a given number of times (e.g. once) with a higher number of recalculations (the details of choosing this higher number of recalculations will be explained below).

2. 50% <a value function for the last number of recalculations <75% of the max response. : Use C2> Cl once and then explore once with another higher number of recalculations

3. 5% <a value function for the last number of recalculations <90% of the max reward. : Use C3> C2 times and then explore once with a higher number of recalculations.

4. 90% <a value function for the last number of recalculations <100% of the max reward. : Use C4> C3 once and then explore once with a higher number of recalculations.

The variables Cl, C2, C3 and C4 must be adjusted with an empirical approach to obtain the desired behavior (for example, in terms of speed of convergence).

In response to the determination that the agent is allowed to explore in the current compute cycle, the number of recalculations larger for exploration must be decided. It is assumed that:

- The reward calculated for a given number of recalculations is better when the KPIs, after the decision for the given number of recalculations, are closer to the threshold of KPIs than for the last number of recalculations.

- The evolution of the reward is not proportional to the number of recalculations (subadditivity), p. ex. :

AWARD * W) <2 * AWARD

Because in some embodiments, the system has a conservative approach to avoid failure states, convergence to the best recalculation for a given share is done carefully, step by step, in some embodiments. In some embodiments, the new number of _new recaïcuïsW is given by:

REWARD _max

W = ---------- W ^ new V _k (Wt) ¹ where Vfc (ÏVi) is the value function of the last number of recalculations used W _k . This is a strictly positive value since the current RL mode is an exploration mode which has been enabled by comparison with the maximum reward. This approach is applied iteratively to determine the number of recalculations for the next exploration phase. On multiple numbers of exploration phases, the respective use of this formula in each of the exploration phases to determine W _new , step by step forms an iterative approach which brings the number of recalculations closer and closer to the reward maximum aggregate RECOMPENSEmax and thus, to select the number of recalculations in an optimized way in order to use efficiently the available recalculation resources.

FIG. 9 gives an example of this balancing selection between the operating mode and the exploration mode. This example details the evolution of the value function of a given part of precalculated query results.

Initially (FIG. 9A), when a share has to be recalculated for the first time, there are no previous recalculations for the share. In this case, the exploration mode is selected by default. In some embodiments, the auto-tuning subsystem 6 can issue a warning at 14 to the recalculation controller 2. The recalculation controller 2 can then include a number of default recalculations in future requests relating to this part at 16. The agency selects a given conservative (ie relatively low) number of recalculations for the share as shown visually in FIG. 9 A. A reward value for this action is calculated based on the reward function explained in detail above. The calculation device 3 then recalculates the precalculated results of interrogation of the part in the calculation cycle in accordance with the determined number of recalculations. This determined number of recalculations is stored on the statistics server 7.

In response to the reception by the calculation device 3 of another triggering of calculation 16 to recalculate the precalculated results of interrogation on the part in a following calculation cycle (always FIG. 9A), the last number of recalculations is recovered from the statistics server 7. The agent then determines that the last number of recalculations was below the maximum reward, for example in a range of 50-75% of the maximum reward according to case 2 above. In accordance with the above balancing algorithm, the agent determines the RL mode of operation for the C2 intervals, in the example of FIG. 9B for three additional calculation cycles. This ensures that the previous number of recalculations is a stable solution (but not yet necessarily an optimal choice). In this case, C2 similar recalculations on the part with the same precalculated results of interrogation will be carried out before proceeding to a new exploration.

At the recalculation C2 + 1 ^st on the part (FIG. 9B), a new exploration is determined with a higher number of recalculations, as explained previously. This results in a higher number of recalculations with a higher reward value. During the following recalculations, the RL operating mode is again selected (always FIG. 9B). This time, the reward value for the higher number of recalculations is in case 3, so the next C3 recalculations on the part are done in operating mode. As in FIG. 9 A, this ensures that the number of recalculations explored is stable.

This alternative selection of an exploration followed by a number of exploitations is reiterated later in the same way to determine the optimal number of recalculations for this particular part (FIG. 9C). The mode selection algorithm will enter in case 4, where the number of subsequent operations C4 is the highest. As a result, new numbers of potentially more optimal recalculations are now explored less often than in previous iterations.

At a point in time, a new number of recalculations explored may enter the warning area, i.e. that the KPI threshold for one or more KPIs is exceeded, producing a negative reward value (FIG. 9D). As explained before, in some embodiments, this is a warning, because the threshold has been calculated at a given percentage below the hard threshold. This means that the number of recalculations has reached (in fact even exceeded) the phase in which one can expect little improvement in the reward. Consequently, the number of recalculations is stabilized at the last value which produced the best rewards. However, this stabilization only means that exploration is done less often, but never excluded. Indeed, it is desirable to continue the exploration from time to time, because a system as a whole is not stationary and can change over time, p. e.g., when the precalculated query results, stored by the search platform 4, change. Another exploration phase allows System 1 to discover and update with these changes and developments.

FIG. 9E represents a special situation which could indicate a degradation of the system 1 in which a reinitialization could be appropriate. Possible reasons for performance degradation are a substantial change in the pre-calculated search results, a change in the database plan or the model of the research platform 4, the computing device 3 and / or the resulting recalculation controller. in a modified / refined functionality or a modified reserve of the precalculated results of interrogation. The term degradation indicates that the optimal number of recalculations can be reduced to these changes compared to the state of system 1 before the changes.

In the scenario of FIG. 9E, multiple rewards returned later for recalculated numbers explored later are negative although System 1 has been stable for some previous length of time. After several rewards integrations, the weight value function is still negative. The fall in rewards in negative values then implies that there are no other apparent reasons than a possible deterioration. Therefore, this indicates that the value function is probably inaccurate or at least is no longer suitable for incorporating changes into the system. A new number of recalculations is then determined by analyzing the value function for the current number of recalculations to determine the new starting point. For example, the self-adjusting subsystem 6 can start from a default initial value of the number of recalculations as described above in the case of a part recalculated for the first time. In another example, the self-adjusting subsystem 6 can perform a "dichotomous approach", ie. use the very last number of recalculations that produced a positive reward and, to reduce it even more, p. ex. by dividing it p. ex. by two.

Details of the readjustment of the number of recalculations will now be described.

In certain embodiments, the computing device 3 does not have enough computing resources to recalculate an entire part of a part of the part which has been the subject of a request to be recalculated by the recalculation controller. 2 in a single recalculation cycle. In these embodiments, the mechanisms described herein can be used to divide the portion, or the portion of the portion that is the subject of the request, into optimal slices which are recalculated by the computing device 3 during 'a plurality of calculation cycles. Here the number of recalculations determined by the mechanisms described herein indicates the size of a slice to be recalculated in each cycle.

Furthermore, in certain embodiments, the precalculated query results are arranged in cells with two dimensions. A nonlimiting example has already been mentioned above with reference to FIG. 3 where the fare-based travel recommendations are precalculated and stored in a respective share for each pair of origin and destination. In this nonlimiting example, the two dimensions are the starting data and the length of stay. In these embodiments, a number of recalculations indicates a portion of a part having the shape of a square (i.e., it indicates a tuple of two numbers, one for each dimension.

The determination of this number of recalculations according to the mechanisms described above would respect d = ceilingÇ ^ / w) s = plancBer (—J in which d is the range of departure days [a departure in the next d days, the first dimension ] and s is the length of stay range [a length of stay from 1 to s days, the second dimension] and W is an optimal weight recommended by the expert knowledge database [ie in the current state of the RL algorithm, it is supposed to produce the highest reward.] However, depending on the range of departure dates and the range of length of stay requested in triggering calculation 16, this solution can bring to small slices of recalculations at the end of the [part of the] part to be recalculated.

The following algorithm provides an improved division of the part of the part to be recalculated into slices by the calculation device 3, each of the slices must be recalculated during a plurality of calculation cycles, in particular to choose a number of recalculations having the form of a parallelogram as follows:

- Determine a range of length of stay s which is close to Vïÿ and divide the range of length of stay indicated in request 16 from the recalculation controller 2 until the pieces are as equal as possible.

- Determine the highest possible number of recalculations being a square of s · d with the least not full slice [ie the last part of the part to be recalculated is the closest to the number of recalculations (i.e. the size of the other parts) and with s> d in order to optimize the efficient use of calculation (if s> d, fewer start dates must be recalculated during each calculation cycle, eg recalculating a 3d15s square is less expensive than recalculating a 15d3s square ).

The determined numbers s and d are fixed at floor 0 =) and therefore have:

s = ceiling (---- _e td = planner (-).

floor ^) s

Implicitly, this algorithm considers that the range of length of stay indicated in request 16 is smaller than the range of departure dates indicated in request and that the range of length of stay indicated is therefore the one that must be divided. If on the other hand, the length of stay indicated in request 16 is equal to or higher than the range of duration of stay, the same algorithm must be applied with a specified duration of stay exchanged and a specified day of departure exchanged, and therefore s and d exchanged.

As an example, the recalculation controller 2 requests the calculation device 3 to recalculate a specific portion for the next 31 days of departure with a range of length of stay of 16 [ie. all lengths of stay being from 1 to 16 days]. Therefore, overall, a square of 31 · 16 = 496 of precalculated query results must be recalculated. The calculation platform has enough calculation resources to recalculate 50 precalculated query results per recalculation cycle.

The naive solution mentioned above would produce a number of recalculations of 7d -7s = 49 which would result in 8 slices of 7d-7s, as well as 7 non-full slices, in particular 4 slices of 2d-7s, 2 slices of 7d-4s, and a final slice of 2d-4s, in order to complete the recalculation of the 496 precalculated query results. The average number of recalculations beyond these 15 calculation cycles is only 33. The more sophisticated approach drawn above leads to a division of 6d-8s. This results in 12 slices, in particular 11 slices of 6d-8s and one remaining slice not full of ld · 7s, and an overall average number of recalculations of 4L

This approach can be generalized to other use cases with precalculated query results having other dimensions or a different number of dimensions [p. ex. three dimensions].

Finally, FIG. 10 is an example of a schematic representation of a computer system 100 arranged as a computing device to execute a set of instructions, in order to cause the computer system 100 to implement any of the methodologies implementing the determination of an optimal number of recalculations for a part in order to recalculate an optimal number of precalculated results of interrogation during a cycle of computation, as described above. The computer system 100 includes a processor 102, a main memory 104 and a network interface 108 which communicate with each other via a bus 101. The computer system 100 also includes a static memory 106, for example. ex. non-removable flash and / or a hard disk and / or a removable micro or mini SD card which permanently store the instructions allowing the computer system 100 to implement its

Patent Application No. 18 50581 A5013307FRP00Mm Functions, in particular to execute the reinforcement learning algorithm to select the number of recalculations in response to the reception of a calculation trigger 16. In addition, the computer system 100 can including a user interface 110 supported by a corresponding user interface control module.

In some examples, the instructions programmed to implement the routines to determine the number of recalculations are also stored on the static memory 106. When the instructions are executed, the process data resides in the main memory 104 and / or in the processor 102. The instructions can also be transmitted or received as a signal propagated by the network interface device 108 from / to a software server in a local network or on the Internet. The network interface device 108 may be a wireless network interface [such as a WIFI and / or Bluetooth interface] and / or a 2G / 3G / 4G / 5G mobile network interface.

The mechanisms described herein for determining an optimal number of recalculations increase the efficiency of the recalculation of precalculated query results which are composed of a plurality of parts with a respective number of precalculated results, in which the computing resources of the calculation device, necessary to recalculate a precalculated result of interrogation result in a first part which depends, or not on the fact of recalculating other precalculated results of the first part during a given time interval forming a current recalculation cycle. The use, as described above, of the reinforcement learning algorithm optimizes the use of the computing resources of the computing device 3 by allowing the recalculation of more precalculated query results with the same quantity computing resources only if the algorithm was not used. The mechanisms are independent of the particular use case of the calculation device and can be used by any system which recalculates precalculated query results having an interdependence in terms of the computing resources required to recalculate them, as described by way of example. above with reference to the example of FIG. 3.

Although certain products and processes constructed according to the teachings of the invention have been described herein, the scope of coverage of this patent is not limited to them. On the contrary, this patent covers all the embodiments of the teachings of the invention, either literally or under the doctrine of equivalents.

Claims (11)

1. A computing device for recalculating pre-computed query results stored on a database server, wherein the pre-computed query results are composed of a plurality of parts, wherein each part includes a number of results precalculated interrogation, in which the computing resources of the calculating device which are necessary to recalculate a precalculated interrogation result on the one hand depend on whether or not to recalculate other precalculated interrogation results during a given time interval forming a current recalculation cycle, the calculation device being arranged for: - receive a request for recalculation of the precalculated query results for at least a first part; - determine a number of precalculated query results in the first part to be recalculated in the current recalculation cycle on the basis of a reinforcement learning algorithm, the determined number of precalculated search results being limited by the resources of calculation available from the calculation device during the given time interval; and - recalculate the determined number of precalculated query results in the first part during the recalculation cycle. 2. The calculation device according to claim 1, in which the request for recalculation of precalculated interrogation results for at least one first part indicates precalculated interrogation results for said at least one first part which is to be recalculated and the device calculation is arranged to determine the number of pre-calculated interrogation results in the first part to be recalculated on the basis of the pre-calculated interrogation results indicated for said at least one first part. 3. The calculation device according to claim 1 or claim 2 being arranged to determine the number of pre-calculated query results in the first part to be recalculated by being arranged for - retrieve an indication of the precalculated interrogation results for said at least one first part which has been recalculated during a previous time interval; - determine the application of a mode of exploitation of the reinforcement learning algorithm or the application of a mode of exploration of the reinforcement learning algorithm; and - in response to the determination to apply the operating mode, determining the number of precalculated query results in the first part to be recalculated during the current calculation cycle on the basis of a function value of the reinforcement learning algorithm for the number of precalculated interrogation results indicated by the retrieved indication, the value function associating possible selections of precalculated interrogation results in said at least one first part for a recalculation with respective rewards estimated. 4. The computing device according to claim 3, wherein the estimated rewards associated with the value function are specified by a rewards function of the reinforcement learning algorithm which assigns an aggregated value of a plurality of indicators of key performance to a corresponding selection of precomputed results of interrogation of said at least a first part. 5. The calculation device according to claim 4, in which the aggregated value of key performance indicators is based on at least one of the following key performance indicators: the sum of the numbers of precalculated query results to be recalculated, the elapsed time to recalculate the precalculated query results to be recalculated during the given time interval, the maximum computing resources to recalculate the precalculated query results indicated in the request and the maximum memory required to recalculate the precalculated results of interrogation indicated in the request. 6. The computing device according to any one of claims 3 to 5 being arranged to determine the application of the operating mode of the reinforcement learning algorithm or the application of the exploration mode of the algorithm reinforcement learning by being arranged for - determine the reward given by its value function for the precalculated query results indicated by the indication retrieved; - determine a number of recalculation cycles to apply the operating mode of the reinforcement learning algorithm based on a comparison of the determined reward with a reward threshold; - apply the operating mode of the reinforcement learning algorithm during the current recalculation cycle and the subsequent recalculation cycles given by the determined number of recalculation cycles. 7. The calculation device according to claim 6, in which the reward threshold is defined by given percentages of threshold values respectively for each of the plurality of key performance indicators for which recalculation during the given time interval would result. in a state of error of the calculation device. 8. The calculation device according to claim 6 or 7 being arranged for - in response to the determination to apply the exploration mode, iteratively adapt the number of precalculated interrogation results indicated by the indication retrieved on the basis at least of the number of precalculated interrogation results indicated by the indication retrieved, the value function of the reinforcement learning algorithm for the number of precalculated query results indicated by the indication retrieved and the reward threshold. 9. The calculation device according to any one of claims 1 to 8 being arranged to store the determined number of precalculated query results in the first part on a statistics server for a subsequent determination of the number of precomputed query results to be recalculated in a subsequent calculation cycle. 10. A method for the recalculation of pre-calculated query results stored on a database server, wherein the pre-calculated query results are composed of a plurality of parts, wherein each part includes a number of pre-calculated results of interrogation, in which the computing resources of a computing device, necessary to recalculate a precalculated result of interrogation on the one hand, depend on the fact of recalculating, or not, other precalculated results of interrogation of the first part during a given time interval forming a current recalculation cycle, the method comprising, at the level of the calculation device: - the receipt of a request for recalculation of precalculated query results for at least a first part; - the determination of a number of precalculated query results in the first part to be recalculated during the current recalculation cycle on the basis 5 of a reinforcement learning algorithm, the determined number of precomputed search results being limited to the available computing resources of the computing device during the given time interval; and - the recalculation of the determined number of precalculated query results in the first part during the recalculation cycle. 11. A computer program product comprising program code instructions stored on a computer-readable medium for implementing the steps of the method according to claim 10 when said program is running on a computer.