AU2016202814A1 - Systems and methods for managing cpu usage during qualitatively assessment of task data - Google Patents

Abstract

Disclosed are systems and computer implemented methods of a task release gate in a task scheduling system, the task release gate for determining which tasks to move to a task buffer, the computer including a CPU, processing task data through a filter of an acceptability band providing a qualitative assessment having a particular qualitative threshold, monitoring CPU usage or elapsed time to determine whether the processing step is utilising CPU usage above a predetermined CPU threshold, when CPU usage or elapsed time exceeds the predetermined CPU threshold, changing the particular qualitative threshold to another qualitative threshold and repeating the previous steps until there is a distribution mix of tasks to the resources to distribute the tasks to the resources within a preset CPU or elapsed time limit and delivering to a task buffer a mix of tasks to resources. Time estimates can be established by an x-factor process. 301 receiving FIG 3 task data 33 qualitatively 305a assessing monitoring task data .CPU usage according to or elaspse time nst criteria 311a 307a does CPU usage no0 deliver mix or elapsed time of tasks exceed threshold? to resources 309a yes 309a0 change 305b qualitative .oiorn assessment toCPusg according to orU elap et m 2nd criteriaoreap tm 311b 307b does CPUJ usage no deliver mix or elapsed time of tasks exc ed to reso u rce s 309b yes change------------ qualitative assessment to according to 3nd criteria

Description

WI00096014 2016202814 02 May 2016

SYSTEMS AND METHODS FOR MANAGING CPU USAGE DURiNG QUALITATIVELY ASSESSMENT OF TASK DATA

Field

Disclosed are systems and methods for managing CPU usage during qualitatively assessment of task data.

Background in task scheduling systems, tasks or jobs are delivered to resources based upon various criteria, including time estimates and resource capabilities. In one example, task scheduling systems for software developers deliver tasks to a task buffer from where the software developer resources can take up the tasks. The buffer may be displayed on a computer display screen, in another example, tasks can be carried out in a warehouse distribution centre where the task management system directs Stocker and/or picker resources. In another example, tasks can be carried out in manufacturing where assembly is directed by a task scheduling system.

Generally, there are different levels of assessments and processing in a task scheduling system. A system can include a plurality of sub-systems including one or more task management tools and/or automated task managers. By the task scheduling system, there may be an establishment of a task including the attributes and/or qualifications of a resource required for completing the task, a time estimate evaluation of the task completed by a resource or by some other means, an assessment to determine a distribution mix of tasks to resources, output of tasks to a task buffer, display of tasks on a task buffer board, managing progress and completion of tasks, delivery of output of the task to the appropriate resource, and so on. Delivering tasks to a task buffer can 1 WI00096014 2016202814 02 May 2016 provide a mix of tasks to one or more particular resources, which can be a part of a larger mix of tasks to a resource group. It is desirable to provide a mix of tasks to a resource of an achievable quality and with as little resource waste as possible.

Summary

In many task scheduling systems, resource capabilities are typically derived through quantitative measures. However, a task management system that includes substantial quantitative criteria can render the mix of tasks to resources less than optimal. Therefore, there is a need for a qualitative assessment which can provide a more optimal mix of tasks to a resource or to plurality of resources. However, qualitative analysis can utilise excessive CPU and/or extend time for processing beyond predetermined preferable limits. Therefore, there is a need for a task management system that provides qualitative analysis while avoiding utilisation of the CPU beyond predetermined limits and/or extending the time for processing beyond predetermined limits. As discussed below, a qualitative assessment may be provided by one or more acceptability bands.

Given entity requirements which oftentimes are complex, an approach of task management that includes a qualitative analysis of resource capabilities can provide a more optimal mix outcome. In an effort to provide a mix of a particular quality, the disclosed computer implemented methods and systems utilise a qualitative assessment of a task according to a first qualitative criteria in generating a distribution mix. To avoid utilising the CPU beyond predetermined limits and/or extending the time for processing beyond predetermined limits, when CPU usage or elapsed time exceeds the predetermined threshold, the disclosed computer implemented methods and systems utilise a qualitative assessment of a task according to a second qualitative criteria in generating a distribution mix. in this way, the disclosed methods and systems can provide for delivering to a task buffer a mix of tasks to the resources based upon the first qualitative criteria or the second qualitative criteria. 2 WI00096014 2016202814 02 May 2016

Determining a distribution mix of tasks to resources can further include determining capabilities of available resources and matching task types to capabilities types.

Disclosed are systems and computer implemented methods of a task release gate in a task scheduling system, the task release gate for determining which tasks to move to a task buffer, the computer including a CPU, processing task data through a filter of an acceptability band providing a qualitative assessment having a particular qualitative threshold, monitoring CPU usage or elapsed time to determine whether the processing step is utilising CPU usage above a predetermined CPU threshold, when CPU usage or elapsed time exceeds the predetermined CPU threshold, changing the particular qualitative threshold to another qualitative threshold and repeating the previous steps until there is a distribution mix of tasks to the resources to distribute the tasks to the resources within a preset CPU or elapsed time limit and delivering to a task buffer a mix of tasks to resources. Time estimates can be established by an x-factor process.

Disclosed are systems and methods of a task release gate in a task scheduling system, the task release gate for determining which tasks to move to a task buffer and for moving the tasks to a task buffer, the computer including a CPU, including the steps of receiving task data, qualitatively assessing the received task data according to first qualitative criteria, the task data qualitative assessment to determine a distribution mix of tasks to resources, monitoring CPU usage or elapsed time to determine whether the qualitative assessment is utilising CPU usage above a predetermined threshold, when CPU usage or elapsed time exceeds the predetermined threshold, changing the qualitative assessment from the first qualitative criteria to a second qualitative criteria, ceasing qualitatively assessing according to the first qualitative criteria to then qualitatively assessing the received task data according to the second qualitative criteria to generate a distribution mix of tasks to the resources to distribute the tasks to the resources within a preset CPU or elapsed time limit and delivering to a task buffer a mix of tasks to the resources. 3 WI00096014 2016202814 02 May 2016

Also disclosed are systems and methods wherein the qualitative assessment includes subjecting the task data to acceptability bands including a range of values, wherein the first qualitative criteria is a first range of values and the second qualitative criteria is a second range of values. Also disclosed are systems and method including a third qualitative criteria of a third range of values. Moreover disclosed are systems and methods wherein upon ceasing qualitatively assessing according to the first qualitative criteria of a first range of values, the second qualitative criteria is a second range of values. Furthermore, disclosed is wherein the task data includes time estimates which are is established by an x-factor process. Also disclosed are methods and systems wherein the task data includes time estimates which are is established by an x-factor process.

Brief Description of the Drawings FIG. 1 shows a production environment a plurality of work stations including displays; FIG. 2 illustrates the display managing server 111 in more detail; FIG. 3 is a flow chart of an embodiment of the methods and systems of a task scheduling system; FIG. 4 depicts, in a software development production environment, what an acceptability band may look like when displayed on a display of a work station or board as depicted in FIG. 1; FIG. 5 depicts a user interface graphic to set when the default value will be overridden; FIG. 6 depicts a user interface graphic to set where Pass 1 Acceptability Bands (ABs) are bypassed and work is released according to a fall back of according to a release sequence or Pass 2 (see above) depending upon the circumstances;

Figure 7 is a block diagram of an example of a workflow acceptability assessment system;

Figure 8 is a flowchart of an example of a workflow acceptability method;

Figure 9 is a worked example of filtering a set of workflow items based on subset criteria;

Figure 10a to lOd provide examples of different acceptability bands; 4 WI00096014 2016202814 02 May 2016

Figure 11 is a block diagram of an example of a workflow acceptability system integrated into a workflow management system. FIGs. 12a-12d depict visually displayed acceptability band values and output, including mouse overs that show values associated with the output; FIGs. 13a and 13b show screen shots where a component is selected and bands are applied; FIGs. 14a and 14b show some definitions provided for establishing acceptability bands in the application of the screen shots of FIGs.13a and 13b;

Figure 15 is a block diagram of an example of an estimation system for use in a project management or work tracking;

Figure 16 is a block diagram of an example of a project management and work tracking system including an estimation system;

Figure 17 is a flow chart of an example of a process for providing a consolidated estimate;

Figure 18 is an example of a skewed probability distribution;

Figure 19 depicts empirical data in carrying out the described systems and methods; FIG. 20 depicts a screen shot of acceptability bands that are utilised in a workflow management system product; FIG. 21 depicts workflow items subjected to one or more acceptability bands; and FIG. 22 depicts a screen shot of a workflow buffer.

Detailed Description FIG. 1 illustrates a production environment 100, such as an open plan office where, for example, a new software application is coded. The production environment 100 is any production environment. For example, it may also be related to manufacturing goods, such as cars, or a call centre processing customer calls, a warehouse receiving and distributing goods, or any other production environment. It is understood that the disclosed task scheduling systems and methods is applicable to any production environment. 5 WI00096014 2016202814 02 May 2016 FIG. 1 shows a production environment 100 plurality of work stations, 101,102,103, 104 and 105 including displays. The tasks, upon mix assessment as described in detail below (as well as any other suitable assessments), are delivered to a task buffer which may reside in storage unit. The tasks stored in the task buffer can in turn be displayed on a board. The tasks can be displayed, for example, on a large display 107 and/or the work station displays. The tasks can be displayed in an arrangement that indicates how long the tasks have been on the board, that is, during a defined time period, in this example, the tasks are in columns and rows. As an example, column 109 receives a task icon from the bottom of the column, in the lower most row. As time passes, it can progress upwardly if the task has not been completed and/or otherwise eliminated from the board until it reaches a row representing the limit of a defined time period.

As an example five workstations 101-105 perform multiple tasks. In some examples completed tasks contribute to an entire product, such as a coding a function or class of a software product, the translation of a particular user interface into a different human language, testing of a software module and other tasks. In these examples, the work stations 101-105 are computer clients, such as Personal Computers. A task management tool of a task scheduling system can estimate how long it will take or is estimated to take to complete a task from when the task is commenced. Each workstation may be associated with one operator, such as a programmer with particular attributes. The task manager can associate a task with that programmer, a group of resources or any other combination of entities involved with the task. The task manager may include a time estimate tool utilising an x-factor to determine time estimates. The x-factor is discussed in detail below.

As mentioned above, a production environment 100 can include a tool that further includes an electronic display 107 for displaying a board of the multiple tasks. In one example, the display 107 is a TV screen. The display 107 is communicatively coupled to the multiple workstations 101-105. For 6 WI00096014 2016202814 02 May 2016 example, an HDMI cable or Wi-Fi connects the display 107 to a display managing server 111, which can be, in turn, connected to the same network that the workstations 101-105 are connected to.

This way, the display 107 can be communicatively coupled to the workstations 101-105. In one example, the task manager is a further server on the same network and sends task assignment data to the display managing server 111. The task assignment data is indicative of the assignment of each task to one of the workstations 101-105 or operators.

While some examples relate to the task board being displayed on display 109 it is to be noted that task board may also be displayed on the workstations 101-105 or on any other computer system or printed. Task board may be accessible as a HTML webpage or other graphics page by using a web browser. It is beneficial that the task board is a large board that can be seen from a distance, as the visualisation feature that is described herein provides to the viewer notice of a potential risk of a resource falling behind in task completion substantially without need for detailed analysis.

Fig. 2 illustrates the display managing server 111 in more detail. Display managing server 111 can include a processor 202 connected to a program memory 204, a data memory 206, a communication port 208 and a user port 210. The program memory 204 is a non-transitory computer readable medium, such as a hard drive, a solid state disk or CD-ROM. Software, that is, an executable program stored on program memory 204 causes the processor 202 to perform the method in FIG. 3.

The processor 202 may store the tasks of the task board 109 in data store 206, such as on RAM or a processor register in the form of a bitmap image, pixel values in a framebuffer or a graphics card buffer. Creating, generating, modifying the board including related actions, may relate to the processor 202 generating a coded representation, such as an HTML table or an SVG file, that is accessible by display 109 through an internet browser in kiosk mode, for example. Processor 202 7 WI00096014 2016202814 02 May 2016 may also send the task data via communication port 208 to a server in accordance with the task manager.

The processor 202 may receive data, such as task data, from data memory 206 as well as from the communications port 208 and the user port 210, which is connected to display 109 that shows a visual representation 109 of the board to an observer 216. In one example, the processor 202 receives task data from the task manager via communications port 208, such as by using a Wi-Fi network according to IEEE 802.11. The Wi-Fi network may be a decentralised ad-hoc network, such that no dedicated management infrastructure, such as a router, is required or a centralised network with a router or access point managing the network.

In one example, the processor 202 receives and processes the task data in real time. This means that the processor 202 can update the board 109 when task data is received from the task manager and completes this calculation before the task manager sends the next task data update, such as data indicating the completion of a task or the assignment of a new task. Timing of the task scheduling system can be configurable.

Although communications port 208 and user port 210 are shown as distinct entities, it is to be understood that any kind of data port may be used to receive data, such as a network connection, a memory interface, a pin of the chip package of processor 202, or logical ports, such as IP sockets or parameters of functions stored on program memory 204 and executed by processor 202. These parameters may be stored on data memory 206 and may be handled by-value or byreference, that is, as a pointer, in the source code.

The processor 202 may receive data through all these interfaces, which includes memory access of volatile memory, such as cache or RAM, or non-volatile memory, such as an optical disk drive, hard disk drive, storage server or cloud storage. The computer system 200 may further be implemented within a cloud computing environment, such as a managed group of interconnected servers hosting a dynamic number of virtual machines. 8 WI00096014 2016202814 02 May 2016

It is to be understood that any receiving step may be preceded by the processor 202 determining or computing the data that is later received. For example, the processor 202 determines the task data and stores the task data in data memory 206, such as RAM or a processor register. The processor 202 then requests the task data from the data memory 206, such as by providing a read signal together with a memory address. The data memory 206 provides the data as a voltage signal on a physical bit line and the processor 202 receives the penetration value via a memory interface.

It is to be understood that throughout this disclosure unless stated otherwise, nodes, edges, graphs, solutions, variables, values and the like refer to data structures, which are physically stored, for example, on data memory 206 or processed by processor 202. Further, for the sake of brevity when reference is made to particular variable names, such as "period of time" this is to be understood to refer to values of variables stored as physical data in, for example, computer system 200 in a suitable manner or form. FIG. 3 is a flow chart of an embodiment of the methods and systems of a task scheduling system. The method includes receiving task data 301. In a software development production environment, task data can include fields, for example, including description, task type, task status, task assigned to, required capability, low estimated duration, estimate variation factor, standard estimated duration, high estimated duration, estimated time to complete, actual duration, interruptible, estimated handover time, staff (full name), card note, actual start, assigned group, capability name, card status description, competed time (UTC), created calendar appointment, estimated, group name, relevant estimate hours, replenishment items, status description, task ID, workflow, and the like. Task data fields, structure or definitions can be configurable. It is understood that this disclosure is not intended to limit the manner in which task data is configured. Task data can be in groups of tasks, such as three and/or five tasks, and can be treated as work flow or a set of work flow items instead of individual tasks to which qualitative assessment is applied. 9 WI00096014 2016202814 02 May 2016 A release gate, by which tasks are released to the task buffer, may be influenced by a variety of factors, one of which is a qualitative assessment. In one example, a qualitative assessment can be based on acceptability bands. The other factors may be nudge, work queue, tags, rules, workflow date created, available capacity, release delay time and factor, universal copy schedule, critical chain project management criteria, and earliest start date in addition to the qualitative analysis.

As mentioned, depending upon the circumstances, qualitative assessment can utilise the CPU beyond predetermined limits and/or extend the time for processing beyond predetermined limits. In the case of acceptability bands as a qualitative assessment, when CPU usage or elapsed time exceeds the predetermined threshold, the strictness of criteria can be relaxed. Alternatively, different criteria can be applied. In this way qualitatively assessing the received task data according to first qualitative criteria 303, the task data qualitative assessment to determine a distribution mix of tasks to resources and monitoring CPU usage or elapsed time to determine whether the qualitative assessment is utilising CPU usage above a predetermined threshold 305, and when CPU usage or elapsed time exceeds the predetermined threshold 307a, changing the qualitative assessment from the first qualitative criteria to a second qualitative criteria 309, ceasing qualitatively assessing according to the first qualitative criteria to then qualitatively assessing the received task data according to the second qualitative criteria to generate a distribution mix of tasks to the resources to distribute the tasks to the resources within a preset CPU or elapsed time limit, can deliver a mix of tasks to the task buffer 311a or 311b.

As depicted by FIG. 3, the process can continue until the tasks are delivered. That is, if while monitoring CPU usage or elapsed time, it is determined that the qualitative assessment according to the second criteria CPU usage or eiapsed time exceeds the predetermined threshold 307b, then the systems and method of the disclosed task scheduling system may change the qualitative assessment from the second criteria to the third criteria, and to a fourth, and so on, depending upon the 10 WI00096014 2016202814 02 May 2016 circumstances, in this way a mix of tasks delivered to the task buffer can be of the highest qualitative assessment, given the computational and/or timing circumstances.

As mentioned, a qualitative analysis can be based on acceptability bands. FIG. 4 depicts, in a software development production environment, what an acceptability band may look like when displayed on a display of a work station or board as depicted in FIG. 1. In this case, it is called "Red Priority" as depicted in the window 401. The acceptability band boundary values are depicted so that the ranges (which are configurable) are selected for a particular reason. Ranges for example can include empirically determined values. In use, for example, acceptability bands can have ratings of risk associated with them, here including caution. FIG. 4 further shows that the desirable state 403 in this example is "excellent max" 405.

In applying the acceptability bands to task data for qualitative assessment, it may be that task data can be processed 407 more than one time, that is via Pass 1, Pass 2, Pass 3 and Pass 4. It is preferable that the task data which is delivered to a resource is of a mix so that the work is achievable and/or of a particular quality within time constraints. However, it is not always possible to deliver an optimal task mix. In a first instance, releasing work in Pass 1 only if range violations are avoided so that the mix does not get any worse may result in no work being released at all. In that event, Pass 2 may be required, and so on.

However, if during Pass 1, while monitoring CPU usage or elapsed time to determine whether the qualitative assessment is utilising CPU usage above a predetermined threshold, and if CPU usage or elapsed time exceeds the predetermined threshold, Pass 1 may be aborted so that Pass 2 may be initiated. Furthermore, depending upon whether the qualitative assessment is utilising CPU usage or time elapsed above a predetermined threshold, Pass 2 may be skipped and Pass 3 may be processed, and so on. While this example shows the same acceptability band used in Pass 1, Pass 2, etc., and the strictness relaxed, a different example could include application of a different qualitative assessment, such as a different acceptability band. 11 WI00096014 2016202814 02 May 2016 A process can embody the conditions for when CPU usage or eiapsed time exceeds the predetermined threshoid. For example, conditions can be applied in a software development production environment as depicted in the graphics of FSGs. 5 and 6. FIG. 5 depicts a user interface graphic to set when the default value will be overridden. For example, where Pass 1 is aborted and refreshes so that Pass 1 may be re-initiated. In this example, the default value will be overridden in 40 minutes. FIG. 6 depicts a user interface graphic to set where Pass 1 Acceptability Bands (ABs) are bypassed and work is released according to a fail back of according to a release sequence or Pass 2 (see above) depending upon the circumstances. The threshold conditions for CPU usage and/or time eiapsed during CPU usage are configurable.

Disclosed are systems and methods for assessing a measure of acceptability for sets of workflow items, where the assessment is based on the composition of a set of a plurality of workflow items, measures being referred to as acceptability bands.

Workflow management systems aim to improve efficiency in an organisation using defined workflows for different jobs and processes. Workflow management systems can be software systems that help define, administer and coordinate business processes. Such systems enable automation of notification of task allocation to individuals and monitoring work progress to ensure tasks are followed up. Some workflow management systems may automatically allocate tasks for execution. However, methodologies applied for automating allocation of tasks from a task queue tend to be simple, for example First in First out (FIFO), deadline based or following schedule defined manually by a project or business manager.

Most workflow management systems utilise a high degree of project or business manager input for scheduling as the manager can instinctively reorganise work allocations to address potential bottlenecks or balance workloads across different department and employees and modify scheduling or task allocation to address these potential problems. Managers can act flexibly to address the problem. However, automated task allocation in a workflow management system is 12 WI00096014 2016202814 02 May 2016 typically based on defined rules and specific inputs. For example, this may result in the system continuing to allocate tasks to an already overloaded resource or team. In such a situation, instead of efficiency, productivity wanes. An automated system may be able to provide an indication of a bottleneck or other issue occurring but human intervention is typically required to address the problem, for example to manually reallocate tasks or modify allocation rules/criteria.

Productivity in private and public entities is of critical importance, and in some cases, to the survival of an entity in harsh business climates and/or to the overall gross domestic product (GDP) of a town, city, state or country. It cannot be overstated that productivity is a substantial component of innovation and evolving business paradigms. Productivity is directly related to the management of workflow. As mentioned above, entities utilise workflow management system products, many of which operate in conjunction with one or more computers. A management workflow system is a product that can be either sold as a standalone product or can be sold as a software as a service (SAS) or otherwise. Disclosed is a product including acceptability bands that can enable the release of work to resources (human or otherwise) in a way that avoids bottlenecks at the outset and can provide a workflow situation where the resources are able take up and complete tasks in a way that avoids over allocation and encumbering other resources.

Disclosed are methods and systems for configuring and utilising acceptability bands in a product that is a management workflow system. Such a product is utilised in different private and public entities to manage resources, in particular, human resources that are performing tasks. While this description involves a software development management workflow system, any system in which resources are utilised is within the scope of this discussion and some are mentioned herein.

In furtherance of that end, also disclosed are systems and methods processing workflow items utilising acceptability bands to determine whether to release certain work items into a workflow buffer. Also disclosed are systems and methods for visualising acceptability bands as display output and/or utilising the acceptability bands without visualising them as display output. 13 WI00096014 2016202814 02 May 2016

More particularly, disclosed are workflow management methods and systems carried out on one or more computers for determining a measure of acceptability based upon one or more particular workflow management variants that are exhibited in workflow items of a set of workflow to determine a set attribute score corresponding to an acceptability band, including receiving input of workflow management variant values of the workflow items of the set of workflow items, the workflow management variant values providing at least one of a quantitative and qualitative evaluation indicative of one or more workflow management variants exhibited in the workflow items wherein workflow management variant values include quantitative time estimates established according to input of a responsible person of a low end time value estimate and an x-factor which is a multiplier value of the first time estimate value, determining the quantity of workflow items of the set of workflow items that satisfy one or more subset criteria, the subset criteria providing filtering values for filtering the workflow items, the subset criteria being based upon one or more workflow management variant values, and generating based on the workflow items that satisfy the subset criteria the set attribute score for the set of workflow items. Moreover, disclosed is comparing the set attribute score to the acceptability band wherein predetermined acceptability and diminishing acceptability relative to the predetermined acceptability are determined as a plurality of quantifiable values for the set of workflow items and from the comparison, generating as output one of the plurality of quantifiable values for the set of workflow items as a measure of the acceptability of the set of workflow items, determining based at least in part upon the measure of the acceptability of the set of workflow items to release one or more workflow items to a workflow buffer of a resource and delivering to the resource the workflow items of the workflow buffer. Also disclosed is that generating the set attribute score comprises aggregating the quantity of workflow items that satisfy the subset criteria.

Also disclosed are methods and systems that include other workflow management variant values, not necessarily time estimates that are utilised in the disclosed workflow management systems products. Also disclosed are methods and systems for configuring an acceptability band. 14 WI00096014 2016202814 02 May 2016 including determining at least one workflow management variant that is exhibited in a set of workflow items, providing a criteria filter based upon one or more criterion to be applied to the set of workflow items to determine a set attribute score based upon a subset of workflow items that satisfy criteria of the criteria filter, establishing an acceptability band having acceptability levels to measure acceptability based upon one or more particular workflow management variants that are exhibited in workflow items of the set of workflow items to correspond to the set attribute score and establishing a correlation of the set attribute score to one or more levels of the acceptability band to provide output of the acceptability band.

Also disclosed is a collective attribute assessment system for quantitatively assessing the collective qualities of a set of items, the system comprising a data store and a processor, the data store storing for the set of items a plurality of subset criteria such that the membership of a subset can be determined for each item in the set and the plurality of memberships thereby determined for each item in the set, the data store also storing a plurality of attributes of the items, item attributes, which are used to determine subset membership, the data store also storing for the set of items a plurality of attributes of the set of items, set attributes, and for each set attribute an acceptability band for the set attribute whereby preferred acceptability and diminishing acceptability relative to the preferred acceptability can be determined as a plurality of a quantifiable values for the set attribute, the data store also storing which quantitative item attribute will be aggregated into a quantitative set attribute, and also storing which aggregation function such as count, sum, mean, median, mode, standard deviation, variance or any other statistical aggregate function is to be used to generate a score of the subset which will be compared with the acceptability scale. Further disclosed is that the processor is configured to receive and store in the data store an input set of items, wherein each item in the set exhibits one or more item attributes of the plurality of attributes of the items, for each item in the set, determine its membership of each of the plurality of subsets, for each subset compute the relevant aggregation function to provide an aggregated quantitative set attribute being a score which can be compared with the acceptability scale; for each of the plurality 15 WI00096014 2016202814 02 May 2016 of set attributes, compare the score with the acceptability scale to choose an acceptability value from one of the plurality of a quantifiable values for the acceptability band and output for the set the collective acceptability measures for each attribute of the set as in input to further processing based on the collective acceptability measures for the set or display of the collective acceptability measures of one or more attributes of the set utilising a visualisation methodology.

Disclosed is a management workflow system product including methods and systems for configuring and utilising acceptability bands and in one embodiment, utilising acceptability bands to release certain work items into a workflow buffer. Disclosed is generating as output one of the plurality of quantifiable values of an acceptability band for the set of workflow items as a measure of the acceptability of the set of workflow items. For particular workflow management variants that are exhibited in workflow items of a set of workflow items, by determining a set attribute score, that score is compared to the acceptability band wherein predetermined acceptability and diminishing acceptability relative to the predetermined acceptability are determined as one or more quantifiable values for the set of workflow items. Workflow management variants, in one embodiment, can include time estimates established by utilising an x-factor. Acceptability band values can be provided visually or may be utilised in the system background.

The disclosed workflow acceptability assessment systems and methods provide a method that is inextricably linked to on a computer. Data representing a set of workflow items are stored in a database which are filtered according to criteria to determine which of the workflow items satisfy the criteria, those that satisfy the criteria being a subset of the workflow items. The set of workflow items then receives a set attribute score which is compared to an acceptability band which is a measure of the acceptability of the set of workflow items. The data can be stored on a SQL server. Data filters, or filter strips which provide criteria can be applied to the data stored at various times, including time intervals. 16 WI00096014 2016202814 02 May 2016

As an example, were the data stored to include workflow management variants such as time estimate values of work items, a filter can be applied to the time estimate values to determine a subset of the work items that satisfy certain criteria. For example, if the criteria were that the time value is one to two hours, and 10% of the work items satisfied that criteria, the 10% can be represented as a set attribute score. The acceptability band may assign values like those of FIG. 4b. Percentage is a form of aggregation which includes any type of aggregation including such as count, sum, mean, median, mode, standard deviation, variance or any other statistical aggregate function is to be used to generate a score of the subset which will be compared with the acceptability scale. Flere in this example, the set of work items where 10% has a time value of one to two hours may be flagged as problematic. The mix of the work items could therefore be changed so that time values are instead, for example, three to five hours, so that instead of a set attribute score of 10%, the number is closer to 40%, therefore providing a mix of work items which has a better acceptability. The buffer management system can utilise acceptability bands to determine a mix of work items released into a workflow buffer and therefore move workflow items to a workflow buffer (that is distributed to resources), the buffer often displayed on a workflow display board (such as an electronic board or computer display screen).

When there are thousands or even million work items being generated, in for example, a software development company, keeping the mix of work right for each resource, and/or for each group of resources, can provide for efficiencies. There may be other factors than time, some of which are discussed below, impacting the mix, however, time expected to complete is of course important to, for example, managing handoffs and reviews.

The time estimates for work items can be established by one or more responsible person where input is a low end time value estimate and an x-factor which is a multiplier value of the first time estimate value. High end time value estimates and a divisor as an x-factor is within the scope of this discussion as well. As will be discussed in more detail below, providing time estimates for 17 WI00096014 2016202814 02 May 2016 workflow items would be a step in establishing the workflow item variants of workflow items stored in the database. While time estimates can be made in any suitable manner, the x-factor method of providing time estimates is within the scope of this discussion.

As mentioned above, the disclosed workflow acceptability assessment systems and methods provide a method that is inextricably linked to on a computer. In particularly, acceptability bands can be used to assess the state of a system and provide reports on that state in terms of boundaries (as mentioned above with reference to FIG. 10c.) Acceptability bands, or portions thereof, can be a visibility tool exposed on visual boards such as computer displays. For example, on a display board or computer display screen, a particular component of a workflow item, either a bucket or a buffer component, may have attached to it an acceptability band. Examples of components in the described system are Channel and Chunk, waiting on Prerequisites to Clear, Waiting on Capacity, Capacity Bypass, Waiting on External Prerequisite. As will be shown in more detail below, a window on a computer display screen can be used to configure the acceptability bands for components (of workflow items). Acceptability bands attached to a component can contribute to the overall status of the component regardless of whether they are exposed on the display board.

An acceptability band gives an operator an opportunity to describe a profile of desirability for specified criteria, where the profile of desirability is assessed for a set of items collectively. Any one or more of the items of the set may exhibit the specified criteria but the assessment is based on the set composition, based on the contribution of the items exhibiting the specified criteria in the context of the whole set, rather than considering individual items in the set. The profile of desirability can be defined by an operator based on any applicable criteria, and defines a predetermined acceptability and diminishing acceptability as a plurality of quantifiable values for the set of items.

Considering a set of currently active workflow items in a workflow management system, each of the workflow items will exhibit particular workflow variant values, for example task type, 18 WI00096014 2016202814 02 May 2016 allocated operator, task duration, task cost, customer type, customer location and other workflow item attributes applicable for the workflow items. The workflow management variant values provide at least one of a quantitative and a qualitative evaluation indicative of one or more workflow management variants exhibited in the workflow items. Below are examples of the disclosed methods and systems in first a general example, and then one specific to workflow management variants.

An example of an acceptability assessment system is shown in block diagram form in Figure 7 and the flowchart of Figure 8 illustrates an example of the process executed by the system 100.

The system 700 is implemented in a computer system comprising a processor 710 and memory 720. Stored in memory 720 is acceptability band definition data 760, including subset criteria 770 and quantifiable vales 780 for one or more acceptability bands. A current set of workflow items 730 may be accessed from a workflow system database or data file accessible to the processor 710. In step 805, the processor 710 receives input of workflow management variant values for the set of current workflow items 730, for example forwarded to the processor in a data signal from a workflow management system or received in response to a query from the processor.

The processor is configured to process the received workflow item data to determine the workflow items that satisfy the subset criteria 770, for example applying a filter 740 to the workflow items 810. In an alternative embodiment the workflow item data is stored in a database and the subset items identified by a database query structured based on the subset criteria to return the workflow items of the set matching the subset/filter criteria, for example an SQL query. In one example, depending on the structure of the query, the data returned may be a count of the number of set items complying with the subset criteria. Alternatively the data returned may be data extracted from the workflow item data of the workflow items complying with the subset criteria, for example task duration data extracted from the workflow item data record of each workflow item complying with the subset selection criteria. 19 WI00096014 2016202814 02 May 2016

For a set of workflow items, the workflow variant values are used to determine a subset of workflow items that satisfy subset criteria associated with an acceptability band. Before further describing FIG. 8, the present discussion turns to FIG. 9.

Figure 9 illustrates a general example of selecting subsets of items based on subset criteria.

In the example of Figure 9 shows a set of five items 900 and a table of item attributes 910. The table 910 shows attribute values of each set item for three attributes: a) size b) colour c) texture.

Subset criteria are defined based on the attributes. The subset criteria 920 for subset A 925 is the colour attribute (attribute b) has the value "red". From the table 910 the two matches 922, 924 with the attribute b value of "red" occur in the set, therefore items 1 926 and 3 927 are members of subset A 925.

Subset criteria 930 for subset B 935 are the size (attribute a) value is greater than five and the texture attribute (attribute c) has a value of "smooth". From the table 910 two matches are found for the "smooth" attribute value 932 and three matches are found for size value exceeding five 934. The only items that satisfy both criteria 936 are items 3 and 4 so these two items are members of subset B 935.

In the context of workflow items the item attributes are workflow variants and the attribute values workflow variant values. The same logic for set membership can be applied for a wide variety of different attribute types and combinations.

An acceptability band may define measures of acceptability based on the proportion of the set of workflow items that have the task type, allocated operator, task duration, task cost, customer type, customer location and other workflow item attributes applicable for the workflow items, for 20 WI00096014 2016202814 02 May 2016 example a predetermined acceptability may be 30%, indicating a target for 30% of the workflow items of the set to use, with diminishing acceptability for proportions of workflow items above and below 30%.

Thresholds may be defined within an acceptability band for indicating diminishing acceptability and risk or warnings. For example an acceptability band may be defined having a target value of 30% and a range around 30%, say 25% to 35% as "on target", "Excellent", or level 1, ranges of 15% to 25% and 35% to 45% as "near miss", "good" or level 2, ranges of 5% to 15% and 45% to 55% as "off target", "caution" or level 3, and ranges below 5% and above 55% as "high risk", "warning" or level 4.

It should be appreciated that any arbitrary or non-arbitrary value or label may be assigned for a quantifiable value to be returned as an acceptability measure for the set of workflow items. Arbitrary values may be useful to gauge the effectiveness of the range and establish a suitable acceptability measure. The acceptability measures and ranges or acceptability functions will vary between embodiments and implementations. It should be appreciated that the different workflow item attributes can vary greatly between workflow item types and system embodiments. Similarly acceptability band definitions will vary greatly. Acceptability bands may be defined based on subjective opinion of a workflow manager or based on historical assessment of workflow operations, any method of defining acceptability bands is contemplated within the scope of the described systems and methods.

For a returned subset, of workflow items a set attribute score is generated 920 by the processor 710 for each subset. In some embodiments the step 820 of generating the set attribute score comprises aggregating the quantity of workflow items that satisfy the subset criteria, for example, counting of the number of items in the subset. For example, with reference to the example of Figure 3 the attribute score for attribute b = "red" is two. Aggregation is meant to include any type of aggregation. 21 WI00096014 2016202814 02 May 2016

Alternatively the step of generating the set attribute score 820 can be based on at least one of the quantitative and qualitative evaluation indicative of one or more workflow management variants exhibited in the workflow items that satisfy the subset criteria. For example the attribute score may be calculated based on the workflow variant values for the subset. The processor may implement a calculator 750 to apply defined calculation rules defined as part of the subset criteria 770. The step of generating the set attribute score 820 may also comprise determining a quantitative value for the workflow items that satisfy the subset criteria relative to the total set. For example, for subset B 935 a set attribute score may be based on a proportion of the set comprising the subset, in the example of Figure 9, the set attribute score for subset B may be calculated as two fifths of the set, 0.4, or 40%.

The set attribute score for the subset is then compared with the acceptability band defined for the subset 825. The acceptability band may define two or more contiguous ranges for comparison with the set attribute score and each range is associated with one or the plurality of quantifiable values. For example, subset A may have an acceptability band, for "redness" of the set, defined based on count value. This acceptability band may be defined as a of one or less is undesirable, preferred redness is a count of two or three, a count of four is undesirable and a count of five or above highly undesirable. Thus, qualitative values can be assigned to quantitative set attribute values. For subset B of figure three an acceptability band for "large smoothness" may be defined to deem values below 20% or above 40% as undesirable and acceptable within the range of 20% to 40%. The acceptability band defines a relationship between set attribute values and acceptability. From which predetermined acceptability and diminishing acceptability can be determined for the set of workflow items.

Some acceptability band examples are illustrated in Figures 10a to lOd. Figures 10a to 10c illustrates acceptability bands having a plurality of contiguous ranges and an acceptability measure associated with each range. In the example of Figure 10a the acceptability band may be based on a 22 WI00096014 2016202814 02 May 2016 count value or other calculation returning an integer for the set attribute score. In this example a predetermined acceptability of "excellent" is defined for scores within the range of 25 to 30 and diminishing acceptability relative to the predetermined acceptability are defined. In this instance the measure of acceptability diminishes with increasing distance from the target acceptability. In this instance ranges of "good" acceptability are defined for the ranges 11 to 24 and 31 to 40 and further diminishing acceptability of "caution" for the ranges of 0-10 and above 41 or above. For example, such an acceptability band may be applicable for allocation of workflow items to a specified resource and risk management for the resource. In this case the "caution" ranges indicate under and over utilisation respectively, and the good and excellent ranges acceptable and target utilisation respectively.

Figure 10b is an example of an similar acceptability profile to that of figure 10a, having a target acceptability bounded by ranges of diminishing acceptability, however in the case of Figure 10b the ranges are defined based on percentage values. For example, the percentage of time for execution of tasks of a given task type, out of the total execution time for the set of workflow items. In another example, a proportion of workflow items of the set having a specified item type, calculated from subset count compared with total set count. In this example the processor determined with range the calculated value x falls within.

In the examples of Figure 10a and 10b the contiguous ranges radiate from a target range representing the predetermined acceptability and the acceptability of each range diminishes based on increasing distance from the target range. The example of Figure 10c shows an acceptability band where acceptability measures diminish from a target acceptability at one end of the acceptability band range. For example this acceptability band may be applied for assessing defect or errors. In a manufacturing context this may be product defects. In a workflow management contents this may represent workflow item errors, for example handover errors or data entry errors requiring rectification in the currently set of workflow items. In the example of Figure 4c the target 23 WI00096014 2016202814 02 May 2016 value is zero or no errors, only one or two errors is deemed "good", three errors acceptable, four to 6 errors "caution", seven to ten "high risk" and above ten "warning". As should be apparent the acceptability bands can be useful for defining and determining risks or providing indicators for potential risk within a set of workflow items.

The example of figure lOd is an example of a continuous acceptability function where values indicative of varying acceptability y are defined as a function of the acceptability score x. For example, configuring an acceptability band may result in a function which may be based on a defined equation or modelled using historical data and probabilistic analysis. It should be appreciated that acceptability bands are configurable to suit the workflow management environment in which the system is implements and the acceptability band may also be changed or reconfigured in response to changing environments. For example, acceptability bands may be redefined in accordance with business practice changes or in response to changes in conditions, for example in response to financial stress, or seasonal changes.

Again referring to FIG. 8, from the comparison 825, the processor generates as output 830 one of the plurality of quantifiable values for the set of workflow items as a measure of the acceptability of the set of workflow items 790. Where there is more than one acceptability band to evaluate 835, the processor can iteratively assess each band by applying the next subset criteria 840 and repeating steps 810 to 835.

In some implementations the system is configured to compare the output acceptability measures to particular workflow management parameters 850 to determine whether adjustment of the set of workflow items should be made 825. This is the case where mix of workflow items may not be optimal and either a less than optimal mix is provided (as a fall back) or the acceptability bands' standards may need to change (as a fall back). If the determination indicated that no adjustment is required 825, this may be signalled to the workflow management system and optionally output to an operations manager 860. The system can then be configured to monitor for 24 WI00096014 2016202814 02 May 2016 changes to the current workflow item set 865 to trigger automatic repeat of the assessment process, or wait for an external (user or associated system initiated) or internal (periodic) trigger to repeat the assessment process.

As mentioned, tasks can be, for example, of different types of work: projects, defects, paid work, and ordered by the boss. In a computer programming environment, work items may wait for distribution to resources, that is, computer programmers. The work items prior to distribution would be stored, in for example, a database of, for example an SQL server. The following table may represent the filters used on tasks stored in for example a SQL server to release tasks into the workflow buffer from a pre-buffer bucket.

Type Caution Min Good Min Excellent Min Excellent Good Max Caution Max Project 0 0 0 40 50 60 Defects 0 0 0 30 35 40 Paid work 0 0 0 20 30 40 Ordered by the Boss 0 0 0 3 3 3

From these ranges, a mix of work items released to the work buffer can be provided. As mentioned, other ranges may be utilised. It may be a matter of trial and error to determine the optimised ranges.

Acceptability bands can be of different types, the type specifying how the value returned by the conditions is used. The acceptability bands may also provide information about workflow items that may be displayed. In this way, the user interface and display controller can be configured to 25 WI00096014 2016202814 02 May 2016 allow users to monitor their work items and work item progress. The user interface and display controller may also be configured to display acceptability measure data for current workflow items. For example, icons indicative of the status of one or more acceptability bands for the set of current workflow items for a team may be displayed, with the icon indicative of the acceptability measure. For example, colour, image, text/numerals or position of the icon configured to convey information regarding the acceptability measure. For example, the colour green for "excellent", yellow for "good" and red for "caution". Alternatively rankings may be used, for example ABCD, 1234 etc. In another example "smileys" or thumbs up/down icons may be used to convey acceptability measures.

In a work environment, a resource (a software developer) may have a particular function. A new resource may need all work reviewed. A senior resource may be tasked with reviewing work. A display board for example, may be set to show different acceptability bands with reference to particular resource. Examples of acceptability bands that can be used against work items include, for example, pending review, critical incidents. More are discussed with reference to FIG. 14 below. When the acceptability band is applied against work items, the acceptability can be noted and displayed in a manner for example, as described above.

For the pending reviews, the subset criteria that provide filtering values for filtering workflow items, the subset criteria being based upon one or more workflow management variants exhibited in the workflow items, for pending review, can include: • Where the workflow is in the 12 day buffer • And the workflow's release group is in a particular group • And the workflow's name does not contain 'Quality iteration' • And the workflow is open • And the work to be reviewed is completed • And the review is not completed. 26 WI00096014 2016202814 02 May 2016

The above-list of subset criteria may include one or more criterion that are functional. That is, the determination of whether the subset criterion is met is a based upon one or more attributes. The subset criterion therefore can be dependent upon other measurable qualities.

Pre-filtering of workflow items may take place to provide a short cut for determining whether the subset criteria being based upon one or more workflow management variants exhibited in the workflow items exists. That is, the prior filtering can determine the measure of acceptability, and the work item can be tagged as such. In this way, the item has been pre-filtered.

Filtering can occur at any suitable time. It may occur when a display board showing work items is refreshed. It may occur at particular intervals, such as thirty minute intervals. Filtering can occur in any suitable manner.

Determining whether or not adjustment of the workflow items should be made 225 can be performed based on one or more of the output measures of acceptability and any further defined criteria. For example, a hierarchy may be defined for different acceptability measures, whereby a determination of adjustment being required may be based on a single acceptability measure. In other examples acceptability measures may be aggregated or combined in accordance with defined rules to provide a collective assessment of the acceptability of the set based on assessments based on a plurality of acceptability bands.

In some disclosed systems and methods a target proportionate mix for two or more measures of acceptability is defined for the set of workflow items. In this example, the processor is further configured to determine the set attribute score and quantifiable value for the set of workflow items for each of the measures of acceptability for the target proportionate mix. Process the set attribute scores and quantifiable values for the measures of acceptability to determine the relative proportional values for the two or more measures of acceptability for the set of workflow items, and compare the relative proportional values with the target proportionate mix for the set of workflow items. The comparison result is the output to trigger adjustment of workflow items 270. 27 WI00096014 2016202814 02 May 2016

The adjustment of workflow items may be performed manually by a project or business manager.

For example, manually reallocating workflow items to add and/or remove workflow items to the set. The adjustment of the workflow items may also be performed automatically by a workflow management system. For example, based on difference between the target proportional mix and the current set proportional mix, selecting work items to add or remove from the set based on the biggest difference between the current and target mix.

The desired proportional mix may be defined based on a subset of a set of a plurality of measures of acceptability determinable for the set of workflow items. For example, a desired proportionate mix may be defined based on a workflow variant type. In different industries, workflow variants can be defined to suit the industry. Up until now the discussion has been with regard to software development. However, others including shipping can utilise acceptability bands in much the same way. For example transport mode, which may take on the values of "sea freight", "air freight" and "land transport". A target proportionate mix for the set may be defined based on this workflow variant. For example, the target proportionate mix being 40% air freight, 30% sea freight and 30% land transport. The target proportionate mix may correspond to target acceptability assessment values. For example, a proportionate mix for a current set may be 10% air freight, 40% sea freight and 50% land transport. In this instance a set attribute value for air freight of 10% may return a high risk acceptability assessment measure, the 40% value for sea freight a good acceptability measure, and 50% land transport a caution acceptability measure. In this instance the workflow variant values giving rise to low acceptability are also workflow variant values showing the biggest deviation from the target mix. However, if the acceptability band is based on a workflow variant value that is not calculated proportional to the set, for example a revenue measure the comparison between acceptability measure and proportional mix may be different. For example the 50% of land transport type tasks may have revenue acceptability measure indicating "on target" or "excellent". Whereas the 40% sea freight type tasks may have revenue acceptability of "adequate" and the 10% air freight have a revenue acceptability of "poor". In this example of an application of 28 WI00096014 2016202814 02 May 2016 acceptability bands, considering the acceptability measures for revenue and proportionate mix in combination, indicates that the business needs to look at generating and bringing into the current workflow item set more workflow items relating to air freight and sea freight, rather than remove land transport type workflow items.

The workflow management system may trigger a reassessment of the acceptability measures to determine whether the adjustment has made a comparatively positive or negative impact. It should be appreciated that set adjustment and recalculation can be repeated iteratively until the target proportionate mix is achieved or other end criteria have been met. For example, a percentage improvement, a set number of iterations, diminishing improvement, maximum options tried, etc.

The disclosed systems and methods can be utilised as an component of a work scheduling and tracking system or as a standalone workflow assessment system. An example of a work scheduling and tracking system incorporating an embodiment of the estimation method is illustrated in Figure 11. The work scheduling and tracking system is implemented in a computer system environment comprising processor resources and memory 1120 resources used to implement program system functionality including scheduling of workflow items and tracking execution by a scheduler/tracker 1180 using defined work item data 1170 and managing acquisition and dissemination of work item data from users by a user interface and display controller 1160.

Work item data 1170 for a plurality of work items is stored in memory 1120, for example as one or more data files or in a database structure. The scheduler/tracker 1180 is configured to enable work item data to be entered into the system, for example based on project task definition, customer orders etc., for execution and tracking. The scheduler/tracker may provide tools to facilitate work item definition and scheduling. The scheduler/tracker may facilitate work item definition and storage in a work item definition storage location of a database. Data elements representative of each of the work items can be separately stored in data fields defined in the 29 WI00096014 2016202814 02 May 2016 database, the data representing each work item can include an identifier data element which can be used to render for display information to allow a user to identify the work item and work item variant data fields for storing work item variant values. The work item variant values may be stored based on input data or in response to workflow item scheduling and execution. The scheduler/tracker may include functionality for automated monitoring and update of work item execution data. For example, managers and team leaders may input initial work item definitions and initial data, the data for these work items can be subsequently updated manually or automatically based on activity of project team members, for example in response to deliverable being completed or periodic status updates of work progress input form user terminals and updated in the system memory 1120. The user interface and display controllerl60 is configured to control display of project information to users, (for example, via their user terminals 1140a-c or common resources such as a project tracking display screen 1150), outputting of reports, and receipt of input data from users, for example from networked user terminals 1140a-c. For example, the user interface and display controller 1160 may be configured to control presentation of a display screen displaying workflow items identification data and data for workflow items allocated to a user for execution.

This may include display of graphical user interfaces to assist input of data values, for example displaying duration selection sliders, date selectors etc. manipulable by a user to graphically indicate a numerical value, for example, number of person minutes or hours, calendar based estimates etc. to input data for a workflow item.

The user interface and display controller may also be configured to perform data transformation of input data from a file input into alternative file formats or database entries. For example, conversion of a data input in a spreadsheet format to database data entries. Input format conversion may also convert input data from input formats incompatible with other system components into compatible formats. Alternatively an input interface and display controller embodiment may be configured to perform data extraction or data mining to automatically identify project relevant data from monitored inputs or associated systems, for example data mining from 30 WI00096014 2016202814 02 May 2016 change requests, and extraction of project relevant time entries from a timekeeping system. It should be appreciated that the above described examples of project management and work tracking functionality is non-limiting and the different functions implemented by the project management and work tracking system will vary between embodiments and different functions can be implemented to tailor the system for different project environments. The acceptability processor is configured to retrieve workflow variant value data for a set of workflow items from the work item data 1170 stored in memory 1120 and perform acceptability assessment utilising defined acceptability band data 1175 as described above with reference to Figure 8. The system 1100 may also be configured to allow users to define acceptability bands. For example, the user interface and display controller may be configured to display data entry screens to facilitate definition of subset or filter criteria for an acceptability band and define the acceptability measures and associated ranges and functions. This may also include defining set attribute calculation criteria. The data entry screens may also be configured to facilitate re-configuring or re-defining acceptability bands. Data entry screens can also be provided to allow definition of one or more desired proportionate mix of workflow variants for a set of workflow items.

The system can be implemented using a variety of different system architectures. For example, in an embodiment the processor and memory resources may be provided by a general purpose computer such as a personal computer (PC), laptop/tablet computer, or server having internal memory including volatile (for example, random access memory RAM, buffers etc.) and nonvolatile (hard disk, solid state memory, optical disk etc.) memory resources and optionally external memory resources accessible by the processor via wired or wireless data connections. In alternative embodiments the processor and memory resources may be network accessible distributed resources, for example distributed processing and memory resources within a company network, or "cloud based" on demand processing a memory resources accessible via the internet. 31 WI00096014 2016202814 02 May 2016

In this example the work scheduling and tracking system processing and memory resources are in data communication with user terminals 1140a-c and other communication resources such as display screens via a network 1130. For example, the network 1130 may be a local area network, such as an Ethernet network or secure WiFi within a company premises, or a secure intranet. Alternatively, the network may be a public network such as a telecommunication network or the Internet. The user terminals 540a-c may be laptop or desktop computers, tablets or smart phones etc., having input interfaces such as keyboards, touch screens, scanners, transceivers, microphones and sensors (for example, sensors for motion, speed, temperature, humidity, light, location etc.). Output interfaces can include display screens, speakers, printers etc. The types of input and output devices provides at a user terminal can vary between users terminals and many variation of terminals can be supported by any one system embodiment.

The system functionality is implemented in some embodiments as one or more software programs which execute using the processing and memory resources as described above and interact with external interfaces of the networked computers 1140a-c. The system may be implemented as software programs or routines executable using the processing and memory resources which are accessible via a network 1130 from one or more user terminals which provide user interface capability and local processing capability and memory. In alternative embodiments some system functionality may be implemented using a combination of hardware and software. For example, the network 1130 may be a local area network, such as an Ethernet network within a company premises, or a secure intranet. Different users may be enabled access to the system for specific purposes only. For example, some employees or job categories may be provided with access to view a broad range of data but limited to updating only defined data elements, typically related to work defined for the user. Other users, such as project managers and team leaders may be provided access to enable viewing and updating of broader ranges of data. In some embodiments, updating of some categories of data may be restricted to ensure that only the user to whom work is allocated or an authorised delegate may update the data relating to the work. 32 WI00096014 2016202814 02 May 2016

The acceptability processor may be configured to automatically assess and output workflow item set acceptability measures based on system triggers, for example periodically or in response to actions, such as removal of workflow items from a current operating set in response to workflow item completion and addition of workflow items to the current operating set. For example, in response to automatic work scheduling by the scheduler 1180 or manual allocation by a manager. In some embodiments the acceptability processor may be configured to assess acceptability measures for a nominal set of work items. For example, a nominal set of work items may be defined for hypothetical work item reallocation/rescheduling scenarios either by a project or business manager or generated by the system. The disclosed embodiments may be implemented as a component of a project management system or as a stand-alone system.

As mentioned above, the acceptability band can be a visual tool. FIG. 12a shows how an acceptability band can be visually depicted in, for example, colours. FIG. 12b shows, for example, looking at Green: 0.77% which is a code word for "project" (see above), the fact that it is at 77% says that in the acceptability band that which applies a criteria filter to one or more components of the "project" work items, that 77% is within the excellent range, and that can be determined by the colour associated with it, which is green. Any other form of visualisation is within the scope of this discussion. FIG. 12c depicts that a component "Chunk and Channel" is yellow which according to FIG. 12a, means caution. On a display board, a feature can include the ability to mouse over the visualised acceptability band to see what the criteria for the acceptability band is that has been applied. FIG. 12d further shows that a mouse over may reveal the acceptability band itself. FIG. 13a depicts an acceptability band configuration screen. FIG. 13b depicts a filter configuration screen for the acceptability band configuration. FIG. 13a depicts that a component is selected, in this case "channel and chunk" and aspects of the acceptability band is depicted. In the next screen, FIG. 13b, boundary values can be provided. 33 WI00096014 2016202814 02 May 2016 FIGs. 14a and 14b depict instructions and definitions for the configuration screens. It is understood that any manner in which to define an acceptability band is within the scope of this discussion. Criteria in the form of filters such as those shown in FIG. 9 can be applied. Above, a plurality of criteria for pending reviews, was described. As mentioned, one or more criterion can be functional. That is, the determination of whether the subset criterion is met can be based upon one or more attributes. The subset criterion therefore can be dependent upon other measurable qualities. Time estimates is an example of a criteria (filter strip) that can be utilised as criteria in acceptability bands.

Many methodologies and systems exist for resource estimation in project planning and some systems can be quite effective where resource requirements can be well defined and predictable. For example, where processes are mechanical, routine repetitive tasks such as laying carpet per square metre, or consumption of resources where historical data allows relatively accurate estimation, such as time for consumption of a tank of fuel during average city driving. Reasonably reliable estimates, particularly time estimates, can be produced using historical data for repetitive or routine tasks.

Estimation of effort (typically measured in time or value based on time estimates) for professional services presents a much more difficult problem due to the nature of the work. For example, in research and development work in a project is often new. A new problem to solve or set of requirements to implement may have no accurate historical body of data from which to infer estimates exists. Further, the execution of the work is highly dependent on the individuals performing the work. This can include individual competence and efficiency, available working hours, task loading and other external factors. Other external factors may include personal problems which may impact on work performance, for example, health or family problems, or work related impacts such as office politics or performance perceptions. All of these factors which may influence the individual performing the work can be highly variable and individualised. 34 WI00096014 2016202814 02 May 2016

Some known approaches to try to improve estimation accuracy in highly variable environments include detailed definition of work components and estimating based on each narrowly defined component. Project planners start drilling into the task durations attempting to fix the estimation problem, based on the theory that finer granularity enables more accurate prediction - i.e. the more we know, the more accurate the estimate. This approach may be effective if one were dealing with something that is highly predictable or based on historical data, for example when measuring how long it would take for a tank of petrol to be consumed with average city drivers. In this case, one can obtain a distribution to use for estimation from experimental or historical data. However, finer granularity of estimation may not solve the problem, particularly for professional services. Some narrowly defined tasks may be more accurately estimated but others still suffer from high variability as discussed above.

Some known systems aim to solve estimation problems using various statistical methodologies, involving estimating standard deviations around the means. For example, an estimate of five hours being interpreted as a mean of five hours and a probability distribution existing around that mean estimate, typically assuming a normal "bell" curve distribution. Many methods involve some number that is deemed to be a mean, and then some other numbers that are standard deviations. Such systems typically use a three or four point estimation technique, for requiring estimators to try to accurately predict three of four different values for a task - for example, mean, median, mode, standard deviation - giving an estimate of variability about likely/expected outcomes. Several problems with existing statistical estimation methods have been noted when these are applied for estimation of professional services.

First, many methods assume a normal distributions for probability for the outcome, however the probability distributions tend to be skewed. Typically, a time estimate for a body of work has a much higher chance of being longer than a most likely estimate than it has of being shorter than the 35 WI00096014 2016202814 02 May 2016 most likely estimate. Thus, any method based on a standard distribution is flawed, and in reality losses will accumulate and gains are lost.

Second, significant consideration and/or calculation is required for the estimator to derive the three or four point estimation data. However, due to the nature of the work this calculation requires many assumptions - so the estimates may have been estimated using incorrect assumed values and therefore be inaccurate. There is a reluctance to provide such estimates due to the effort and once an estimate is given, pressure to commit to generating the outcome "on time" at the maximal likelihood estimate given.

Third, estimates tend to be optimistic or pessimistic, this may also be influenced by external human factors, such as a project manager putting pressure on staff to provide shorter time/lower cost estimates. When making a consolidated estimate, an over representation of under or over estimators can significantly influence the outcome.

Another problem is network effects whereby some tasks are executed serially but others may be executed in parallel. Estimates in this case are not all additive, serial tasks are additive but others are not. This means additional data is required to enable network effects to be taken into consideration for a consolidated estimate encompassing several tasks. The effort required to model network effects to enable accurate compensation for network effects in estimates is typically not performed and therefore the mathematics used for calculating consolidated estimates is typically flawed. Historical data may be used to regressively determine an empirical value to use for compensation of network effects, but this will only be effective if the amount of parallel and serialism is relatively constant between projects.

Typically the result of these estimation methods and problems is erroneous estimations which can be misleading, when used in project planning management and tracking systems and induce frustration and lack of confidence. There is a need for a more effective estimation system for estimation of effort for integration into project management and tracking systems. 36 WI00096014 2016202814 02 May 2016

Also disclosed are quantitative time estimation methods and systems of a project management system. Utilising a display driver and a processor, methods include rendering for display a sub-task identifier and one or more user input fields for a sub-task representing high confidence assessment of a first time estimate value and a high confidence assessment of a second time estimate value for the sub-task. The second time estimate value is a multiplier or divisor value representing high confidence assessment of variability from the first time estimate value. The disclosed method further includes from user input, the processor generating an associated centre value derived from two of the low end time value, the high end time value and the range width value and the processor aggregating the associated centre value for each sub-task of a task to generate a consolidated quantitative time estimate for the task for utilisation in the project management system.

The disclosed X-factor method can include in a task scheduling system including a processor configured as an estimation processor and a display, a quantitative time estimation method of generating a consolidated quantitative time estimate for a task comprising a plurality of defined subtasks the steps of for each one of the plurality of defined sub-tasks of a task, outputting for display on the display a sub-task identifier an indicator of data input fields for input of a first time estimate value being in a unit of time and an x-factor value, the fields for receiving input of a responsible person of a first time estimate value determination and an x-factor value determination by the responsible person, the first time estimate value being a quantitative value indicative of one end of a time estimate range, the first time estimate value being a value that the person responsible for the user input has a particular level of confidence in being an end point of a range for time estimation purposes and the x-factor value being an integer multiplier value of the first time estimate value representing the degree of the particular level of confidence that the responsible person has in the first time estimate value. The method can further include receiving instructions by the estimation processor via user input to the display fields for input of a first time estimate value and an x-factor value, the instructions representative of a user data input of the responsible person indicating the 37 WI00096014 2016202814 02 May 2016 first time estimate value, and the x-factor value, wherein the first time estimate value is the low end time value and wherein the multiplier value is used to determine a high end time value being in a unit of time by multiplying the first time estimate value by the multiplier value and in response to receiving the first time estimate value and the multiplier value, the estimation processor determines, using the received first time estimate value and the multiplier value for the sub-task, a low end time value, a high end time value, a range width value being in a unit of time and an associated centre value of the range value. Moreover, the method can include in response to having received the first time estimate value and multiplier value for all sub-tasks of the task and aggregating by the estimation processor the associated centre value for each sub-task of a task to determine the consolidated quantitative time estimate being in a unit of time for the task, without an assumption of a particular statistical distribution. Also the method can include outputting the consolidated quantitative time estimate being a standard estimate which is not an estimate of mean, median or mode for the task for task scheduling by the task scheduling system and incorporating the consolidated quantitative time estimate for the task into a resource scheduling tool of the task scheduling system that is processing a plurality of tasks for work flow management output to generate project estimates.

In a disclosed quantitative time estimation system, the system can include a display driver, a processor and a database. The database can be for storing in a task definition storage location of the database task data representative of one or more defined tasks and wherein each task comprises a plurality of sub-tasks, wherein data elements representative of each of the sub-tasks are separately stored in data fields defined in the database, the data representing each-sub task including a sub-task identifier data element. The display driver can be configured to generate for display on a display screen, a user interface displaying at least one sub-task identifier and provide input fields to input a first time estimate value and a second time estimate value for the sub-task, wherein the first time estimate value is a quantitative value representing high confidence 38 WI00096014 2016202814 02 May 2016 assessment of one end of a time estimate range, and the second time estimate value is a quantitative value representing high confidence assessment of the other end of the time estimate range. The processor can be configured to, in response to receiving data input of the first time estimate value and the second time estimate value, determine using the first time estimate value and the second time estimate value for the sub-task, a low end time value, a high end time value, and a range width value and store the low end time value, high end time value, and range width value as data elements for the sub-task in data fields defined in the database, generate for the subtask an associated centre value derived from the low end time value and the high end time value and store the associated centre value as a data element for the subtask in a data field defined in the database. The processor can be further configured to provide output representative that an associated centre value has been stored for all sub-tasks of a task and aggregate the associated centre value for each sub-task of a task to determine the consolidated quantitative time estimate for the task. The processor can be further configured to output the consolidated quantitative time estimate for the task for utilisation in a project management system.

The disclosed project management systems and methods include a data base for storing in a task definition storage location, task data representative of one or more defined tasks and wherein each task includes a plurality of sub-tasks. Data elements representative of each of the sub-tasks are separately stored in data fields defined in the database, the data representing each-sub task including a sub-task identifier data element. A processor can be in data communication with the database, the processor configured to automatically update task and sub-task data stored in the database in response to receiving data input indicative of task planning and execution. A display driver can be configured to generate and display on a display screen, a user interface to enable data input to define tasks and sub-task in the task definition storage location of the database, wherein for a project planning phase the display driver can be configured to generate for display on a display screen, a user interface displaying at least one sub-task identifier and providing input fields to input a first time estimate value and a second time estimate value for the sub-task, wherein the first time 39 WI00096014 2016202814 02 May 2016 estimate value is a quantitative value representing high confidence assessment of one end of a time estimate range, and the second time estimate value is a quantitative value representing high confidence assessment of the other end of the time estimate range. The processor can be configured to, in response to receiving data input of the first time estimate value and the second time estimate value, determine using the first time estimate value and the second time estimate value for the sub-task, a low end time value, a high end time value, a range width value and store the low end time value, high end time value, and range value as data elements for the sub-task in data fields defined in the database. The process can furthermore be configured to generate for the sub-task an associated centre value derived from the low end time value and the high end time value and store the associated centre value as a data element for the subtask in a data field defined in the database. The processor can be also configured to provide output representative that an associated centre value has been stored for all sub-tasks of a task, aggregate the associated centre value for each sub-task of a task to determine the consolidated quantitative time estimate for the task, store the consolidated quantitative time estimate. The processor can be further configured to apply the consolidated quantitative time estimate for one or more tasks in an automated task scheduling process.

The disclosed systems and methods provide a computer system implemented quantitative estimation method and system enabling generation of a consolidated estimate for a task comprising a plurality of sub-tasks. In a first example, as illustrated in the block diagram of Figure 15, the system comprises an estimation processor which utilises a task definition 1520 data file or database and a user interface and display driver 1530 to facilitate communication with user terminals or other input and output devices. The task definition data file or database 1520 stores data for one or more tasks broken down into sub-tasks. The estimation processor is configured to, for each task, trigger the user interface and display driver to cause display of sub-task data to users and receive input of a first value and a second value for each subtask. In response to receiving data input of the first and second values, the estimation processor determine an estimated range for the sub-task. From these 40 WI00096014 2016202814 02 May 2016 two values only the estimation processor determines, for a range, a low end estimate, a high end estimate, a range width and a centre value. The estimation processor determines a consolidated estimate by aggregating the centre values calculated for each sub-task from the input values for all the subtasks of a task.

The disclosed systems and methods can be utilised as an component of a project management or work tracking system or as a standalone estimation system. An example of a project management and work tracking system incorporating an embodiment of the estimation method is illustrated in Figure 16. The project management and work tracking system is implemented in a computer system environment including processor resources and memory 1020 resources used to implement program system functionality including scheduling of tasks/sub-tasks and tracking execution by a scheduler/tracker 1080 using defined task and sub-task data 1070 and managing acquisition and dissemination of project data from users by a user interface and display controller 1060.

Project tasks are broken down into sub tasks and the task/subtask data 1070 stored in memory 1020, for example as one or more data files or in a database structure. The scheduler/tracker 1080 is configured to enable project data to be entered into the system for initial project definition and project execution tracking. The scheduler/tracker may provide tools to facilitate the initial task definition and subtask breakdown from higher level project specifications and requirements, and task/subtask scheduling. The scheduler/tracker may facilitate task and subtask definition and storage in a task definition storage location of a database task data representative of one or more defined tasks and sub-tasks. Data elements representative of each of the sub-tasks can be separately stored in data fields defined in the database, the data representing each-sub task including a sub-task identifier data element which can be used to render for display information to allow a user to identify the subtask. The scheduler/tracker may include functionality for automated monitoring and update of project execution data. For example, managers and team 41 WI00096014 2016202814 02 May 2016 leaders may input initial project task and subtask definitions and initial data, the data for these subtasks can be subsequently updated manually or automatically based on activity of project team members, for example in response to deliverable being completed or periodic status updates of work progress input form user terminals and updated in the system memory 1020. The user interface and display controller 1060 is configured to control display of project information to users, (for example, via their user terminals 1040a-c or common resources such as a project tracking display screen 1050), outputting of reports, and receipt of input data from users, for example from networked user terminals 1040a-c. For example, the user interface and display controller 1060 may be configured to control presentation of a display screen displaying sub-task identification data and data fields allowing input of estimation values, for example estimation of effort required for completion of a task which will typically be a time based estimate. This may include display of graphical user interfaces to assist input of data values, for example displaying duration selection sliders, date selectors etc. manipulable by a user to graphically indicate a numerical value, for example, number of person minutes or hours, calendar based estimates etc.

The user interface and display controller may also be configured to perform data transformation of input data from a file input into alternative file formats or database entries. For example, conversion of a data input in a spreadsheet format to database data entries. Input format conversion may also convert input data from input formats incompatible with other system components into compatible formats. Alternatively an input interface and display controller embodiment may be configured to perform data extraction or data mining to automatically identify project relevant data from monitored inputs or associated systems, for example data mining from change requests, and extraction of project relevant time entries from a timekeeping system. It should be appreciated that the above described examples of project management and work tracking functionality is non-limiting and the different functions implemented by the project management and work tracking system will vary between embodiments and different functions can be implemented to tailor the system for different project environments. 42 WI00096014 2016202814 02 May 2016

The system can be implemented using a variety of different system architectures. For example, in an embodiment the processor and memoryresources may be provided by a general purpose computer such as a personal computer (PC), laptop/tablet computer, or server having internal memory including volatile (for example, random access memory RAM, buffers etc.) and nonvolatile (hard disk, solid state memory, optical disk etc.) memory resources and optionally external memory resources accessible by the processor via wired or wireless data connections. In alternative embodiments the processor and memory resources may be network accessible distributed resources, for example distributed processing and memory resources within a company network, or "cloud based" on demand processing a memory resources accessible via the internet.

In this example the project management system processing and memory resources are in data communication with user terminals 1640a-c and other communication resources such as display screens via a network 1630. For example, the network 1630 may be a local area network, such as an Ethernet network or secure WiFi within a company premises, or a secure intranet. Alternatively, the network may be a public network such as a telecommunication network or the Internet. The user terminals 1640a-c may be laptop or desktop computers, tablets or smart phones etc., having input interfaces such as keyboards, touch screens, scanners, transceivers, microphones and sensors (for example, sensors for motion, speed, temperature, humidity, light, location etc.). Output interfaces can include display screens, speakers, printers etc. The types of input and output devices provides at a user terminal can vary between users terminals and many variation of terminals can be supported by any one system embodiment.

The system functionality is implemented in some embodiments as one or more software programs which execute using the processing 1610 and memory 1620 resources as described above and interact with external interfaces of the networked computers 1640a-c. The system may be implemented as software programs or routines executable using the processing and memory resources which are accessible via a network 1630 from one or more user terminals which provide 43 WI00096014 2016202814 02 May 2016 user interface capability and local processing capability and memory. In alternative embodiments some system functionality may be implemented using a combination of hardware and software. For example, the network 1630 may be a local area network, such as an Ethernet network within a company premises, or a secure intranet. Different users may be enabled access to the system for specific purposes only. For example, some employees or job categories may be provided with access to view a broad range of data but limited to updating only defined data elements, typically related to work defined for the user. Other users, such as project managers and team leaders may be provided access to enable viewing and updating of broader ranges of data. In some embodiments, updating of some categories of data may be restricted to ensure that only the user to whom work is allocated or an authorised delegate may update the data relating to the work. The disclosed embodiments may be implemented as a component of a project management system or as a stand-alone system.

An example of an estimation method in accordance with an embodiment is illustrated in the flowchart of Figure 17. The disclosed methods and systems provide a quantitative estimation method of generating a consolidated quantitative estimate for a task comprising a plurality of defined sub-tasks. Each task is broken down into a plurality of sub-tasks, the task and sub-task data is stored in a system database or data file. In an embodiment the task and sub-task data is stored in a database accessible to the estimation engine via data communication, for example, via a wired or wireless private network (i.e. Ethernet LAN, WiFi WLAN or combination thereof) or communication network and the Internet.

As shown in Figure 17, for each one of the plurality of defined sub-tasks of a task, sub-task identifiers are retrieved 1710 from memory, for example using a database query, or extracted from a data file or lookup table etc. A displayed sub-task identifier can be a numeric, alphanumeric, text, an icon, image or any other suitable indication of the sub-task requiring a time estimate, and may provide access to data relevant to the sub-task, including, for example, descriptive content, available resources, time constraints, physical constraints and associated attributes about the sub-task and/or 44 WI00096014 2016202814 02 May 2016 the task. The data relevant to the sub-task may be displayed in response to a user input such as clicking on an icon or button to expand a window or display additional data fields.

The user interface and display driver 1530 controls the formatting and rendering for display of the sub-task identifiers 1720, so when displayed a user can input time estimate values. The formatting of data for display may include display of data entry fields or cells and instructions for time estimate value entry. The sub-task identifier can be presented in any form that allows a user, typically a responsible person for the sub-task, to recognise the sub-task and input time estimation data. The data defined for each subtask can include a task/sub-task allocation identifier indicating the team member or team to which the task/sub-task has been allocated. For example, using text, numeric, alphanumeric codes, icons, colours, images, maps etc. Effects such as zoom in and zoom out or expansion and collapsing of tasks and subtasks may also be used to allow visualisation of different levels of detail and granularity of tasks and task breakdown during the estimation process. The user interface and display driver can be configured to selectively render for display sub-tasks for a responsible person or based on allocated users for tasks. For example, subtasks for display may be grouped based on team or team member allocated to the sub-task and display of sub-task ordered by team and/or allocated team member, alternatively only tasks/sub-tasks allocated to a team or team member may be rendered on displays/terminals associated with the allocated team.

For each sub-task 1730 the estimation processor (see FIG. 15) receives and stores in memory a data input indicating a value for one end of a range for the estimate 1740 and a data input indicating range width 1750. In a first embodiment the first time value is a low end estimate for the sub-task and the second time value is a value indicating range width, for example a multiplier. In a second embodiment the first input is a high end time estimate for the sub-task and the second time value is a value indicating range width, for example a divisor. The second value may also be a width value. In a third embodiment the first time value is a low end estimate and the second time value is a high end estimate. In response to receiving input of the first time value and the second time value, 45 WI00096014 2016202814 02 May 2016 the estimation processor calculates from the first time value and second time value for the sub-task a low end time value, a high end value, a range width value and a centre value 1760. In the first embodiment described above the low end time value is multiplied by the multiplier to calculate the high end estimate time value. In the second embodiment described above the high end time value is divided by the divisor to calculate the low end time value. In the third embodiment the high end time value is divided by the low end time value to derive a multiplier indicative of the range width. For each of the embodiments the centre value, which may also be referred to as the standard estimate, can be calculated: , , low end estimate+hiqh end estimate r,_ „. center value = - [Eq. 1]

The steps 1730 to 1760 can be repeated until all subtasks for a task have been completed 1770.

In response to having received and processed the first time value and second time value for all sub-tasks of the task 1770, the estimation processor aggregates the centre value for each subtask of a task to determine the consolidated quantitative estimate for the task 1780, and outputs the consolidated quantitative estimate for the task 1790. The estimate can also be stored in memory or a database. The consolidated estimate may be output to a user via a user interface. Alternatively or additionally the consolidated estimate can be output to one or more further project management or tracking system components, for example a resource scheduling tool. In some embodiments, data output to the project management and work tracking system can also include any one or more of the low end time estimate, high end time estimate, multiplier/range and centre value for sub-tasks. For example, the subtask estimate values output as a data file, data signal or stored in a database part of or accessible to the project management and work tracking system. 46 WI00096014 2016202814 02 May 2016

Alternative or additional consolidated estimates are contemplated within the scope of this discussion. For example, the data defined for each subtask can include an allocated user identifier the estimation processor is further configured to aggregate the centre value of each sub-task for an allocated user of one or more task to determine a consolidated estimate for the allocated user.

The above consolidated estimate generation can be repeated for a plurality of tasks, for example a group of tasks making up a body of work, and a further consolidated estimate generated for a plurality of tasks by aggregating the consolidated estimates for each task.

Consolidated estimates may also be prepared based on a team of allocated users or other task attributes, for example project critical path tasks, project phase, physical resource based task or subtask grouping etc.

Embodiments include the estimate range ends and width which are determined from two input subjective values. For example, in one embodiment one value indicates one end of an estimation range and the other value indicates the width of the range. The person responsible for making the estimation, the responsible person, can be instructed to choose the values to input as subjective low granularity estimates, that the responsible person has high confidence that the subtask outcome can be achieved or delivered within the estimate range. For example, the first value may be a very optimistic estimate, which the user has a high confidence in being a "best case scenario" but not beyond feasibility for execution of the subtask, for example 85-90% confident that a better result will not be achieved. The other value indicative of the width may be indicative of the other end of the range, or a multiplier "n", for example, based on the reasoning that after nominating an optimistic estimate value a pessimistic estimate may be n times the optimistic estimate. For example, 85-90% confident that a "worst case" would not be more than triple the time estimated for the "best case". This estimate input request enables users to input an intuitive "guess" estimate range, using only 2 numbers, which requires little effort to come up with but the responsible person is reasonably certain and prepared to commit to an outcome being achieved 47 WI00096014 2016202814 02 May 2016 somewhere within that range. This system acknowledges that estimation of work is typically inaccurate. Further, by providing an end point and range, or two end points, it is easier for a person to consider other factors which may affect their performance during the task duration, for example state of health, concurrent workload, reliance on external deliverables and other obligation/commitments that may impact on the task execution. Variables and unknowns can be accommodated in the estimation system simply by increasing the multiplier and hence range width.

It should be appreciated that the responsible person, may be an experienced project team making estimates for tasks for which they are responsible for delivering, tasks allocated to that user. The responsible person may also make estimates on behalf of other (typically less experienced or not yet allocated) team members. The responsible person may, in some circumstances, the responsible person may be a notional "responsible person" with the estimate in fact being provided and committed to by a project team. The two estimate values are input by a user, who may be the responsible person or may be another user inputting values with input or under instruction of the responsible person.

Embodiments of the methods and systems enable estimates to be made independent of an attempt to characterise or assume a probability distribution of outcomes. From historical and empirical evidence it has been observed that, typically for professional services and in particular software development, task outcome probabilities tend to show a positively skewed distribution (an example is shown in Figure 18) with a longer tail on the right of the distribution, therefore estimation theories based around normal distributions is typically flawed. Further, the distribution can change from task to task and due to the unknowns for estimation of software projects the actual distribution cannot be predicted with any accuracy in the planning phase of a project.

The disclosed methods and the systems provide a simplified estimation method robust against varying probability distributions. Estimators which may be responsible persons are asked to make a low estimate (90% sure they cannot beat it), which compels one to consider an optimistic 48 WI00096014 2016202814 02 May 2016 and probably lucky low number. Then apply a multiplier, which can be referred to as an "X factor" (a multiplication factor), from which a high end estimate can be derived. In one embodiment the system restricts the multiplier value to being an integer value. The reason an integer value is preferred for a multiplier is because prediction with better accuracy is unlikely without the benefit of hindsight. For example, it may be shown that a correct multiplier was 2.1 with the benefit of hindsight. However, when trying to make predictions humans do not estimate with accuracy. In invoking a predictive and subjective heuristic for coming up with a number, and the method does not suffer the side-effects of the illusion of accuracy and does not suffer from needing historical data. Figure 18 is an example of a skewed probability distribution indicative of likely time taken for a software project task, the low end (LE) estimate, 85% to 90% confidence a better result would not be achieved, and high end (HE) estimate, 85% to 90% confident the work can be done by this time, are easier to predict and commit to achieving than the mean, median, mode, deviation and skew of the distribution.

An advantage of the disclosed embodiments is that the estimation for a task only requires two numbers, indicative of a range without making any commitment to a "most likely" outcome within that range or assumption of a particular statistical distribution.

Empirical test evidence shows that this estimation system does not seem to suffer as badly from human factors as known previously methods. The reason that it works is that it encourages a responsible person to simultaneously consider the optimistic and pessimistic outcome and give a rating, indicative of a degree of confidence they have in their number. If it is a tight skew, it will have a low multiplier. If it is a long skew, it will have a high multiplier. Thus, a qualitative assessment of the median is provided whilst also providing a qualitative assessment of the spread. So from two numbers, the disclosed systems and methods can determine the range endpoints and width. From these values a fourth value can be calculated, which is the average of the high and the low (the 49 WI00096014 2016202814 02 May 2016 standard estimate). This centre value for sub-tasks is aggregated to provide the consolidated estimate for the task.

Additionally, from empirical test evidence illustrated in Figure 19, it has been found that the standard estimate is likely to be slightly higher than the median and the mode. The empirical test evidence of the standard estimate provides a slightly more paranoid estimate, closer to the likely mean probability.

The applicant has found from test data that even for relatively small sample sizes, around 12-24 people, even though there may be high variety in optimism and pessimism between individual estimators, the aggregated consolidated estimate value has been shown to give a reasonable prediction of the task effort outcome. Test evidence indicates that projects that are estimated using the disclosed methods and systems have been shown to be much easier to bring in on time, than projects estimated using a statistical factor method. The problem with the three or four point statistical estimation methods is that people don't have the data to enable an accurate estimate, but the method provides an (unreliable) illusion of accuracy which can, in turn, lead to unrealistic expectations and frustration.

The disclosed estimation methods and systems starts from a presumption of inaccuracy and acknowledgement of subjectiveness of estimates. By asking for estimators to identify a range that they have high confidence of being able to achieve an outcome somewhere within that band, there is no requirement to specify any "most likely" outcome. The range or band is nominated based on either an optimistic or a pessimistic outcome estimate and an integer multiplier - indicative of variability of the work being estimated. Thus, the estimation values are input independent of any particular probability distribution shape or requirement to commit to any particular value within the band.

The width of the band can be indicative of how confident the estimator is to estimate the work. For example, a low value multiplier is a narrow band indicating low variability between the 50 WI00096014 2016202814 02 May 2016 high and low estimates, whereas a sub-task with many unknowns may have a high multiplier value indicating low confidence in the ability to accurately estimate the required effort. By aggregating mid points of the bands for the consolidated estimates this assumes inaccuracies will "even out" in the end. This method has been found to produce much more commercially useable numbers than any other method the applicant has observed. Further the applicant has used retrospective data to identify estimators who are typically pessimistic and those who are typically optimistic and observed that there is a normalising effect. The optimists are happy to consider the low estimate and a "risky factor", whereas the pessimists are happy to consider the high estimate and a "lucky factor".

Further, empirical evidence has shown that network effects are typically accommodated within the estimates, as the estimates are subjective and estimators can instinctively adjust estimates based on anticipated parallel or serial tasks. For example, allowing extra time between optimistic and pessimistic estimates for tasks that are likely to be worked on in parallel or have some dependency.

Examples of estimation methods and systems have been discussed above with reference to project planning, work task scheduling and tracking which typically use time based estimates. Flowever, the described estimation methods and systems may be employed for estimating values other than time, for example monetary values for professional services where deadlines may be fixed but costs dependent on the resources applied to the tasks. For example, planning deadline based services such as legal or contract services. In another example, for example for delivery or logistics planning estimates may be based on anticipated speed and distance, for example for sea or air freight enabling alternative routing and speed based on environmental factors to be accommodated for in a planning process. It should be appreciated that embodiments of the systems and methods described above may be utilised in a variety of planning and tracking systems.

The disclosed systems and methods can be retrofitted into current or existing scheduling systems making those systems more robust. Current or existing systems may have additional 51 WI00096014 2016202814 02 May 2016 parameters providing computations specific to the industry in which they are applied. Utilising the same or different hardware, the disclosed systems and methods can enhance those systems as a retrofit, improving their results.

In a manufacturing environment or plant, where for example a customisation unit assembles products for specific orders, the products need to move through the manufacturing plant for assembly, and preferably do so in the most efficient and productive manner. Customisation is common, for example, in the computer industry where a buyer orders a computer with a customised specification. In preparing for delivery of that computer, the manufacturer will build the computer for the customer. Tasks for customisation may be without an established time history.

Customisation is common in many businesses and industries where a larger task including sub-tasks or different smaller tasks may be required on short notice and where a history of the time taken for one or more of the sub-tasks is not available. While this example is related to a high tech industry, the same would hold true of any industry. One can consider the same discussion for custom furniture building, framing, construction, landscaping, and so on that may have existing scheduling systems specific to their industries.

In a manufacturing environment, while tasks can be repetitive, their initial establishment can require the time estimation as described herein. The disclosed systems and methods, for example, may be used prior to establishing a history. The history may be found to be equal to the time estimate in accordance with the disclosed systems and methods.

Many different tasks may be involved in a manufacturing process. In the example of custom computer building, the task of building a computer to specification, the sub-tasks may include fetching a component, positioning the component into a computer casing, connecting the component to other components, securing the component within the casing, moving the computer to the next assembly station, and so on. For specific customised orders as described immediately above, different combinations of previously time estimated sub-tasks may be required so the 52 WI00096014 2016202814 02 May 2016 aggregation of those time estimations can be provided by the systems and methods described herein. In this way, the disclosed methods and systems can be retrofitted into existing scheduling systems, changing an existing system to having the capability to build in time estimates and the aggregation in the manner that is described herein, and ultimately, providing a robust outcome for time estimation where history is unavailable.

It may be found in some circumstances that the step of establishing a time history is not necessary when a particular scheduling system is retrofitted with the disclosed time estimation systems and methods. The robustness of the disclosed systems and methods may replace the need to monitor time to establish a historical value. A transformation of an existing scheduling system with a retrofit of the disclosed time estimation systems and methods may eliminate the need for the step of establishing a time history and replacing that with the estimated time based upon the retrofitted system.

While the discussion above is with respect to time estimates and utilisation of acceptability bands with respect to time estimates, it will be understood that a product with an acceptability band sub-product is applicable to different types of components or sub-tasks of tasks. A component may be considered a small part of an overall project where a plurality of components are pieced together in series and/or parallel to carry out a task. Potentially a component can be a subtask as described with respect to the x-factor product. Components can be named in any suitable manner. Another term, used below is "filter strips". This is a term that is used in the drop down menu of FIGs. 14a. Any term that is criteria that is used to interrogate the database is a filter strip. Acceptability bands use criteria and therefore the term filter strip is used in conjunction with acceptability bands as well. FIG. 20 depicts some configured acceptability bands that are utilised in a workflow management system as they established with respect to different components. While some of the above discussion has been with regard to time estimates, as is shown, different acceptability bands can be applied to one component which may not refer to time estimates. For example, the 53 WI00096014 2016202814 02 May 2016 component entitled "waiting on capacity", there are at least ten different acceptability bands configured for that component. Also, in the column called "type" the different types can configure how the output is presented. For example, NUP provides the number of matching workflows as a percent of the total number of workflows in the component; AGR provides aggregate workflows returned by filter strips using SQL; NUM provides number of matching workflows; REL provides the time between now and when an item matching the filter strips was most recently released to the buffer; DUP provides the planned duration of all matching workflows as a percent of planned duration of all workflows in the components; QAG provides the time between now and when the earliest open, un-released item was added to a work queue, SQL provide SQL; and DUR provides the total planned duration of all open items matching the filter strips. The list of FIG. 20 is may be a partial list of configured acceptability bands and does not limit the scope of this discussion. FIG. 21 depicts workflow items subjected to one or more acceptability bands. FIG. 21 further depicts that one or more acceptability bands can be applied at various stages of the workflow management system. The icons are used to represent workflow items that have time estimates associated with them and potentially other workflow variants. As illustrated in FIG. 20, a component may have more than one acceptability band associated with it as discussed above. One or more processes involving acceptability bands as described above may be applied to workflow items before the workflow items are assigned to resources. FIG. 22 depicts a screen shot of a workflow buffer to give context to the output that may be provided by the utilisation of acceptability bands. As mentioned, acceptability bands can be running in the background on components that do not necessarily generate workflow items that move to the workflow buffer. However, in the context of acceptability bands that do provide workflow items to the workflow buffer, FIG. 22 depicts examples of workflow items delivered to resources. 54 WI00096014 2016202814 02 May 2016

It will be understood to persons skilled in the art that many modifications may be made without departing from the spirit and scope of the invention. It should be appreciated that in the context of the present specification the terms "module" and "component" are used to refer to a system component implementing defined functionality. The system component can be implemented as a software program executable using a processor or as a software routine integrated into a software program providing additional functionality. The system component may be equally implemented using a combination of hardware and software. The disclosed embodiments may be implemented using any suitable combination of hardware, software and firmware, and may utilise a combination of shared and dedicated data processing hardware and memory resources. For example, a dedicated hardware circuit, such as an application specific integrated circuit (ASIC) for programmable logic such as a field programmable gate array (FPGA) or programmable logic controller (PLC), may be used to implement some system functionality. This hardware circuit may be used in a data processing system having at least one processor, memory and other resources for executing cooperating firmware and software to support the full functionality of the system and integrate with external systems such as user terminals. It should be appreciated that many alternative system architectures could be used to implement the system and all such alternatives are envisaged within the scope of the present application. 55

Claims (17)

1. Claims:

   1. A computer implemented method of a task release gate in a task scheduling system, the task release gate for determining which tasks to move to a task buffer and for moving the tasks to a task buffer, the computer including a CPU, comprising: receiving task data; qualitatively assessing the received task data according to first qualitative criteria, the task data qualitative assessment to determine a distribution mix of tasks to resources; monitoring CPU usage or elapsed time to determine whether the qualitative assessment is utilising CPU usage above a predetermined threshold; when CPU usage or elapsed time exceeds the predetermined threshold, changing the qualitative assessment from the first qualitative criteria to a second qualitative criteria; ceasing qualitatively assessing according to the first qualitative criteria to then qualitatively assessing the received task data according to the second qualitative criteria to generate a distribution mix of tasks to the resources to distribute the tasks to the resources within a preset CPU or elapsed time limit; and delivering to a task buffer a mix of tasks to the resources.
2. 2. The method of claim 1 wherein the qualitative assessment comprises subjecting the task data to acceptability bands including a range of values.
3. 3. The method of claim 2 wherein the first qualitative criteria is a first range of values and the second qualitative criteria is a second range of values.
4. 4. The method of claim 3 further comprising third qualitative criteria of a third range of values.
5. 5. The method of claim 3 wherein upon ceasing qualitatively assessing according to the first qualitative criteria of a first range of values, the second qualitative criteria is a second range of values.
6. 6. The method of claim 1 wherein the task data includes time estimates which are is established by an x-factor process.
7. 7. The method of claim 6 wherein the x-factor process comprises a quantitative time estimation method of generating a consolidated quantitative time estimate for a task comprising a plurality of defined sub-tasks, the method comprising the steps of: for each one of the plurality of defined sub-tasks of a task: outputting for display on the display a sub-task identifier an indicator of data input fields for input of a first time estimate value being in a unit of time and an x-factor value, the fields for receiving input of a responsible person of a first time estimate value determination and an x-factor value determination by the responsible person, the first time estimate value being a quantitative value indicative of one end of a time estimate range, the first time estimate value being a value that the person responsible for the user input has a particular level of confidence in being an end point of a range for time estimation purposes and the x-factor value being an integer multiplier value of the first time estimate value representing the degree of the particular level of confidence that the responsible person has in the first time estimate value; receiving instructions by the estimation processor via user input to the display fields for input of a first time estimate value and an x-factor value, the instructions representative of a user data input of the responsible person indicating the first time estimate value, and the x-factor value, wherein the first time estimate value is the low end time value and wherein the multiplier value is used to determine a high end time value being in a unit of time by multiplying the first time estimate value by the multiplier value; and in response to receiving the first time estimate value and the multiplier value, the estimation processor determines, using the received first time estimate value and the multiplier value for the sub-task, a low end time value, a high end time value, a range width value being in a unit of time and an associated centre value of the range value; and in response to having received the first time estimate value and multiplier value for all subtasks of the task: aggregating by the estimation processor the associated centre value for each subtask of a task to determine the consolidated quantitative time estimate being in a unit of time for the task, without an assumption of a particular statistical distribution; outputting the consolidated quantitative time estimate being a standard estimate which is not an estimate of mean, median or mode for the task for task scheduling by the task scheduling system; and incorporating the consolidated quantitative time estimate for the task into a resource scheduling tool of the task scheduling system that is processing a plurality of tasks for work flow management output to generate project estimates.
8. 8. The method of claim 1 wherein in determining a distribution mix of tasks to resources further comprises determining capabilities of available resources and matching task types to capabilities types.
9. 9. A sub-system of a task release gate in a task scheduling system, the task release gate for determining which tasks to move to a task buffer and for moving the tasks to a task buffer, the computer including a CPU, the sub-system comprising: a processor configured for receiving task data; the processor configured to qualitatively assessing the received task data according to first qualitative criteria, the task data qualitative assessment to determine a distribution mix of tasks to resources; the processor configured for monitoring CPU usage or elapsed time to determine whether the qualitative assessment is utilising CPU usage above a predetermined threshold; when CPU usage or elapsed time exceeds the predetermined threshold, the processor configured for changing the qualitative assessment from the first qualitative criteria to a second qualitative criteria; the processor configured for ceasing qualitatively assessing according to the first qualitative criteria to then qualitatively assessing the received task data according to the second qualitative criteria to generate a distribution mix of tasks to the resources to distribute the tasks to the resources within a preset CPU or elapsed time limit; and the processor configured for delivering to a task buffer a mix of tasks to the resources,
10. 10. The sub-system of claim 9 wherein the qualitative assessment comprises the processor being configured for subjecting the task data to acceptability bands including a range of values.
11. 11. The sub-system of claim 9 wherein the first qualitative criteria is a first range of values and the second qualitative criteria is a second range of values.
12. 12. The sub-system of claim 1 further comprising third qualitative criteria of a third range of values.
13. 13. The sub-system of claim 9 wherein the task data includes time estimates which are is established by an x-factor process.
14. 14. A computer implemented method of a task release gate in a task scheduling system, the task release gate for determining which tasks to move to a task buffer, the computer including a CPU, comprising: a) processing task data through a filter of an acceptability band having a particular qualitative threshold; b) monitoring CPU usage or elapsed time to determine whether the processing step is utilising CPU usage above a predetermined CPU threshold; c) when CPU usage or elapsed time exceeds the predetermined CPU threshold, changing the particular qualitative threshold to another qualitative threshold; and d) repeating steps a-c until there is a distribution mix of tasks to the resources to distribute the tasks to resources within a preset CPU or elapsed time limit
15. 15. The method of claim 14 further comprising delivering to a task buffer a mix of tasks to the resources.

    15. The method of claim 14 wherein the task data includes time estimates which are is established by an x-factor process.
16. 17. The method of claim 16 wherein the x-factor process comprises a quantitative time estimation method of generating a consolidated quantitative time estimate for a task comprising a plurality of defined sub-tasks, the method comprising the steps of: for each one of the plurality of defined sub-tasks of a task: outputting for display on the display a sub-task identifier an indicator of data input fields for input of a first time estimate value being in a unit of time and an x-factor value, the fields for receiving input of a responsible person of a first time estimate value determination and an x-factor value determination by the responsible person, the first time estimate value being a quantitative value indicative of one end of a time estimate range, the first time estimate value being a value that the person responsible for the user input has a particular level of confidence in being an end point of a range for time estimation purposes and the x-factor value being an integer multiplier value of the first time estimate value representing the degree of the particular level of confidence that the responsible person has in the first time estimate value; receiving instructions by the estimation processor via user input to the display fields for input of a first time estimate value and an x-factor value, the instructions representative of a user data input of the responsible person indicating the first time estimate value, and the x-factor value, wherein the first time estimate value is the low end time value and wherein the multiplier value is used to determine a high end time value being in a unit of time by multiplying the first time estimate value by the multiplier value; and in response to receiving the first time estimate value and the multiplier value, the estimation processor determines, using the received first time estimate value and the multiplier value for the sub-task, a low end time value, a high end time value, a range width value being in a unit of time and an associated centre value of the range value; and in response to having received the first time estimate value and multiplier value for all subtasks of the task: aggregating by the estimation processor the associated centre value for each subtask of a task to determine the consolidated quantitative time estimate being in a unit of time for the task, without an assumption of a particular statistical distribution; outputting the consolidated quantitative time estimate being a standard estimate which is not an estimate of mean, median or mode for the task for task scheduling by the task scheduling system; and incorporating the consolidated quantitative time estimate for the task into a resource scheduling tool of the task scheduling system that is processing a plurality of tasks for work flow management output to generate project estimates.
17. 18. The method of claim 19 wherein in determining a distribution mix of tasks to resources further comprises determining capabilities of available resources and matching task types to capabilities types.