EP4778006A1 - Alternate potency targeting system and method - Google Patents

Abstract

The present embodiments relate to alternate potency targeting manufacturing. Subject matter of the present embodiments provides computer-implemented methods, computer systems and computer-readable storage media for predicting the results of alternate potency targeting manufacturing methods.

Description

Alternate Potency Targeting System and Method Field Systems, methods, and computer programs disclosed herein relate to biopharmaceutical manufacturing. Cross Reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63/537,583, filed September 11, 2023, the disclosure of which is incorporated by reference in its entirety. Background Manufacturing of biologics requires a series of complex processes that broadly fall into categories of drug substance (DS) manufacturing and drug product (DP) manufacturing. DS manufacturing is characterized by the critical unit-operations of cell culture and purification, while DP manufacturing encompasses the key steps involved with sterile fill- finish. The sterile fill-finish unit-operation involves combining DS with excipients, through validated processes including bulking, filling, freeze drying, capping, inspection, and packaging. More specifically, bags of frozen drug substance are thawed, pooled, and partially diluted with formulation buffer. Next, partially diluted bulk is tested for potency, sterile filtered, and further diluted with formulation buffer to a pre-defined target potency concentration (also known as the bulk potency target). Subsequently, the sterile bulk is filled into glass vials, freeze-dried, and capped to produce the drug product (DP) vials. During the inspection step, DP vials are tested by Quality Control (QC) per standard operating procedures (SOP) to ensure that all critical quality attributes (CQAs) are within specification limits, a requirement for commercial release. The specification limits of various DP CQAs are regulated by the Regulatory health agencies, to ensure safety and effectiveness of the manufactured DP. Generally, individual vials are tested to be within specification limits, however, mean internal action limits can be implemented in certain cases. DP batches with CQAs falling outside of validated ranges are considered ineligible for commercial release. Thus, a continued process verification (CPV) program is implemented to provide continual assurance that manufacturing processes remain in a state of control (the validated state) during commercial manufacturing, demonstrate understanding of process variability at commercial-scale, and drive process changes. A major CQA of interest is the potency of DP vials, which is monitored by the CPV program and known to be controlled by the bulk potency target concentration set during the bulking stage of DP manufacturing. Intuitively, increasing or decreasing the bulk potency target will cause a proportional shift in the DP vial’s potency distribution mean and impact the portion of DP batches that fall out of specification (OOS) at release. Because optimization of process is desirable, refinement of the bulk potency target presents an opportunity to improve process capability (robustness), by reducing OOS probability (risk) at release. Established (current) bulk potency targets of a DP manufacturing process are set for each fill size (dosage strength) of the DP. Changes to the current bulk potency target of a DP are generally made based on manufacturing experience (subject matter expertise) to ensure business and quality needs are being met. Historically, prior to making bulk potency target adjustments, statistical assessment of theoretical (alternate) potency targets is conducted, involving estimation of the risk of OOS at release for each alternate potency target of interest. While commercial software exists to carry out these analyses, conducting the analyses in an ad-hoc manner is often time-consuming due to the tedious manual steps involved in the workflow. The full workflow required to assess alternate potency targets entails a series of manual steps involving data acquisition, data-preprocessing, and statistical calculations. Particularly when OOS events require decisions regarding good manufacturing practices (GMP), having an automated statistical analysis workflow is desirable to facilitate timely response. In addition to limiting the efficiency of the overall workflow, commercial software is also constrained regarding customizability and accessibility. Barring major updates to the software, commercial software is generally fixed in their functionality and user-interface (UI), which cannot meet evolving user needs. Further, traditional commercial software requires installation on the user’s local operating system and requires local computing resources, potentially limiting future accessibility and scalability across the organization. This contrasts with software based on a cloud platform, which relies on a network of remote servers to provide computing resources (i.e., storage, processing power, etc.). There is a need to automate the evaluation of alternate potency targets and methods. Development of a system would be used to facilitate efficient statistical assessment of alternate potency targets, and have potential to prevent future deviations, product loss, and associated investigation efforts. Deployment of any system facilitates accessibility and scalability for end users by removing the overhead associated with managing their local computing environment (i.e., installation of programming languages, and software packages). In summary, there is a need for systems and methods for implementation of potency targeting systems, which can be used to assist manufacturing process scientists with efficient assessment of alternate potency targets for DP in biologics manufacturing. Further, there is a need for software systems which deploy quickly, effectively, and efficiently to accomplish such results. These and other issues are addressed by the present embodiments. Summary In a first aspect, the embodiments provide a computer implemented method, comprising: providing a front-end dashboard comprising a user input for inputting user input parameters into a data acquisition module to determine alternate potency targets, and an output for displaying processed results; providing a backend for: pre-processing data from the data acquisition module including normalization; applying statistical modelling calculations using out of specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the alternate potency target, and the output displays the respective OOS risks. The embodiments provide a computer implemented method and system, wherein the user input parameters comprise drug product, fill size, batch selection, and manufacturing date range. The embodiments provide a computer implemented method and system, wherein the data acquisition module further comprises source data systems. The embodiments provide a computer implemented method and system, wherein the data acquisition module further comprises a data acquisition system. The embodiments provide a computer implemented method and system, wherein the data acquisition module further comprises cloud-based data storage. The embodiments provide a computer implemented method and system, wherein the data acquisition module further comprises cloud-based computing. The embodiments provide a computer implemented method and system, wherein the pre-processing step further comprises normalizing the data by statistical analysis. The embodiments provide a computer implemented method, wherein the data is normalized by calculating P normalized, according to the following equation: P _୪୧^^^ = ^{P^்} ^_୭୰୫ୟ P ∗ P _ெ where: P = Potency Values, PCT = Current Bulk Potency Target, and P_TM = Bulk Potency Target at Time of Manufacture. The embodiments provide a computer implemented method, wherein the OOS risk selection criteria for a candidate alternate potency target can be determined from a model where: 1) ^^ ^^ ^^_{^^^^ ^^^^}: % OOS risk of at least one individual replicate result failing upper release limit given that the calculated % CV (Coefficient of variation) is within the user defined limit; 2) ^^ ^^ ^^_{^^^^ ^^௪}: % OOS risk of at least one individual replicate result failing lower release limit given that the calculated % CV is within the user defined limit; 3) ^^ ^^ ^^_{ெ^^^ ^^௪}: % OOS risk of average result failing lower mean release limit given that the calculated % CV is within the user defined limit; and 4) ^^ ^^ ^^_ை௩^^^^^: % OOS risk of failing in at least one of the above 1-3 scenarios. The embodiments provide a computer implemented method, wherein the data acquisition module further comprises a cloud-based relational database. The embodiments provide a computer implemented method, wherein a data query can be made between the user input and the cloud-based relational database. In another aspect, the embodiments provide a computer implemented system, comprising a front-end dashboard comprising a user input for inputting user input parameters into a data acquisition module to determine alternate potency targets, and an output for displaying processed results; a backend for pre-processing data from the data acquisition module including normalization; applying statistical modelling calculations using out of specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the system determines the alternate potency target, and the output displays the respective OOS risks. The embodiments provide a computer implemented system, wherein the user input parameters comprise drug product, fill size, batch selection, and manufacturing date range. The embodiments provide a non-transitory computer readable storage medium having stored thereon software instructions that, when executed by a processor of a computer system, cause the computer system to execute the following method: allowing a user to input user input parameters into a data acquisition module to determine alternate potency targets, and an output for displaying processed results; pre-processing data from the data acquisition module including normalization; and applying statistical modelling calculations using out of specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the alternate potency target, and the output displays the respective OOS risks. These and other features of the present teachings are set forth herein.

Brief Description of the Drawings The skilled artisan will understand that the drawings, described below, are for illustration purposes only. The drawings are not intended to limit the scope of the present teachings or claims in any way. FIG.1 shows an embodiment of the computer system pipeline architecture for the Potency Targeting System in a cloud-based environment. Major components of the Data pipeline comprise source data systems, data integration system, cloud data storage, and cloud computing (Domino Data Lab). platform. The source data systems are illustrative and serve as examples for what some data science systems can use. FIG.2 shows an embodiment of the Potency Targeting System architecture. FIG.3 shows a flow-chart for statistical calculation workflow, including confidence interval estimation. User-input including drug product, fill size, and manufacturing date range define the relevant potency data used for analysis. Information on historical bulk potency target adjustment is used to normalize the raw potency results data to the current bulk manufacturing potency target. Potency data is then simulated at the alternate potency target(s) of interest and used for OOS risk estimation at release. FIG.4 shows the data scope section of the Potency Targeting System UI, Drug A is selected to be acquired from the online data source Discoverant, https://www.3ds.com/products-services/biovia/products/manufacturing-analytics/biovia- discoverant/ (See also Discoverant User’s Guide. Aegis Analytical Corporation, 2001). The dashboard allows users to use a combination of release potency results from Discoverant and offline data sources (e.g., .csv or .xlsx files). In this example, the online data source “Discoverant” has been used, when the offline data source is not uploaded. FIG.5 shows the Data Filter section of the Potency Targeting System, a fill size of Fill Size #1 is selected, between the manufacturing start dates of 01/01/2022 and manufacturing end date of 04/01/2023. As shown by the select batch drop down, a total of 17 DP batches are available and selected for further analysis. FIG.6 shows the Data Normalization section of the Potency Targeting System being used to normalize the potency results at release for Drug A of Fill Size #1. Potency results at release for all available data are normalized to the current bulk potency target of “118” IU/mL FIG.7 shows the OOS Risk Estimation (Simulation) Parameter Settings section of the Potency Targeting System. Parameter settings and specification limits are selected for illustrative purposes. The specification limits for Drug A Fill Size #1 were A, B, and C respectively for vial low, vial high, and mean low. Simulation parameters were selected based on user guide recommendations. FIG.8 shows Potency Targeting System’s bar chart of different types of release OOS risk estimates for each of the different alternate potency targets. The error bar represents 3 standard deviation (SD) limits, while the height of the bar is the median of the OOS risk estimates. OOS risk estimates are provided for vial low, vial high, mean low, and overall. FIG.9 shows OOS Risk Estimation (Simulation) Parameter Settings section of the Potency Targeting System, with a specified assay uncertainty. The specification limits and simulation parameters remain unchanged for Fill Size #1 of Drug A (compared to FIG.7). FIG.10 shows a Bar chart of different types of release OOS risk estimates for each of the different alternate potency targets (based upon a specified assay uncertainty). The error bar represents 3 SD limits, while the height of the bar is the median of the OOS risk estimates. OOS risk estimates are provided for vial low, vial high, mean low, and overall. FIG.11 is a block diagram of an exemplary computer system of the present disclosure suitable for determining one or more alternate potency results for an alternate potency method based on at least two alternate potency process parameters.

Detailed Description The embodiments will be more particularly elucidated below without distinguishing between the aspects of the embodiments (method, computer system, computer-readable storage medium). On the contrary, the following elucidations are intended to apply analogously to all the aspects of the embodiments, irrespective of in which context (method, computer system, computer-readable storage medium) they occur. If steps are stated in an order in the present description or in the claims, this does not necessarily mean that the embodiment is restricted to the stated order. On the contrary, it is conceivable that the steps can be executed in a different order or else in parallel to one another, unless one step builds upon another step, this absolutely requiring that the building step be executed subsequently (this being, however, clear in the individual case). The stated orders are thus embodiments of the present disclosure. As used herein, the articles “a” and “an” are intended to include one or more items and can be used interchangeably with “one or more” and “at least one.” As used in the specification and the claims, the singular form of “a”, “an”, and “the” include plural referents, unless the context clearly dictates otherwise. Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has”, “have”, “having”, or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise. Further, the phrase “based on” can mean “in response to” and be indicative of a condition for automatically triggering a specified operation of an electronic device (e.g., a controller, a processor, a computing device, etc.) as appropriately referred to herein. The operations in accordance with the teachings herein can be performed by at least one computer system specially constructed for the desired purposes or general-purpose computer system specially configured for the desired purpose by at least one computer program stored in a typically non-transitory computer readable storage medium. A “computer system” is a system for electronic data processing that processes data by means of programmable calculation rules. Such a system usually comprises a “computer”, that unit which comprises a processor for carrying out logical operations, and peripherals. In computer technology, “peripherals” refer to all devices which are connected to the computer and serve for the control of the computer and/or as input and output devices. Examples thereof are monitor (screen), printer, scanner, mouse, keyboard, drives, camera, microphone, loudspeaker, etc. Internal ports and expansion cards are, too, considered to be peripherals in computer technology. Computer systems of today are frequently divided into desktop PCs, portable PCs, laptops, notebooks, netbooks and tablet PCs and so-called handhelds (e.g., smartphone); all these systems can be utilized for carrying out the embodiments. The term “non-transitory” is used herein to exclude transitory, propagating signals or waves, but to otherwise comprise any volatile or non-volatile computer memory technology suitable to the system. The term “computer” is intended to be broadly construed to cover any kind of electronic device with data processing capabilities, including, by way of non-limiting example, personal computers, servers, embedded cores, computing system, communication devices, processors (e.g., digital signal processor (DSP)), microcontrollers, field programmable gate array (FPGA), system specific integrated circuit (ASIC), etc.) and other electronic computing devices. The term “process” as used above is intended to comprise any type of computation or manipulation or transformation of data represented as physical, e.g., electronic, phenomena which can occur or reside e.g., within registers and/or memories of at least one computer or processor. The term processor comprises a single processing unit or a plurality of distributed or remote such units. By “computer readable medium” is meant any component capable of containing, storing, communicating, propagating, or transporting one or more of executable code or data. By “computer readable storage medium” is meant any computer readable medium capable of storing data, e.g., executable code. By “executable code” is meant a set of instructions in any language capable of being used by or in connection with a physical computing device, such as a computer. By “display device” is meant any component capable of digitally, electronically, or visually displaying a result. By “memory” is meant any computer readable medium capable of storing data and/or executable code. By “network” is meant two or more physical computing devices connected through any type of communications line or communications protocol. By “output” is meant to transfer, transmit, or display one or more of information, data, executable code, or any result from executing an executable code. By “processor” is meant a component capable of performing a method encoded by an executable code. By “receive” is meant to accept, retrieve, or obtain one or more of information, data, executable code, or any result from executing an executable code. By “server” is meant a device or machine that serves a network. By “data” is meant any information in any language capable of being used by or in connection with a physical computing device, such as a computer. “Digital” means that the weed map may be processed by a machine such as a computer system. Some implementations of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all implementations of the disclosure are shown. Indeed, various implementations of the disclosure can be embodied in many different forms and is not to be construed as limited to the implementations set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Biopharmaceutical manufacturing entails a series of highly regulated steps. At various stages throughout drug product (DP) manufacturing, various critical quality attributes (CQAs) are tested against regulatory specification criteria. DP potency concentration, which measures the dosage strength of a particular DP, is a CQA of great interest, varying for each DP fill size based upon its bulk potency target. For this reason, implementation of changes to the bulk potency target of a DP, requires evaluation of the theoretical (alternate) potency target for the probability (risk) of being out of specification (OOS). Data As established by many manufacturing standard operating procedures (SOP), manufactured DP are subjected to multiple assays at many QC analytical laboratories for testing of various CQAs before commercial release. As illustrated by FIG.1, before commercial release, individual vial’s potency results (along with other CQAs) are evaluated and stored in an on-premises database. Data Source and Pipeline As shown by FIG.1, some internal data systems comprise several source data systems, a data integration system, and cloud data storage. The individual vial potency results for DP batches at release and relevant meta-data (e.g., bulk-manufacturing date, fill size, product, etc.), are initially captured through various source data systems. The data from various source data systems are made available using the BIOVIA Discoverant data integration system. As established by product manufacturing standard operating procedures (SOP), manufactured DP batches are subjected to multiple assays at QC analytical laboratories for testing of various CQAs before commercial release. The assay results are stored in Laboratory Information Management System (LIMS) database and batch information is stored in Enterprise Resource Planning (ERP) system database. The source data from LIMS and ERP databases are ingested into cloud-based storage for consumption within the software application. Amazon Web Services (AWS) was used to provide the cloud storage. Specifically, PostgreSQL relational database (AWS-RDS) was used to store data for easy access by the application. It is within the scope of the embodiments that other data integration systems can be used with the present embodiments. For instance, other data integration systems comprise ETQ Reliance QMS, Arena PLM & QMS, Greenlight Guru, Teamcenter, Qualio, Ideagen Quality Management, MasterControl Quality Management System, G2 Deals, Virgin Pulse, Propel, and Orcanos. Other data integration systems known or used in the art can be employed with the present embodiments. Analysis Groups (AGs) are defined in Discoverant to regularly query the various source data systems, for the most current and available data, and contextualize them based upon batch number. Amazon Web Services (AWS) provides multiple cloud-computing services that can be leveraged to support cloud deployment of systems. External Data Access is a feature of Discoverant that enables Discoverant AG data to be retrieved and used by third-party systems. An extract-transform-load (ETL) pipeline (e.g., AWS Glue) was developed to routinely extract historical process data from Discoverant, transform them into an appropriate data structure (e.g., a spreadsheet), and load them into a cloud storage system. For the Potency Targeting System, a PostgreSQL relational database (AWS-RDS) was used to store data for easy access by the system. It is within the scope of the embodiments that other open-source relational databases besides PostgreSQL can be employed which connect with a wide range of apps and services. For instance, other embodiments can comprise and not be limited to Microsoft SQL Server, MSQL, Oracle Database, MongoDB, Apache Cassandra, MySQL, Redis, DynamoDB, IBM Cloudant, Orient DB, Jaguar DB, CouchDB, Firebase, ArangoDB, CockroachDB, RethinkDB, ScaleGrid, RavenDB, Intersystems IRIS, Knack, MariaDB, and Airtable. Other relational databases known or used in the art can be employed with the present embodiments. System Architecture The Potency Targeting System comprises several modules that work in tandem, ranging from data acquisition, pre-processing, statistical calculations, front-end visualization, and reporting. An overview of these modules is shown in FIG.2. The front-end is an interactive dashboard that allows the user to specify the drug product potency data to query from AWS-RDS and use for evaluation of alternate potency targets. The data is then pre-processed by the backend Python library, prior to performing statistical calculations based on simulation parameters specified by the user. Output is provided in the form of graphs and tables shown on the user-interface (UI) of the dashboard and captured in an automatically generated PDF report. Throughout the development cycle, software engineering best practices of object-oriented programming, version control (GitLab), and in-line code documentation were adopted across the entire python codebase (back-end and front-end) to facilitate efficient collaboration amongst developers working on the potency targeting library. It is within the scope of the embodiments that other alternatives to GitLab can be employed with the present embodiments. Version Control Hosting Software is a widely used technology for users searching for simple software solutions with customization. Other solutions can comprise software which comprises project management and further software development. Still other options to GitLab can comprise and not be limited to GitHub, CloudBees, G2 Deals, CircleCl, Jenkins, Red Hat Ansible Automation Platform. Azure Pipelines, Bitbucket, Azure DevOps Server, Copado Cl/CD, and Jira. It is within the scope of the embodiments that other codebase or libraries can be employed with the present embodiments. For instance, Python CodeBase Tools provide the capabilities of creating, opening, reading, and indexing data and tables in a format for use with many popular software products. Other codebase tools comprise Java, NodeJS, PHP, Ruby, Golang and Scala. Other embodiments known in the art can be employed with the present embodiments. Still further, other libraries can be employed with the present embodiments comprising Tkinter/pyQt, Pandas/Numpy, Tensorflow/Pytorch, Matplotlib/Bokeh, and Numpy/(are you kidding me). It is within the scope of the embodiments that other languages or codes can be employed with the present embodiments. For instance, instead of Python, C++, C#, or R can be employed with the present embodiments. Further, other languages or codes can be employed which are known in the art. Below, the system back-end development process is discussed. Subsequently, the front-end UI (dashboard) development and overall verification process are detailed in other sections, respectively. System Back-end (Python Library) Development The back end was developed as a statistical software library in the python programming language to provide the requisite functionality of the front-end dashboard, while neatly modularizing python code for future reusability. As shown by FIG.2, the steps required for potency target evaluation entail data acquisition, data pre-processing, and statistical calculations. Each of these steps were developed as part of separate code modules that leveraged other open-source python libraries to facilitate development. Other sections detail the approaches taken for data pre-processing and statistical calculation, respectively. Data Pre-Processing For each dosage strength (file size), the DP batch mean potency value is calculated as the overall historical mean of the normalized potency measurements. Potency data is normalized prior to using it for statistical analysis. For the fill size of interest, if the dataset contains final container potency results with different bulk potency targets, the final container results for batches manufactured using a previous potency target are normalized to the current potency target. For example, the final container potency results (P) are divided by the bulk potency target at the time of manufacture and then multiplied by the current bulk potency target to calculate the normalized final container potency results (P_Normalized): P ^{P ^்} ^_{୭୰୫ୟ୪୧^^^} ^{= P ∗} P _ெ ⁽¹⁾ Where: P = Potency Values, P_CT = Current Bulk Potency Target, and PTM = Bulk Potency Target at Time of Manufacture. Based on Equation 1, if there has not been a bulk potency target change for a given fill size of a DP, then the normalized potency result (P_Normalized) will be equal to the raw potency result (P). Statistical Calculations and Modeling Approach The probability of DP potency results exceeding acceptance limits is influenced by several factors. The Bulk potency target affects the location of the DP potency distribution mean relative to the release limits. At a fixed bulk potency target, the mean potency result for a DP batch can be above or below the distribution mean due to process factors that cause batch-to-batch variation, as well as potential lab-to-lab testing differences. In addition, process fill weight variation and test method precision contribute to variation among the individual vial potency results during batch release testing. Analytically determining the joint probability distribution for this situation is challenging. Hence, a Monte Carlo simulation was used to combine the intra- and inter-batch probability distributions and numerically estimate the overall risk of having DP potency results exceed release limits for the alternate bulk potency target options. It is within the scope of the embodiments that other simulation methods and software can be employed with the present embodiments. For instance, other simulation alternatives to Monte Carlo simulation can be employed which are known in the art. The end-goal of this statistical calculation module is to calculate OOS risk for a candidate bulk potency target. The following types of OOS risks can be numerically calculated: 1) ^^ ^^ ^^_{^^^^ ^^^^}: % OOS risk of at least one individual replicate result failing upper release limit given that the calculated % CV is within the user defined limit; 2) ^^ ^^ ^^_{^^^^ ^^௪}: % OOS risk of at least one individual replicate result failing lower release limit given that the calculated % CV is within the user defined limit; 3) ^^ ^^ ^^_{ெ^^^ ^^௪}: % OOS risk of average result failing lower mean release limit given that the calculated % CV is within the user defined limit; and 4) ^^ ^^ ^^_ை௩^^^^^: % OOS risk of failing in at least one of the above scenarios. The above types of OOS risks have by way of example produced the best results for candidate potency targeting. For instance, the ^^ ^^ ^^_{^^^^ ^^^^}, ^^ ^^ ^^_{^^^^ ^^௪} , ^^ ^^ ^^_{ெ^^^ ^^௪}, and ^^ ^^ ^^_ை௩^^^^^, cover each of the possible scenarios where bulk potency targets can fall within a given limit. Both the precision and accuracy of such predictions are contingent upon the data set available and employed in the calculations. Obviously. It is preferable to have a larger data set. However, in certain instances this may not be possible. For instance, the dataset comprises only few batches outside specifications which makes the reliable estimation of OOS risk challenging to provide adequate accuracy and precision. It would, therefore, be counter intuitive to employ the above module for calculating or predicting a bulk potency target. The power of the present method and system lies in the ability to provide both accurate and precise results using the above-described module with little raw or actual data or input provided regarding a bulk potency target. Both software and hardware can be employed to simulate accurate and precise data sets. This can produce fast and cheaper results for users. Further, by way of example the above module can also be employed where there are significant actual historical data sets available. For a given type of OOS risk, the statistical analysis of DP potency release data conducted as part of the Monte Carlo simulation are detailed in the following section. Monte Carlo Simulation The size of the dataset (number of batches) is not large enough for numerical calculation of OOS risk. Therefore, many synthetic potency data points are generated using the Monte Carlo simulation method. As a first step, sample statistics are estimated from the normalized historical potency results at release to characterize the DP potency distribution mean and factors contributing to variability in potency results. An estimate of the DP potency distribution mean ( ^^ത^_^^௧^^^௬) can be estimated as the overall mean of all potency results at release, across the DP batches. While variance component analysis (VCA) is the standard statistical method to breakdown the total variability into variabilities explained by different factors. More specifically, VCA is applied on the potency dataset to determine the percentages of batch-to-batch variability (i.e., process variability), lab-to-lab variability and within batch or assay variability ( ^^_^ ^ଶ ^_௧^^ , ^^_^ ^ଶ ^_^ , ^^_^ ^ଶ ^_^^௬ ). For specified alternate potency target ^^^{௧^} batch mean potency result ( ^ഥ^_^) and ௧^{^} ^^ individual replicates result ( ^^_^^) are generated using the Normal distributions described by equations (2) and (3), respectively. ^_ഥ^ ^{^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^^} ^ _{= ^^( ^} ^ത _^ ^{^௧} ^_{^௧^^^௬ ×} ⁽²⁾ ^_{^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ , ^^^} ^ଶ ^_{௧^^ + ^^^} ^ଶ ^_{^ ) ^௨^^} ⁽³⁾ Where: ^^ത _{^^^௧^^^௬ is the estimate of the DP potency distribution mean,} ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^_^^௧ and ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^_^௨^^ are alternate and current bulk potency target, respectively, and ^^_^ ^ଶ ^_௧^^ , ^^_^ ^ଶ ^_^ and ^^_^ ^ଶ ^_^^௬ are estimates of batch-to-batch variability, lab-to-lab variability, (assay variability), respectively. In total, ^^_^^^ simulated batches with ^^_^^^ number of replicate potency values per batch are generated using the Equations (2) and (3). The number of replicates ^^_^^^ is determined based upon the manufacturing specifications of the drug product and fill size. Refer below for discussion on the appropriate number of simulated batches ^^_^^^. For each simulated batch, the coefficient of variation (CV) is calculated using the ^^_^^^ individual potency values for ^^^{௧^} simulated batch as ^^ ^^_^ and compared to the test method sample suitability criteria for %CV ( ^^ ^^_^^^^௧). If results for the batch comply with the suitability criteria ^^ ^^_^^^^௧, the simulated batch is considered valid, otherwise it is classified as invalid. Finally, for each valid simulated batch, a determination of OOS or not (1 or 0) is generated per OOS risk definition. The results from all the simulated batches are averaged to calculate the respective OOS risks. For example: ^^ ^^ ^^_{^^^^ ^^௪} = ^{^} ∑^{ேೞ^^} ^_ୀ^ ^^(min ( ^^ _^^, … , ^^ _^^^^^) − ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^) (4) ^_{^, … , ௧ batch,} containing ^^_^^^ replicates. The number of DP batches ^^_^^௧^^ simulated at the alternate potency targets of interest is determined based upon convergence of the estimated OOS risk at release. Convergence is determined by a convergence criterion ∆_^^^௩^^^^, where the difference in estimated OOS risk at release does not exceed ∆_^^^௩^^^^, between an adjustment in ^^_^^௧^^. Convergence is assessed iteratively, and the specific value of ∆_^^^௩^^^^ is determined through discussion with process experts, regarding what a meaningful difference of risk would be and how it would impact decision making. Convergence can be assessed iteratively, and the specific value of ∆_converge can be determined through discussion with process experts, regarding what a meaningful difference of risk would be and how it would impact decision- making. Quantifying Sampling Uncertainty To estimate sampling uncertainty of the point-estimate of OOS risk, a confidence interval is constructed using a bootstrap approach. The same procedure described earlier in the modeling approach is repeated ^^_^^^௧ times using the ^^_^^^௧ subsets of data generated by resampling the historical normalized potency results. Specifically, resampling describes the process of random selection of potency results, with replacement, from the of the original set of potency results, until a dataset of the same size has been constructed. Under a bootstrapping approach, each data point from the original dataset is allowed to be picked as many times as the size of original dataset size or not picked at all. For each risk type, ^^_^^^௧ OOS risks are estimated, and respective confidence intervals (C.I.) are calculated using the appropriate percentiles of the OOS risks distribution generated from ^^_^^^௧ OOS risk estimates. For example, a 100 ∗ (1 − ^^) % C.I. is constructed using the 100 * ( ^{ఈ ఈ} ଶ )_th and 100 * (1 − _ଶ )_th percentile of the bootstrap distribution. FIG. 3 shows the workflow for OOS risk at release calculation and confidence interval calculation. The number of bootstrapped datasets ^^_^^^௧ simulated is determined based upon convergence of the estimated OOS risk at release. Convergence is determined by a convergence criterion ∆_^^^௩^^^^, where the difference in estimated OOS risk at release (upper and lower bound of the C.I.) does not exceed ∆_^^^௩^^^^, between an adjustment in ^^_^^^௧. Alternatively, convergence can be tested iteratively, and the specific value of ∆_^^^௩^^^^ is determined through discussion with process experts, regarding what a meaningful difference of risk would be and how it would impact decision making. System Front-end (Dashboard) Development The system front-end comprises an interactive dashboard that allows users to evaluate alternative potency targets, using the risk of OOS at release as the evaluation metric. Intermediate analyses which comprise distribution visualization, variance component analysis, and Shewhart control charting can be conducted by the user to further explore the historical data. A modular python library (back-end) was developed to include data querying from the RDS database, data processing, and statistical calculations. The back-end library was utilized by the dashboard to perform required calculations under the hood and was described herein (See System Back-end Python Library Development). The dashboard itself is implemented using Dash for Python. Dash is an open-source framework for building data visualization interfaces; it provides python interface for creating flexible, interactive code, and customizable systems that connects directly to analytics code. Dash unifies the graphing library Plot and other dashboard components with Flask, a general web system framework. The use of an open-source data visualization tool addresses many of the constraints imposed by commercial software, including limitations in transparency and customizability. The system is currently hosted through a cloud-based data science platform called Domino Data Lab (available over the internet from their website), which facilitates development and deployment of various data science systems. There are alternative cloud and non-cloud-based data science platforms that can be employed with the present embodiments. For instance, instead of using Domino Data Lab, other platforms can comprise MATLAB, Posit, Alteryx, RapidMiner, SAP HANA Cloud, Qubole, Databricks Lakehouse Platform, and IBM Watson Studio. Still further other platforms can be employed with the present embodiments which are known in the art. Aspects of the present teachings are to be further understood considering the following examples, which are not to be construed as limiting the scope of the present teachings in any way. The following section presents several examples of the Potency Targeting System, highlighting how it can be used to support biologics manufacturing. First, a demonstration of how the end-user interfaces with the UI to acquire relevant data for analysis is depicted followed by examples of how the output of the Potency Targeting System can be used in practice. For simplicity, actual inputs and values are generalized in the following section. FIG.4 depicts how the dashboard inputs are utilized to acquire potency results at release for a theoretical DP “Drug Product A”. After acquiring all available historical potency results at release from Discoverant, the user has the option to further specify a particular fill size of interest and limit the analyses to a specific bulk manufacturing date range (by selecting a start and end date). FIG.5 illustrates how the UI is used to limit historical release potency results for Drug A to the specific fill size of “Fill Size #1”, manufactured during January 1st, 2022, and April 1st, 2023. The “Select Batch” section of this step dynamically updates available batches in the drop down, to provide visual feedback to the user on the number of available batches, given the data filtering criteria used, and an option to deselect specific DP batches from the analysis. Once the acquired release potency results have been filtered based on the desired fill size and bulk manufacturing date range, the user can use historical process knowledge of bulk potency target changes to normalize all potency results to the current bulk potency target, before analysis. For Fill Size #1 of Drug A, the bulk potency target began at 123 IU/mL starting on Date #1. On Date #2, the bulk potency target was decreased by 3 IU/mL Finally, on Date #3, the bulk potency target was again decreased by 2 IU/mL Figure 6 depicts how the user would use the Potency Targeting System to normalize the potency results at release to the current bulk potency target of 118 IU/mL The following sub-sections present examples of the Potency Targeting System, assuming the input data has been acquired, filtered, and normalized, as outlined herein. Example 1: Alternate Potency Target Evaluation Recall, the Potency Targeting System was developed to evaluate DP alternate potency targets, using release OOS risk as the evaluation criteria. For Fill Size #1 of Drug A, the alternate potency targets of 106 IU/mL, 110 IU/mL, 115 IU/mL, 118 IU/mL, and 123 IU/mL are of interest. Before, the release OOS risk estimates can be calculated for each alternate potency target, the user must set the required simulation parameters (described herein), of the OOS Risk Estimation (Simulation) Parameter Settings section of the Potency Targeting System. FIG.7 depicts how the UI can be used to input the required simulation parameters. For each alternate potency target, several types of release OOS risk estimates (vial low, vial high, mean low, and overall) are estimated. The results of the analysis are summarized in a tabular form on the dashboard UI; however, the results are most easily interpreted when visually presented. FIG.8 illustrates how the alternate potency targets can easily be evaluated based upon risk of OOS at release using a bar chart presented by the dashboard. Based upon the alternate potency targets evaluated, the alternate potency target of 115 IU/mL identified to have the lowest release OOS risk estimates (all types). Thus, for Drug A Fill Size #1, when considering OOS risk at release, a decision on the optimal alternate potency target can be made relatively quickly. Other scenarios can exist where the selection of an optimal alternate potency target cannot be as straight forward. For example, the bulk potency target that is estimated to have acceptable OOS risk cannot be in a practical range or can present lower risk of OOS for a given type of OOS, but higher risk of OOS for another type of OOS (i.e., lower risk of vial low OOS, but higher risk of vial high OOS). In such a scenario, manufacturing process scientists must use their subject matter expertise to consider which factors are useful when considering a potential alternate potency target. Example 2: Evaluation of New Assay based upon Release OOS Risk As mentioned earlier the individual vial potency results are affected by the variability of the assay (this is shown explicitly by Equation 3). It follows that the assay variability has potential to affect the estimates of release OOS risk as well. Thus, an additional example has been identified, where the Potency Targeting System can be used to evaluate new potency assays (when the theoretical assay variability is assumed to be known). The Potency Targeting System dashboard provides an optional user-input panel labeled “Specify Assay Uncertainty (CV%)” to evaluate release OOS risk of an assay with a theoretical assay uncertainty (in units of CV%). By providing input for the assay uncertainty, the user can evaluate risk of OOS at release for alternate potency targets, using the new assay uncertainty, rather than using the assay uncertainty estimated from the historical potency results. As described earlier, the resulting estimates for release OOS risk for each alternate potency target can most easily be assessed visually using a bar chart. When compared to the results seen in FIG.7, FIG.10 shows lower OOS risk at release for respective alternate potency targets. Thus, in this illustrative example, the theoretical assay (with an assay uncertainty of 2% CV) improves process robustness by reducing the risk of OOS at release. With the overall reduction of release OOS risk, it appears that the alternate potency targets of 115 IU/mL and 118 IU/mL are now both viable options, when considering the implementation of the new potency assay. As mentioned in the previous use-case (Example 1), other factors outside of release OOS risk are considered when determining a suitable alternate potency target. These considerations, in addition to release OOS risk, are assessed comprehensively by manufacturing process experts to guide decision making regarding the appropriateness of an alternate potency target. Example 3: Evaluation of Action Limits and Potency Targets Thus far, assessment of alternate potency targets has been discussed with fixed specification and action limits. However, in practice there may be instances where the adjustments of action limits for various CQAs are considered. For instance, the batch mean (lower) action release limit for DP potency could be subject to reassessment, as a mitigation measure. In this scenario, it could be useful to evaluate release OOS risk estimates of alternate potency targets, while considering multiple candidates mean action limits. Table 1 illustrates an example of how the overall release risk of OOS for the alternate potency targets of 115 IU/mL, 118/mL, and 123 IU/mL can be estimated, using the mean action limits of 98 IU/mL, 103 IU/mL, and 108 IU/mL Table 1: Select results of overall OOS risk at release for Drug Product A Fill Size #1 given various combinations of mean potency limits and bulk potency targets. Mean Overall OOS risk (%) P t Li it P P P In practice, the results shown in Table 1 could be achieved by utilizing the Potency Targeting System as described previously, running the simulations with different settings for the mean action limit and tabulating the results of each run as a row of the table. The results shown in Table 1 could potentially be used to support the establishment of the appropriate alternate potency target and mean potency limit, required to achieve maximum process capability (lowest risk of DP potency failure). One aspect of the embodiments comprises the process of developing and deploying a software system used to streamline the assessment of alternate potency targets for DP in biologics manufacturing. The system was deployed using various cloud-based technologies, and an interactive dashboard was developed to enable effective usage of the system by manufacturing process scientists. The utility of the system was illustrated through several examples involving evaluation of alternate potency targets, evaluation of a new potency assay, and evaluation of both specification limits and alternate potency targets. Generally, Potency Targeting Systems are developed in biologics manufacturing to support GMP decisions and improve efficiency in the overall decision-making process. Before the results generated by systems can be used to support GMP decisions, comprehensive validation of the system is required to provide certainty in its results and confirm its accuracy in reporting them. Other embodiments can involve establishing a continuous integration and continuous deployment (CI/CD) pipeline, to improve frequency of system delivery to end-users by way automating code testing and building and automating system deployment to the cloud server. The implementation of a CI/CD pipeline would require setting up both a production and development environment, where the development environment would serve as a safe space for code developers to implement new features to the system, while the production environment would be reserved for deployment of tested updates to the final system. The benefit of the implementing a CICD pipeline is that the risk of unexpected impacts on the final system delivered to end-users is minimized. A natural extension would be to integrate the validation steps described for GMP systems into the CI/CD pipeline. Where only validated changes made in the development environment (i.e., passing unit-tests, integration tests, verification of statistical results, etc.) are pushed to the production environment. Ideally, such an implementation would ensure that updates to the software-system would already be validated for GMP use, before reaching the end-user. The present embodiments can be carried out by using a computer system. FIG.11 illustrates an exemplary computer system 200. In connection therewith, the computer system 200 can be configured, by executable instructions, to implement the various algorithms and other operations described herein. The exemplary computer system 200 can comprise, for example, one or more servers, workstations, personal computers, laptops, tablets, smartphones, other suitable computing devices, combinations thereof, etc. In addition, the computer system 200 can comprise a single computing device, or it can comprise multiple computing devices located in proximity or distributed over a geographic region and coupled to one another via one or more networks. Such networks can comprise, without limitations, the Internet, an intranet, a private or public local area network (LAN), wide area network (WAN), mobile network, telecommunication networks, combinations thereof, or other suitable network(s), etc. With that said, the illustrated computer system 200 comprises a processing unit 202 and a memory 204 that is coupled to (and in communication with) the processing unit 202. The processing unit 202 can comprise, without limitation, one or more processors (e.g., in a multi-core configuration, etc.), including a central processing unit (CPU), a microcontroller, a reduced instruction set computer (RISC) processor, and system specific integrated circuit (ASIC), a programmable logic device (PLD), a gate array, and/or any other circuit or processor designed for the functions described herein. The above listing is exemplary only, and thus is not intended to limit in any way the definition and/or meaning of the term processing unit. The memory 204, as described herein, is one or more devices that enable information, such as executable instructions and/or other data, to be stored and retrieved. The memory 204 can comprise one or more computer-readable storage media, such as, without limitation, dynamic random-access memory (DRAM), static random-access memory (SRAM), read only memory (ROM), erasable programmable read only memory (EPROM), solid state devices, flash drives, CDROMs, thumb drives, tapes, hard disks, and/or any other type of volatile or nonvolatile physical or tangible computer-readable media. The memory 204 can be configured to store, without limitation, process parameters, outputs, models and/or other types of data (and/or data structures) suitable for use as described herein, etc. In various embodiments, computer-executable instructions can be stored in the memory 204 for execution by the processing unit 202 to cause the processing unit 202 to perform one or more of the functions described herein, such that the memory 204 is a physical, tangible, and non- transitory computer-readable storage media. It is appreciated that the memory 204 can comprise a variety of different memories, each implemented in one or more of the functions or processes described herein. In the exemplary embodiments, the computer system 200 comprises an output unit 206 that is coupled to (and is in communication with) the processing unit 202. The output unit 206 outputs, or presents, to a user of the computer system 200, by, for example, displaying and/or otherwise outputting information such as, but not limited to, output, process parameters, and/or any other type of data. It is to be further appreciated that, in some embodiments, the output unit 206 can comprise a display device such that various interfaces (e.g., systems (network-based or otherwise), etc.) can be displayed at computer system 200, and at the display device, to display such information and data, etc. In some examples, the computer system 200 can cause the interfaces to be displayed at a display device of another computing device, including, for example, a server hosting a website having multiple webpages, or interacting with a web system employed at the other computing device, etc. Output unit 206 can comprise, without limitation, a liquid crystal display (LCD), a light- emitting diode (LED) display, an organic LED (OLED) display, an "electronic ink" display, combinations thereof, etc. In some embodiments, the output unit 206 can comprise multiple units. The computer system 200 further comprises an input device 208 that receives input from a user. The input device 208 is coupled to (and is in communication with) the processing unit 202 and can comprise, for example, a keyboard, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad or a touch screen, etc.), another computing device, and/or an audio input device. Further, in some exemplary embodiments, a touch screen, such as that included in a tablet or similar device, can perform as both the output unit 206 and the input device 208. In at least one exemplary embodiment, the output unit 206 and the input device 208 can be omitted. In addition, the illustrated computer system 200 comprises a network interface 210 coupled to (and in communication with) the processing unit 202 (and, in some embodiments, to the memory 204 as well). The network interface 210 can comprise, without limitation, a wired network adapter, a wireless network adapter, a telecommunications adapter, or other device designed for communicating to one or more different networks. The network interface (210) and/or the input device (208) are referred to as receiving unit herein. In this manner, a computer system 200 can include processing unit 202 and a computer-readable storage medium or memory 204 coupled to the processing circuitry, where the processing circuitry is configured to execute computer-readable program code instructions stored in the memory 204. It will also be understood that one or more functions, and combinations of functions, may be implemented by special purpose hardware-based computer systems and/or processing circuitry which perform the specified functions, or combinations of special purpose hardware and program code instructions. Execution of instructions by a processing unit, or storage of instructions in a computer- readable storage medium, supports combinations of operations for performing the specified functions. For instance, the instructions input to the processing unit for storage in a computer readable storage medium can comprise input parameters for a data acquisition module to determine alternate potency targets. Further, a non-transitory computer readable storage medium having stored thereon software instructions that, when executed by a processor of a computer system, cause the computer system to execute the following method:: allowing a user to input user input parameters into a data acquisition module to determine alternate potency targets, and an output for displaying processed results; pre-processing data from the data acquisition module including normalization; and applying statistical modelling calculations using out of specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the alternate potency target, and the output displays the respective OOS risks. While the present embodiments have been described with reference to the specific examples, various modifications and changes can be made, and equivalents can be substituted without departing from the true spirit and scope of the claims appended hereto. The specification and examples are, accordingly, to be regarded in an illustrative rather than in a restrictive sense.

Claims

CLAIMS We Claim: 1. A computer implemented method, comprising: (a) providing a front-end dashboard comprising a user input for inputting user input parameters into a data acquisition module to determine alternate potency targets, and an output for displaying processed results; (b) providing a backend for: pre-processing data from the data acquisition module including normalization; applying statistical modelling calculations using out of specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the alternate potency target, and the output displays the respective OOS risks.

2. A computer implemented method as recited in claim 1, wherein the user input parameters comprise drug product, fill size, batch selection, and manufacturing date range.

3. A computer implemented method as recited in claim 1, wherein the data acquisition module further comprises source data systems.

4. A computer implemented method as recited in claim 3, wherein the data acquisition module further comprises a data acquisition system.

5. A computer implemented method as recited in claim 4, wherein the data acquisition module further comprises cloud-based data storage.

6. A computer implemented method as recited in claim 5, wherein the data acquisition module further comprises cloud-based computing.

7. A computer implemented method as recited in claim 1, wherein the pre-processing step further comprises normalizing the data by statistical analysis.

8. A computer implemented method as recited in claim 7, wherein the data is normalized by calculating P normalized, according to the following equation: P = P ∗ ^{P^்} ெ where: P = Potency Values, P_CT = Current Bulk Potency Target, and PTM = Bulk Potency Target at Time of Manufacture

9. A computer implemented method as recited in claim 1, wherein the OOS risk selection criteria for a candidate alternate potency target can be determined from a model where: 1) ^^ ^^ ^^_{^^^^ ^^^^}: % OOS risk of at least one individual replicate result failing upper release limit given that the calculated % CV is within the user defined limit; 2) ^^ ^^ ^^_{^^^^ ^^௪}: % OOS risk of at least one individual replicate result failing lower release limit given that the calculated % CV is within the user defined limit; 3) ^^ ^^ ^^_{ெ^^^ ^^௪}: % OOS risk of average result failing lower mean release limit given that the calculated % CV is within the user defined limit; and 4) ^^ ^^ ^^_ை௩^^^^^: % OOS risk of failing in at least one of the above scenarios.

10. A computer implemented method as recited in claim 1, wherein the data acquisition module further comprises a cloud-based relational database.

11. A computer implemented method as recited in claim 10, wherein a data query can be made between the user input and the cloud-based relational database.

12. A computer implemented system, comprising: (a) a front-end dashboard comprising a user input for inputting user input parameters into a data acquisition module to determine alternate potency targets, and an output for displaying processed results; (b) a backend for pre-processing data from the data acquisition module including normalization; applying statistical modelling calculations using out of specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the system determines the alternate potency target, and the output displays the respective OOS risks.

13. A computer implemented system as recited in claim 12, wherein the user input parameters comprise drug product, fill size, batch selection, and manufacturing date range.

14. A computer implemented system as recited in claim 12, wherein the data acquisition module further comprises source data systems.

15. A computer implemented system as recited in claim 14, wherein the data acquisition module further comprises a data acquisition system.

16. A computer implemented system as recited in claim 15, wherein the data acquisition module further comprises cloud-based data storage.

17. A computer implemented system as recited in claim 16, wherein the data acquisition module further comprises cloud-based computing.

18. A non-transitory computer readable storage medium having stored thereon software instructions that, when executed by a processor of a computer system, cause the computer system to execute the following method: (a) allowing a user to input user input parameters into a data acquisition module to determine alternate potency targets, and an output for displaying processed results; (b) pre-processing data from the data acquisition module including normalization; and (c) applying statistical modelling calculations using out of specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the alternate potency target, and the output displays the respective OOS risks.