RU2799750C9 - Bioinformation systems, devices and methods for secondary and/or tertiary processing - Google Patents

Abstract

FIELD: bioinformatics.

SUBSTANCE: system, method and device for performing bioinformatics analysis on genome sequencing data. In particular, a method and system is provided for improving the accuracy of variant detection by co-scoring reads that map two or more regions of a reference sequence that are homologous. The method includes: accessing a cluster of reads; determining a set of candidate options from the cluster; evaluation of each option; formation of a file of definitions of options. The system contains: a computer and a storage device with instructions for initiating a call to a collection of reads, determining candidate variants and establishing the order of their processing, evaluating each variant and generating a variant definition file. Also provided is a computer-readable storage device with instructions initiating the execution of operations to improve the accuracy of determining options. The invention expands the arsenal of tools for the implementation of bioinformatic protocols.

EFFECT: increased accuracy of determining variants.

20 cl, 51 dwg

Description

Cross reference to related application

[001] This application claims priority under 35 USC §119(e) to US Provisional Patent Application No. 62/347,080, filed June 7, 2016, entitled “Bioinformatics Systems, Apparatuses, and Methods Executed on an Integrated Circuit Processing Platform"; US Provisional Patent Application No. 62/399,582, filed September 26, 2016, entitled "Bioinformatics Systems, Apparatuses, and Methods Executed on an Integrated Circuit Processing Platform"; US Provisional Patent Application No. 62/414,637, filed October 28, 2016, entitled "Bioinformatics Systems, Apparatuses, and Methods Executed on an Integrated Circuit Processing Platform"; US Provisional Patent Application No. 62/462,869, filed February 23, 2017, entitled "Bioinformatics Systems, Apparatuses, and Methods Executed on a Quantum Processing Platform"; by U.S. Provisional Patent Application No. 62/469,442, filed March 9, 2017, entitled "Bioinformatics Systems, Apparatuses, and Methods Executed on an Integrated Circuit Processing Platform", references made under this paragraph and which are incorporated in their entirety by this document by reference. This application is also a continuation application in part and claims priority under 35 USC §120 to US Patent Application No. 15/404,146, filed January 11, 2017, entitled “Genomic Infrastructure for On-Site or Cloud-Based DNA.” and RNA Processing and Analysis"; and US Patent Application No. 15/497,149, filed April 25, 2017, entitled “Bioinformatics Systems, Apparatuses, and Methods Executed on a Quantum Processing Platform.”

Technical field

[002] The subject matter of the invention described herein relates to bioinformatics and, in particular, to systems, devices and methods for implementing bioinformatics protocols, such as performing one or more functions for analyzing genomic data on an integrated circuit, such as a hardware processing platform.

State of the art

[003] As described in detail herein, some of the major computational challenges of high-throughput DNA sequencing analysis are the need to cope with the explosion of available genomic data, the need for increased accuracy and sensitivity in collecting this data, and the need for fast, efficient, and accurate computationally when performing analysis on a wide range of sequencing data sets derived from such genomic data.

[004] To keep up with this increased sequencing throughput provided by new generation sequencers, it has been common to employ multi-threaded software that runs on more and more faster processors in computing clusters with expensive, highly available memory that requires significant power. and significant costs for information technology support. It is important to note that future increases in sequencing throughput will result in accelerating real-money cost increases for these downstream processing solutions.

[005] To solve at least part of these and other similar problems, the devices, systems, and methods for using them described herein are provided.

Disclosure of the invention

[006] The present invention relates to devices, systems, and methods for using them to perform one or more genomics and/or bioinformatics protocols on data generated by a front-end processing procedure, such as genetic sequence data. For example, in various aspects, provided herein are devices, systems, and methods configured to implement protocols for secondary and/or tertiary analysis of genetic data, such as data generated by RNA and/or DNA sequencing, for example, using a next-generation sequencer ( SNP). In specific embodiments, one or more secondary processing pipelines are provided for processing genetic sequence data. In other embodiments, one or more tertiary processing pipelines are provided for processing genetic sequence data, for example, where the pipelines and/or individual elements thereof provide superior sensitivity and improved accuracy over a wider range of sequence-derived data than is currently available in the field. field of technology.

[007] For example, a system is provided herein, such as for implementing one or more sequence and/or genome analysis pipelines on genetic sequence data and/or other data derived therefrom. In various embodiments, the system may include one or more electronic data sources that provide digital signals representing a plurality of genetic and/or genomic data reads, for example, where each of the plurality of genomic data reads includes a nucleotide sequence. The system may also include a memory, such as DRAM or a cache, such as for storing one or more of the sequential reads, one or more genetic reference sequences, and one or more indexes of the one or more genetic reference sequences. The system may further include one or more integrated circuits, such as an FPGA, ASIC or sASIC, and/or a CPU and/or GPU, wherein the integrated circuit, for example in the case of an FPGA, ASIC or sASIC, may be formed from a set of hard-wired digital logic circuits that are interconnected by many physical electrical interconnections. The system may further include a quantum computing processor for use in implementing one or more of the methods described herein.

[008] In various embodiments, one or more of the plurality of electrical interconnections may be an input to one or more integrated circuits that may be connected or connectable, for example, directly, through a suitable wiring connection, or indirectly, for example, through a wireless network connection (for example, cloud or hybrid cloud) to an electronic data source. Regardless of the connection to the sequencer, the integrated circuit of the present invention may be configured to receive multiple reads of genomic data, for example directly from the sequencer or from associated memory. Reads can be digitally encoded in the standard FASTQ or BCL file format. Accordingly, the system may include an integrated circuit having one or more electrical interconnections, which may be a physical interconnection including a memory interface, to enable the integrated circuit to access memory.

[009] In particular, a hard-wired digital integrated circuit logic circuit may be implemented as a set of processing engines, such that each processing engine may be formed from a subset of hard-wired digital logic circuits to perform one or more steps in a sequence, genome and /or tertiary analysis, as described below herein, on multiple genetic data reads, as well as other data derived from the genetic data. For example, each subset of hard-wired digital logic circuits may be configured to perform one or more steps in the analysis pipeline. Additionally, in the case where the integrated circuit is an FPGA, such steps in the sequence analysis or further analysis process may include partially reconfiguring the FPGA during the analysis process.

[0010] In particular, the set of processing engines may include a mapping module, for example in an installation configuration, to, in accordance with at least some of the nucleotide sequences in a read of the plurality of reads, access an index of one or more genetic reference sequences from memory through a memory interface for mapping the read into one or more segments of said one or more genetic reference sequences based on the index. In addition, the set of processing engines may include an alignment module in a mounting configuration to access one or more genetic reference sequences from memory through a memory interface to align a read, such as a mapped read, to one or more positions in one or more segments said one or more genetic reference sequences, for example obtained from a mapping module and/or stored in memory.

[0011] The set of processing engines may also include a sorting module to sort each aligned read according to one or more positions in one or more genetic reference sequences. In addition, the set of processing engines may include a variant detection module, such as for processing mapped, aligned and/or sorted reads, for example, against a reference genome, thereby creating an HMM record file, and/or defining variants to operate on them , and/or detailing variations between sequenced genetic data and reference genome data. In various cases, one or more of the plurality of physical electrical connections may include an output from the integrated circuit for exchanging resultant data from the mapping module and/or alignment and/or sorting and/or variant determination modules.

[0012] In particular, with respect to the mapping module, various embodiments provide a system for implementing a mapping analysis pipeline on multiple genetic data reads using a genetic reference data index. In various cases, a genetic sequence, such as a read and/or genetic reference data, may be represented by a sequence of nucleotides, which may be stored in the system's memory. The mapping module may be included in an integrated circuit and may be formed from a plurality of preconfigured and/or hardwired digital logic circuits that are interconnected by a plurality of physical electrical interconnections, wherein the physical electrical interconnections may include a memory interface to provide the integrated circuit with access capability to memory. In more specific embodiments, the hard-wired digital logic circuits may be implemented as a set of processing engines, such that each processing engine may be formed by a subset of the hard-wired digital logic circuits to perform one or more steps in a sequence analysis pipeline on a plurality of genomic data reads.

[0013] For example, in one embodiment, a set of processing engines may include a mapping module in a hard-wired configuration, where the mapping module and/or one or more of its processing engines are configured to receive a read of genomic data, such as through one or more of multiple physical electrical interconnections, and isolating part of the read in such a way as to form a seed from it. In such a case, the read may be represented by a nucleotide sequence, and the primer may be a subset of the nucleotide sequence represented by the read. The mapping module may include or be configured to connect to a memory that contains one or more reads, one or more read seeds, at least a portion of one or more reference genomes, and/or one or more indexes, wherein such index is constructed from one or more reference genomes. In certain cases, the mapping engine processing engine may use the seed and index to calculate an address in the index based on the seed.

[0014] Once an address is calculated or obtained and/or otherwise stored, such as in on-chip or external memory, the address can be accessed at an index in the memory to receive an entry from the address, such as an entry representing information about positions in the genetic reference sequence. This position information can then be used to identify one or more matching positions from the read in a genetic reference sequence based on the record. At least one of the matching positions may then be output to memory via a memory interface.

[0015] In another embodiment, the set of processing engines may include an alignment module, for example, in a preconfigured and/or hardwired configuration. In this case, one or more of the processing engines may be configured to receive one or more mapped positions for the read data via one or more of a plurality of physical electrical interconnections. Each mapped position in memory (internal or external) can then be accessed to retrieve the reference sequence/genome segment corresponding to the mapped position. On each extracted reference segment, a read alignment can be calculated along with an alignment score. Once the calculation is complete, at least one read alignment with the best alignment score can be selected and output. In various cases, the alignment module also implements a dynamic programming algorithm in computing the alignment, such as one or more Smith-Waterman algorithms with linear or affine gap estimation, a gap alignment algorithm, and/or a gapless alignment algorithm. In specific cases, calculating the alignment may involve first performing a gapless alignment on each reference segment and, based on the results of the gapless alignment, selecting reference segments to further perform gap alignments on them.

[0016] In various embodiments, a variant determination module may be provided to perform improved variant determination functions that, when implemented in one or both software and/or hardware configurations, provide superior processing speed, better processing result accuracy, and improved overall efficiency than methods, devices and systems currently known in the art. Namely, in accordance with one aspect, improved methods are provided for performing variant determination operations in software and/or hardware, for example, for performing one or more HMM operations on genetic sequence data. According to another aspect, new devices are provided including an integrated circuit for performing such improved variant determination operations, wherein at least a portion of the variant determination operation is implemented in hardware.

[0017] Accordingly, in various cases, the methods described herein may include mapping, using a first subset of hard-wired and/or quantum digital logic circuits, a plurality of reads to one or more segments of said one or more genetic reference sequences. In addition, the methods may include integrated circuits and/or quantum accessing, for example, through one or more of a plurality of physical electrical interconnections, one or more mapped reads and/or one or more genetic reference sequences from an associated memory or cache; and aligning, using a second subset of hardwired and/or quantum digital logic circuits, the plurality of mapped reads on one or more segments of said one or more genetic reference sequences.

[0018] In various embodiments, the method may further include having the integrated circuit/or quantum circuit, for example, through one or more of a plurality of physical electrical interconnects, access an aligned set of reads from an associated memory or cache. In such a case, the method may include sorting, by a third subset of hardwired and/or quantum digital logic circuits, the aligned set of reads according to their positions in one or more genetic reference sequences. In certain cases, the method may also include outputting, for example, via one or more of a plurality of physical integrated circuit and/or quantum circuit electrical interconnects, the resulting mapping and/or alignment and/or sorting data, for example, where the resulting data comprises positions of the mapped and/or aligned and/or sorted set of reads.

[0019] In some cases, the method may further include using the resulting resulting data, for example, using another subset of hard-wired and/or quantum digital logic circuits, for the purposes of determining how the resulting data is mapped, aligned, and/or sorted from a subject's sequenced genetic sample differs from the reference sequence to create a variant definition file describing the genetic differences between the two samples. Accordingly, in various embodiments, the method may also include having access by the integrated circuit/or quantum circuit, such as through one or more of a plurality of physical electrical interconnects, to a mapped and/or aligned and/or sorted set of reads from an associated them memory or cache. In such a case, the method may include performing a variant determination function, such as an HMM operation or a paired HMM operation, on the accessed reads using a third or fourth subset of hard-wired and/or quantum digital logic circuits to create a variant determination file. , detailing how the mapped, aligned and/or sorted reads differ from one or more reference sequences, such as a haplotype.

[0020] Accordingly, in accordance with certain aspects of the invention, a compact hardware, such as a chip-based, or quantum accelerated platform is provided herein for performing secondary and/or tertiary analysis on genetic and/or genomic sequencing data. In particular, a platform or pipeline of hard-wired and/or quantum digital logic circuits is provided that are specifically configured to perform secondary and/or tertiary genetic analysis, for example on sequenced genetic data or genomic data derived therefrom. In particular, a set of hard-wired and/or quantum logic circuits may be provided, which may be implemented as a set of processing engines, such that the processing engines may be present in a pre-configured and/or hard-wired and/or quantum configuration on the processing platform of the present invention, and may be specifically configured to perform secondary operations of mapping and/or alignment and/or variant determination related to genetic analysis on DNA and/or RNA data, and/or may be specifically configured to perform other tertiary processing on these results.

[0021] In specific cases, the present devices, systems and methods of use thereof are optimized to, when executing one or more secondary and/or tertiary processing genomics and/or bioinformatics protocols, provide improved processing speeds that are orders of magnitude faster than standard secondary processing pipelines , implemented in software. In addition, the pipelines and/or their components that are described herein provide increased sensitivity and accuracy on a wide range of sequence-derived datasets for genomic and bioinformatics processing purposes. In various embodiments, one or more of these operations may be performed by an integrated circuit that is part of or implemented as a general purpose central processing unit and/or a graphics processing unit and/or a quantum processing unit.

[0022] For example, genomics and bioinformatics are fields concerned with the application of information technology and computer science to the field of genetics and/or molecular biology. In particular, bioinformatics methods can be applied to the processing and analysis of various genetic and/or genomic data, for example, from an individual, to determine qualitative and quantitative information about this data, which can be used by various medical practitioners in the development of preventive, therapeutic and/or or diagnostic methods for preventing, treating, reducing the intensity and/or at least identifying disease states and/or their potential and thereby improving the safety, quality and efficiency of healthcare at the individual level. Consequently, as the fields of genomics and bioinformatics focus on the development of personalized healthcare, they encourage individualized healthcare that is proactive rather than reactive, and this empowers the individual in need of treatment to become more involved in their own health. The advantage of using genetics, genomics and/or biotechnology technologies described herein is that qualitative and/or quantitative analysis of molecular biological (e.g., genetic) data can be performed on a wide range of sample sets at a significantly higher rate of speed and often more accurately, thereby accelerating the emergence of personalized healthcare. Specifically, in various embodiments, the genomics and/or bioinformatics-related tasks may form a genomic pipeline that includes one or more of a microarray analysis pipeline, a genome analysis pipeline, e.g., a genome-wide analysis pipeline, a genotyping analysis pipeline, an exome analysis pipeline, a microbiome analysis, genotyping analysis pipeline including co-genotyping, variant analysis pipelines including structural variants, somatic variants, and GATK, and RNA sequencing analysis pipeline and other genetic analysis pipelines.

[0023] Accordingly, to take advantage of these advantages, there are improved and more accurate software implementations for performing one or a number of such bioinformatics-based analytical methods, for example for deployment using a general purpose CPU and/or GPU, and/or they can be implemented in one or more quantum processing platform circuits. However, traditionally configured software-based methods and systems tend to be labor intensive, time consuming to execute on such general purpose processors, and error prone. Therefore, bioinformatics systems implemented herein that can execute these algorithms, for example implemented in software, using the CPU and/or GPU of a quantum processing device with less labor and/or processing intensity at a higher percentage of accuracy will be useful.

[0024] Such implementations are developed and presented herein, for example, where genomic and/or bioinformatics analyzes are performed by optimized software running on a CPU and/or GPU and/or quantum computer in a system that uses genetic sequence data obtained using processors and/or integrated circuits according to this invention. Additionally, it must be noted that the cost of analyzing, storing and sharing this raw digital data far exceeds the cost of creating it. Accordingly, the present document also proposes just-in-time storage and/or retrieval methods that optimize the storage of such data such that, rather than incurring the expense of collectively storing such data, rapid data regeneration is employed. Therefore, the just-in-time or "JIT" data generation, analysis, and storage techniques presented herein address the major bottleneck that is the long-standing but unresolved obstacle between the ever-increasing data generation and storage and the real possibility of penetrating the essence of medical problems based on them.

[0025] Therefore, provided herein are systems, devices and methods for implementing genomics and/or bioinformatics protocols or portions thereof, such as for performing one or more genomic data analysis functions, for example, in one or both of an integrated circuit, such as hardware a processing platform, and a general purpose processor, such as for performing one or more bioanalytical operations in software and/or firmware. For example, as discussed later herein, various implementations have proposed an integrated circuit and/or a quantum circuit for accelerating one or more processes on a primary, secondary, and/or tertiary processing platform. In various cases, an integrated circuit may be used to perform genetic analytics-related tasks, such as mapping, variant detection, compression, decompression, and the like, in an accelerated manner, and therefore the integrated circuit may include a hardware accelerated configuration. Additionally, in various cases, an integrated and/or quantum circuit may be provided, such as a circuit that is part of a processor, which is configured to implement one or more genomics and/or bioinformatics protocols on the generated mapped and/or aligned data and/or data with certain options.

[0026] Specifically, in a first embodiment, the first integrated circuit may be formed from an FPGA, an ASIC, and/or a sASIC that is coupled to or otherwise attached to a motherboard and configured or, in the case of an FPGA, programmed via firmware , which is to be configured as a set of hard-wired digital logic circuits that are configured to implement at least a first set of sequence analysis functions in a genomic analysis pipeline, for example, wherein the integrated circuit is configured as described herein above, capable of including one or more digital logic circuits that are arranged as a set of processing engines configured to perform one or more steps in the operation of mapping, aligning and/or identifying variants on genetic data to generate sequence analysis result data. The first integrated circuit may also include an output, for example, formed from a plurality of physical electrical interconnections, such as for transferring resultant data from mapping and/or alignment procedures and/or other procedures to memory.

[0027] Additionally, a second integrated and/or quantum circuit may be included coupled to or otherwise attached to the motherboard and communicating with the memory via a communications interface. The second integrated and/or quantum circuit may be configured as a central processing unit (CPU) or a graphics processing unit (GPU), or a quantum processing unit (CPU), which is configured to receive the resulting data mapped and/or aligned and/or subjected to the determination of sequence variants, and may be configured to respond to one or more software algorithms that are configured to issue instructions to the CPU or GPU to perform one or more genomic and/or bioinformatics functions of the genomic analysis pipeline on the resulting mapped, aligned or subjected to determination of sequence variants. Specifically, genomics and/or bioinformatics-related tasks may form a genomic analysis pipeline that includes one or more of a microarray analysis pipeline, a genome analysis pipeline, e.g., a genome-wide analysis pipeline, a genotyping analysis pipeline, an exome analysis pipeline, a microbiome analysis pipeline, genotyping assay pipelines, including co-genotyping, variant analysis pipelines, including structural variants, somatic variants, and GATK, and the RNA-seq analysis pipeline and other genetic assay pipelines.

[0028] For example, in one embodiment, the CPU and/or GPU and/or CPU of the second integrated circuit may comprise software that is configured to organize a genome analysis pipeline to implement a genome-wide analysis pipeline, such as a genome-wide analysis pipeline that includes including one or more of genome-wide variation analysis, exome-wide DNA analysis, transcriptome-wide RNA analysis, gene functional analysis, protein functional analysis, protein binding analysis, quantitative gene analysis, and/or protein assembly analysis. In certain cases, a genome-wide analysis pipeline may be performed for the purposes of one or more of kinship analysis, personal history analysis, disease diagnosis, drug discovery, and/or protein profiling. In a specific case, the genome-wide analysis pipeline is carried out for the purpose of oncology analysis. In various cases, the results of this data may be made available, for example globally, throughout the system.

[0029] In various cases, the CPU and/or GPU and/or quantum processing unit (CPU) of the second integrated circuit and/or quantum circuit may comprise software that is configured to organize a genome analysis pipeline to perform genotyping analysis, such as genotyping, including joint genotyping. For example, a joint genotyping analysis can be performed using a Bayesian probability calculation, such as a Bayesian probability calculation that results in the absolute probability that a given genotype is the true genotype. In other cases, the software may be configured to perform metagenome analysis to generate resulting metagenome data, which can in turn be used to perform microbiome analysis.

[0030] In certain cases, the first and/or second integrated circuits and/or memory may be included in an expansion card, such as a peripheral component interconnect (PCI) card. For example, in various embodiments, one or more of the integrated circuits may be one or more chips coupled to a PCIe card or otherwise coupled to a motherboard. In various cases, integrated and/or quantum circuits and/or chips may be a component in a sequencer, or a computer, or a server, such as part of a server farm. In specific embodiments, integrated circuits and/or quantum circuits and/or expansion cards and/or computers and/or servers may be accessible via the Internet, such as the cloud.

[0031] Additionally, in some cases, the memory may be volatile random access memory (RAM), such as direct access memory (DRAM). Specifically, in various embodiments, the memory may include at least two memories, such as a first memory that is an HMEM, such as for storing haplotype reference sequence data, and a second memory that is an RMEM, such as for storing a read data genomic sequence. In specific cases, each of the two memories may include a write port and/or a read port, for example, where each of the write port and read port has access to a separate clock generator. In addition, each of the two memories may include a trigger configuration for storing a plurality of genetic sequence data and/or processing results.

[0032] Accordingly, in another aspect, the system may be configured to share memory resources among its constituent parts, for example, when performing certain computing tasks through software, such as executed by a CPU and/or GPU and/or quantum processing platform, and/or performing other computing tasks via firmware, such as via associated integrated circuit hardware such as an FPGA, ASIC and/or sASIC. This can be achieved in various ways, such as direct weak or hard coupling between the CPU/GPU/CPU and the FPGA fabric, such as a chip or PCIe card. Such configurations may be particularly useful in distributing operations related to the processing of large data structures associated with genomic and/or bioinformatics analyzes and intended for use and access by both the CPU/GPU/CPU and the associated integrated circuit. In particular, in various embodiments, when processing data through a genomic pipeline as described herein, for example, to speed up overall processing, timing, and efficiency, a number of different operations may be performed on the data, which operations may involve both software and processing hardware components.

[0033] Therefore, it may be necessary to share or otherwise exchange data between software components running on the CPU and/or GPU and/or GPU and/or a hardware component embedded on a chip, such as an FPGA. Accordingly, one or more of the various steps in a genomic and/or bioinformatics processing pipeline, or part thereof, may be performed by a single device, such as a CPU/GPU/CPU, and one or more of the various stages may be performed by a hard-wired device, such as an FPGA . In such a case, the CPU/GPU/CPU and/or FPGA may be coupled to communicate in a manner that allows such data to be transferred efficiently, which coupling may include sharing memory resources. To achieve such distribution of tasks and sharing of information to perform such tasks, various CPUs/GPUs/CPUs may be loosely or tightly coupled to each other and/or hardware devices, such as an FPGA or other chipset, such as through a fast access interconnect.

[0034] In particular, a genomic analysis platform is provided in various embodiments. For example, the platform may include a motherboard, memory, a plurality of integrated and/or quantum circuits such as forming one or more of a CPU/GPU/CPU, a mapping module, an alignment module, a sorting module, and/or a variant determination module. Namely, in specific embodiments, the platform may include a first integrated circuit and/or quantum circuit, such as an integrated circuit forming a central processing unit (CPU) or a graphics processing unit (GPU), or a quantum circuit forming a quantum processor that responds one or more programs or other algorithms that are configured to issue instructions to the CPU/GPU/CPU to perform one or more sets of genomic analysis functions as described herein, for example, where the CPU/GPU/CPU includes a first set of physical electronic interconnects for connection to the motherboard. In various cases, the memory may also be attached to the motherboard and may also be electronically coupled to the CPU/GPU/CPU, for example, through at least a portion of a first set of physical electronic interconnects. In such cases, the memory may be configured to store multiple reads of genomic data, and/or at least one or more genetic reference sequences, and/or an index of one or more genetic reference sequences.

[0035] In addition, the platform may include one or more other integrated circuits, for example, where each of the other integrated circuits forms a field programmable gate array (FPGA) having a second set of physical electronic interconnects for connection to the CPU/GPU/CPU and memory, for example via a point-to-point protocol. In such a case, for example, where the integrated circuit is an FPGA array, the FPGA array may be programmable via firmware to configure a set of hard-wired digital logic circuits that are interconnected by a plurality of physical interconnects to perform a second set of genomic analysis functions, e.g. , mapping, leveling, identifying options, etc. In particular, the hard-wired digital logic circuits of the FPGA may be implemented as a set of processing engines for implementing one or more preconfigured steps in a genomic analysis sequence analysis pipeline, for example, where the sets of processing engines include one or more mapping modules, and /or alignment, and/or definition of variants, and modules can be formed from separate or the same subsets of processing engines.

[0036] As noted, the system may be configured to include one or more processing engines, and in various embodiments, the included processing engine may itself be configured to determine one or more transition probabilities for a nucleotide sequence of a genomic sequence read, moving from one state to another, for example, from a match state to an indel state, or from a match state to a deletion state and/or vice versa, for example, from an insertion or deletion state back to a match state. Additionally, in various instances, the integrated circuit may have a pipelined configuration and/or may include a second and/or third and/or fourth subset of hard-wired digital logic circuits, such as including a second set of processing engines, wherein the second set of processing engines comprises a mapping module configured to map a genomic sequence read to a haplotype reference sequence to create a mapped read. A third subset of hard-wired digital logic circuits may also be included, for example, where the third set of processing engines comprises an alignment module configured to align the mapped read to one or more positions in a haplotype reference sequence. A fourth set of hard-wired digital logic circuits may also be further included, for example where the fourth set of processing engines comprises a sorting module configured to sort the mapped and/or aligned read by its relative positions in the chromosome. As above, in various similar cases, the mapping module and/or the alignment module and/or the sorting module, for example, together with the variant determination module, can be physically built into the expansion card. And in certain embodiments, the expansion card may be physically integrated with a genetic sequencer, such as a next generation sequencer or the like.

[0037] Accordingly, according to one aspect, an apparatus is provided for performing one or more stages of a sequence analysis pipeline, for example on genetic data, where the genetic data comprises one or more genetic reference sequences, such as a haplotype or hypothetical haplotype sequence, an index of one or more genetic reference sequences sequences and/or multiple reads, such as genetic and/or genomic data, wherein the data may be stored in one or more shared storage devices, and/or may be processed by a distributed processing resource such as a CPU/GPU/CPU and/or FGPAs that are connected, for example, tightly or loosely, together. Thus, in various instances, a device may comprise an integrated circuit, wherein the integrated circuit may comprise one or more (e.g., a set) of hard-wired digital logic circuits, wherein the set of hard-wired digital logic circuits may be interconnected, for example, by one or a plurality of physical electrical interconnections.

[0038] Accordingly, the system may be configured to include an integrated circuit formed of one or more digital logic circuits that are interconnected by a plurality of physical electrical interconnections, wherein one or more of the plurality of physical electrical interconnections has one or more memory interfaces and /or a cache for the integrated circuit to access and retrieve memory and/or data stored therein, for example, providing cache coherence between the CPU/GPU/CPU and an associated chip, such as an FPGA. In various cases, the digital logic circuits may include a first subset of digital logic circuits, for example, where the first subset of digital logic circuits may be embodied as a first set of processing engines, wherein the processing engine may be configured to access data stored in a cache and /or directly or indirectly connected memory. For example, the first set of processing engines may be configured to perform one or more steps of mapping and/or alignment and/or sorting analysis, as described above, and/or HMM analysis on the genomic sequence data read and the haplotype sequence data.

[0039] More specifically, the first set of processing engines may comprise an HMM module, for example, in a first configuration of a subset of digital logic circuits, that is configured to access in memory, for example through a memory interface, at least some of the nucleotide sequences in the data read. genomic sequence and haplotype sequence data, and may also be configured to perform an HMM analysis on at least some of the nucleotide sequence in the haplotype sequence data to generate the resulting HMM data. In addition, one or more of the plurality of physical electrical interconnections may include output from an integrated circuit, for example, to transfer resultant HMM data from the HMM, for example, to a CPU/GPU/CPU or to a server or server cluster.

[0040] Accordingly, according to one aspect, a method is provided for implementing a sequence analysis pipeline, for example, on genetic sequence data. The genetic data may comprise one or more genetic reference sequences or haplotype sequences, one or more indices of one or more genetic sequences and/or haplotype sequences, and/or a plurality of genomic data reads. The method may include one or more of receiving, accessing, mapping, aligning, sorting various iterations of genetic sequence data and/or using the results thereof in a method for creating one or more variant definition files. For example, in certain embodiments, the method may include receiving as input into an integrated circuit from an electronic data source one or more of a plurality of genomic data reads, where each genomic data read may comprise a nucleotide sequence.

[0041] In various cases, an integrated circuit may be formed from a plurality of hard-wired digital logic circuits, which may be implemented as one or more processing engines. In such a case, the processing engine may be formed from a subset of hard-wired digital logic circuits that may be in a wiring configuration. In such a case, the processing engine may be configured to perform one or more preconfigured steps, for example to implement one or more of receiving, accessing, mapping, aligning, sorting various iterations of genetic sequence data and/or using the results thereof in a method for creating one or more variant definition files. In some embodiments, the proposed digital logic circuits may be interconnected, such as through a plurality of physical electrical interconnections, which may include an input.

[0042] The method may also include having the integrated circuit access one or more of a plurality of physical electrical connections from memory to data to perform one of the operations described in detail herein. In various cases, the integrated circuit may be part of a chipset, such as embedded or otherwise included as part of an FPGA, ASIC, or structured ASIC, and the memory may be directly or indirectly coupled to one or both of the chips and/or associated CPUs. GPU/CPU. For example, the memory can be a plurality of memories, one of which is connected to the chip and the CPU/GPU/CPU, which is itself connected to the chip, for example loosely.

[0043] In other cases, the memory may be a single memory that may be coupled to a CPU/GPU/CPU that is itself hardwired to the FPGA via a strong processing interconnect or a fast access interconnect, such as a QPI, and is thereby accessible to the FPGA , for example, ensuring cache coherence. Accordingly, the integrated circuit may be directly or indirectly coupled to memory to access data relevant to performing the functions provided herein, for example, to access one or more of a plurality of reads, one or more genetic references, or theoretical references. sequences and/or an index to one or more genetic reference sequences, for example when performing a mapping operation.

[0044] Therefore, in various cases, implementations of various aspects of the invention may include, without limitation: devices, systems and methods including one or more of the features that are described in detail herein, as well as articles that include a tangibly embodied computer-readable storage medium, configured to cause one or more machines (eg, computers, etc.) to perform the operations described herein. Likewise, computer systems are also described that may include one or more processors and/or one or more memories coupled to one or more processors. Accordingly, computerized methods consistent with one or more implementations of the current subject matter may be implemented by one or more data processors located on a single computing system or multiple computing systems comprising multiple computers, such as a computing or supercomputing bank.

[0045] Such multiple computing systems may be connected and may exchange data and/or commands or other instructions and the like. through one or more connections, including, without limitation, a connection over a network (e.g., the Internet, wireless wide area network, local area network, wide area network, wired network, physical electrical interconnection, etc.), through a direct connection between one or more multiple computer networks, etc. The memory, which may be a computer-readable storage medium, may contain, encode, store, or the like. one or more programs that cause one or more processors to perform one or more operations associated with one or more algorithms described herein.

[0046] Details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and in the description that follows. Other features and advantages of the subject matter described herein will be apparent from the description and drawings and claims. Although certain features of the presently disclosed subject matter are described for purposes of illustration in connection with an enterprise resource software system or other commercial software solution or architecture, it is understood that such features are not intended to be limiting. The claims that follow this description are intended to define the scope of the protected subject matter of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are included in and form a part of this specification, show certain aspects of the subject matter disclosed herein and, together with the description, help explain some of the principles associated with the described implementations.

[0048] In FIG. 1A depicts a sequencing platform with multiple genetic samples on it, and also depicts multiple example tiles as well as a 3D representation of the sequenced reads.

[0049] In FIG. 1B shows a representation of a flow cell with various bands represented.

[0050] In FIG. 1C depicts the bottom corner of the flow cell platform shown in FIG. 1B showing a panel of sequenced reads.

[0051] In FIG. 1D shows a virtual array of sequencing results performed on the reads shown in FIG. 1 and 2, where the reads are listed in the output column in column order.

[0052] In FIG. 1E depicts a method by which the resulting reads can be transposed from columns in column order to rows in row order.

[0053] In FIG. Figure 1F depicts the transposition of the resulting reads from columns in column order to row in row order.

[0054] In FIG. 1G shows the components of the system for performing transposition.

[0055] In FIG. 1H shows the order of transposition.

[0056] In FIG. 1I depicts an architecture for electronic transposition of sequenced data.

[0057] In FIG. Figure 2 depicts a 3-state HMM model illustrating the transposition probabilities of a transition from one state to another.

[0058] In FIG. 3A depicts a high-level view of an integrated circuit of the invention including an HMM interface structure.

[0059] In FIG. 3B illustrates the integrated circuit shown in FIG. 3A showing the features of the HMM cluster in more detail.

[0060] In FIG. 4 shows an overview of the flow of HMM-related data through the system, including software and hardware connections.

[0061] In FIG. Figure 5 shows an example of the cuff connections of an HMM cluster.

[0062] In FIG. 6 depicts a high-level view of the main functional blocks within an example HMM hardware accelerator.

[0063] In FIG. Figure 7 shows an example of the HMM matrix structure and hardware processing flow.

[0064] In FIG. 8 is an enlarged view of a portion of FIG. 2, showing the data flow and dependencies between neighboring cells when calculating the M, I and D state of an HMM within a matrix.

[0065] In FIG. Figure 9 shows examples of calculations useful for updating the states of M, I, D.

[0066] In FIG. 10 depicts the M, I, and D state update schemes, including the effect of the simplified assumptions shown in FIG. 9, which relate to transition probabilities, and the impact of sharing some adder resources M, I, D with the final adder operations.

[0067] In FIG. 11 shows the details of calculating the states M, I, D of the logarithmic region.

[0068] In FIG. 12 is a HMM state transition diagram showing the relationship between GOP, GCP and transition probabilities.

[0069] In FIG. 13 depicts a diagram for generating HMM transition probabilities and Prior values to support the general state transition diagram shown in FIG. 12.

[0070] In FIG. 14 is a simplified HMM state transition diagram showing the relationship between GOP, GCP, and transition probabilities.

[0071] In FIG. Figure 15 shows a diagram for generating HMM transition probabilities and Prior values to support a simplified state transition diagram.

[0072] In FIG. Figure 16 depicts an example of a theoretical HMM matrix and shows how such an HMM matrix can be traversed.

[0073] In FIG. 17A illustrates a method for performing a multi-area joint detection pre-processing procedure.

[0074] In FIG. 17B illustrates an example of a method for calculating a connectivity matrix such as the preprocessing procedure shown in FIG. 17A.

[0075] In FIG. 18A depicts an example of an event between two homologous sequenced regions in a read pool.

[0076] In FIG. 18B depicts the constructed reads shown in FIG. 18A, indicating the difference between the two sequences.

[0077] In FIG. 18C depicts various de Bruijn graph bubbles that can be used when performing a fast variant determination operation.

[0078] In FIG. 18D is a representation of a tree trimming function as described herein.

[0079] In FIG. 18E shows one of the bubbles shown in FIG. 18C.

[0080] In FIG. 19 is a graphical representation of an example of a cluster related to the matrix shown in FIG. 17.

[0081] In FIG. 20 is a processing matrix for performing the preprocessing procedure shown in FIG. 17A and B.

[0082] In FIG. 21 shows an example of the formation of a bubble in the de Bruijn graph in accordance with the methods shown in FIG. 20.

[0083] In FIG. Figure 22 shows an example of a variant path through an illustrative de Bruijn graph.

[0084] In FIG. Figure 23 is a graphical representation of an example sort function.

[0085] In FIG. 24 shows another example of a processing matrix for a pruned multi-region joint detection procedure.

[0086] In FIG. Figure 25 shows the joint accumulation of paired reads for two regions.

[0087] In FIG. 26 is a probability table as described herein.

[0088] In FIG. 27 shows another example of a processing matrix for a multi-area joint detection procedure.

[0089] In FIG. 28 shows a selection of candidate solutions for the joint cluster shown in FIG. 25.

[0090] In FIG. 29 shows another selection of candidate solutions for the cluster shown in FIG. 28, after the trimming function is performed.

[0091] In FIG. The 30 final candidate solutions presented are shown in FIG. 28, and their corresponding probabilities after executing the MRJD function.

[0092] In FIG. Figure 31 shows the ROC curves for the MRJD and the conventional detector.

[0093] In FIG. 32 shows the same results as FIG. 3, displayed as a function of the similarity of the reference sequence.

[0094] In FIG. 33A is an architectural example illustrating loose coupling between a CPU and an FPGA according to the present invention.

[0095] In FIG. 33B is an architectural example illustrating hard coupling between the CPU and the FPGA of the present invention.

[0096] In FIG. 34A illustrates loose coupling between a CPU and an FPGA according to the present invention.

[0097] In FIG. 34B depicts an alternative embodiment of the direct communication between the CPU and the FPGA shown in FIG. 34A.

[0098] In FIG. 35 depicts an alternative implementation of a combined CPU and FPFA package, where the two devices share a common memory and/or cache.

[0099] In FIG. 36 shows a CPU core sharing one or more memories and/or caches, the CPUs being configured to communicate with one or more FPGAs, which may also include shared or shared memories or caches.

[00100] In FIG. Figure 37 shows an example of how data is transferred throughout the system.

[00101] In FIG. 38 shows in more detail the embodiment shown in FIG. 36.

[00102] In FIG. 39 illustrates an example of a method for processing one or more jobs of the system of the present invention.

[00103] In FIG. 40A depicts a block diagram for a genomic infrastructure for local or cloud-based genomic processing and analysis.

[00104] In FIG. 40B depicts a block diagram of a cloud-based genomic processing platform for performing the BioIT analysis described herein.

[00105] In FIG. 40C depicts a block diagram of an example genomic processing and analysis pipeline.

[00106] In FIG. 40D depicts a block diagram of an example genomic processing and analysis pipeline.

[00107] In FIG. 41A is a block diagram of the local and/or cloud computing function shown in FIG. 40A, for genomic infrastructure for local or cloud-based genomic processing and analysis.

[00108] In FIG. 41B illustrates in more detail the block diagram shown in FIG. 41A, as it relates to the computing function for genomic infrastructure for local or cloud-based genomic processing and analysis.

[00109] In FIG. 41C illustrates in more detail the block diagram shown in FIG. 40, as it relates to third-party analytics functionality for genomic infrastructure for on-premises or cloud-based genomic processing and analysis.

[00110] In FIG. 42A is a block diagram illustrating a hybrid cloud configuration.

[00111] In FIG. 42B illustrates in more detail the block diagram shown in FIG. 42A, which illustrates a hybrid cloud configuration.

[00112] In FIG. 42C illustrates in more detail the block diagram shown in FIG. 42A, which illustrates a hybrid cloud configuration.

[00113] In FIG. 43A is a block diagram illustrating the primary, secondary and/or tertiary processing pipeline that is presented herein.

[00114] In FIG. 43B provides an example of a tertiary processing epigenetics analysis to be performed using the methods and apparatus of the system described herein.

[00115] In FIG. 43C provides an example of a tertiary processing methylation assay to be performed using the methods and apparatus of the system described herein.

[00116] In FIG. 43D provides an example analysis of tertiary processing design options to be performed using the methods and apparatus of the system described herein.

[00117] In FIG. 43E provides an example of a tertiary cohort analysis to be performed using the methods and apparatus of the system described herein.

[00118] In FIG. 43F provides an example of a tertiary co-genotyping processing analysis to be performed using the methods and apparatus of the system described herein.

[00119] In FIG. 44 is a block diagram of an analysis pipeline according to the invention.

[00120] In FIG. 45 is a block diagram of a processor hardware architecture in accordance with an embodiment of the invention.

[00121] In FIG. 46 is a block diagram of a processor hardware architecture in accordance with another implementation.

[00122] In FIG. 47 is a block diagram of a processor hardware architecture in accordance with yet another implementation.

[00123] In FIG. Figure 48 shows the genetic sequence analysis pipeline.

[00124] In FIG. 49 shows the processing steps using the genetic sequence analysis hardware platform.

[00125] In FIG. 50A shows a device in accordance with an embodiment of the invention.

[00126] In FIG. 50B shows another device in accordance with an embodiment of the invention.

[00127] In FIG. 51 shows a genomic processing system according to an implementation.

IMPLEMENTATION OF THE INVENTION

[00128] As summarized above, the present invention relates to devices, systems and methods for using them in performing one or more genomics and/or bioinformatics protocols, such as a mapping, alignment, sorting and/or variant detection protocol, on data formed through a primary processing procedure, for example on genetic sequence data. For example, in various aspects, the devices, systems and methods provided herein are configured to implement protocols for secondary analysis of genetic data, such as data generated by RNA and/or DNA sequencing, for example, using a next generation sequencer (NGS). In particular embodiments, one or more secondary processing pipelines are provided for processing genetic sequence data, for example, where the pipelines and/or individual elements thereof may be implemented in software, hardware, or a combination thereof with distribution and/or optimization to provide superior sensitivity and improved accuracy over a wider range of sequence data than is currently available in the art. In addition, as summarized above, the present invention relates to devices, systems and methods for using them in performing one or more tertiary genomics and/or bioinformatics protocols, such as a microarray analysis protocol, a genome analysis protocol, e.g., a genome-wide analysis, a genotyping analysis protocol , exome analysis protocol, epigenome analysis protocol, metagenome analysis protocol, microbiome analysis protocol, genotyping analysis protocol including co-genotyping, variant analysis protocols including structural variants, somatic variants, and GATK, as well as RNA sequencing protocols and other genetic analysis protocols, for example, on mapped, aligned and/or other genetic sequence data, such as the use of one or more variant definition files.

[00129] Accordingly, analysis technologies are provided herein using a software and/or hardware, eg, chip-based, accelerated platform for performing secondary and/or tertiary analysis of DNA/RNA sequencing data. More specifically, a platform, or pipeline, of processing engines, for example in a software-implemented and/or hard-wired configuration, that is specifically configured to perform secondary genetic analysis, such as mapping, alignment, sorting and/or variant detection; and/or may be specifically configured to perform tertiary genetic analysis, such as microarray analysis, genome analysis, e.g. whole genome analysis, genotyping analysis, exome analysis, epigenome analysis, metagenome analysis, microbiome analysis, genotyping analysis, including co-genotyping analysis, variant analysis, including structural variant analysis and GATK analysis, as well as RNA sequencing analysis and other genetic analyses, for example, in relation to genetic sequencing data that may have been generated in an optimized format providing improved processing speeds that are orders of magnitude faster than standard pipelines, implemented exclusively in well-known software. In addition, the pipelines presented herein provide better sensitivity and accuracy on a wide range of sequence-derived datasets, such as sequences derived from nucleic acids or proteins.

[00130] As stated above, in various cases, the purpose of bioinformatics processing is to determine individual genomes and/or protein sequences of individuals, and these determinations can be used in gene research protocols, as well as for preventive and/or therapeutic regimens to improve the functioning of everyone individual and humanity as a whole. In addition, knowledge of an individual's genome and/or protein combination can be used, for example, in drug discovery and/or FDA trials to better predict, in a specific manner, which drugs, if any, are likely to affect the individual, and /or which drugs are likely to have harmful side effects, for example by analyzing the genome and/or the resulting protein profile of the individual and comparing them with the predicted biological response to administration of such drug.

[00131] Such processing in bioinformatics typically involves three well-defined, but typically separate, phases of information processing. The first phase, called primary processing, involves DNA/RNA sequencing, where the subject's DNA and/or RNA is obtained and subjected to various treatments that convert the subject's genetic code into a machine-readable digital code, such as a FASTQ file. The second phase, called secondary processing, involves using the subject's generated digital genetic code to determine the subject's genetic makeup, such as determining the subject's genomic nucleotide sequence. And the third phase, called tertiary processing, involves performing one or more tests on the subject's genetic makeup to determine information from it that is useful for therapeutic purposes.

[00132] Accordingly, after a subject's genetic code has been sequenced, for example, using a next-generation sequencer, to obtain a machine-readable digital representation of the subject's genetic code, for example, in a FASTQ and/or BCL file format, further processing of the encoded in digital form of genetic sequence data obtained from a sequencer and/or sequencing protocol, for example, by applying secondary processing to the digitally represented data. This secondary processing, for example, can be used to map and/or align and/or otherwise assemble a whole genome and/or protein profile of an individual, for example when determining the entire genetic makeup, where every single nucleotide of each and every chromosome is determined sequentially so as to identify the composition of an individual's entire genome. In such processing, the genome of an individual can be assembled, for example, by comparison with a reference genome, such as a reference standard, for example, with one or more genomes obtained from the Human Genome Project, etc., to determine how the genetic makeup of the individual differs on the genetic composition of control individuals. This process is commonly referred to as option identification. Since DNA differences from one person to another occur once every 1000 base pairs, this process of identifying variants can be very labor-intensive and time-consuming, requiring many steps that may need to be performed one after another and/or simultaneously, for example, in a pipeline mode to analyze a subject's genomic data and determine how that genetic sequence differs from a given reference.

[00133] When performing a secondary analysis pipeline, such as to generate a variant definition file for a given sequence of interest in an individual subject, a genetic sample, such as a DNA, RNA, protein sample, may be obtained from the subject. The subject's DNA/RNA may then be sequenced, for example using a next-generation sequencer (NGS) and/or sequencer-on-a-chip technology, for example in a front-end processing step, to produce multiple sequence reads covering the entire or part of an individual's genome, for example with redundancy. The final product generated by the sequencing device may be a collection of short sequences, such as reads, that represent small segments of a subject's genome, such as short genetic sequences that represent the entire genome of an individual. As stated, the information represented by these reads may be an image file or a file in a digital format such as FASTQ, BCL or other similar file format.

[00134] Specifically, in a typical secondary processing protocol, the genetic makeup of a subject is collected by comparison to a reference genome. This comparison involves reconstructing an individual's genome from millions and millions of short read sequences and/or comparing the individual's entire DNA to an example DNA sequence model. In a typical secondary processing protocol, an image, a FASTQ file, and/or a BCL containing the raw sequenced read data is received from the sequencer . To compare a subject's genome to a standard reference genome, it is necessary to determine where each of these reads maps onto the reference genome, such as how each read aligns with the other, and/or how each read can also be sorted by chromosome order to determine what position each read is located in and which chromosome it belongs to. One or more of these functions may precede the execution of the variant determination function on the full-length sequence, for example after assembly. Namely, once it is determined which part of the genome each read belongs to, the full-length genetic sequence can be determined, and the differences between the subject's genetic code and the genetic reference code can then be assessed.

[00135] For example, reference-based assembly in a typical secondary processing assembly protocol involves comparing a subject's sequenced genomic DNA/RNA to the sequenced genomic DNA/RNA of one or more standards, such as known reference sequences. Various mapping, alignment, sorting and/or variant identification algorithms have been developed to help speed up these processes. Therefore, these algorithms may include some variation of one or more of: mapping, aligning, and/or sorting the millions of reads obtained from an image (FASTQ and/or BCL file) that are fed by the sequencer to determine the location of each specific read on each chromosome. It should be noted that these processes may be implemented in software or hardware, for example, using methods and/or devices described in US Patent Nos. 9,014,989 and 9,235,680, the rights to which are owned by Edico Genome Corporation, and which are incorporated in their entirety herein. this document by reference. Often a common feature in the operation of these various algorithms and/or hardware implementations is their use of an index and/or array to speed up their processing function.

[00136] For example, with respect to mapping, a large number of sequenced reads (eg, all) can be processed to determine possible locations in the reference genome to which those reads could be aligned. One method that can be used for this purpose is to directly compare the read to a reference genome to find all match positions. Another method is to use a prefix or suffix array or construct a prefix or suffix tree to map reads to different positions in a reference genome. A typical algorithm useful in performing this function is the Burrows-Wheeler transform, which is used to map reads to a reference using a compression formula that compresses repeated sequences of data.

[00137] Another method is to use a hash table, for example, where a selected subset of reads, k-mers of selected length "k", such as a seed, are placed in the hash table as keys, and the reference sequence is split into parts equal to is a k-measure in length, and these parts and their locations are inserted by an algorithm into the hash table at those places in the table to which they are mapped according to the hash function. A typical algorithm to perform this function is "BLAST", Basic Local Alignment Search Tool. Such hash table-based programs compare the nucleotide or protein sequences of interest to one or more standard reference sequence databases and calculate the statistical significance of the matches. In this way, the likely location of any given read relative to the reference genome can be determined. These algorithms are useful because they require less memory, transformations, lookup tables (LUTs), and therefore require less computational resources and time to perform their functions than would otherwise be the case, for example, if a subject's genome were assembled by direct comparison. for example, without using these algorithms.

[00138] Additionally, an alignment function can be performed to determine all possible mapping locations of a given read onto the genome, for example in cases where a read can be mapped to multiple positions in the genome that are actually the location from which it was actually obtained. for example, by sequencing from that location using the original sequencing protocol. This function can be performed on a number of reads (eg, mapped reads) of the genome and can produce a string of ordered nucleotide bases representing part or all of the subject's genomic DNA/RNA sequence. Along with the ordered genetic sequence, each nucleotide at a given position can be assigned a score representing, for any given nucleotide position, the probability that the nucleotide, such as "A", "C", "G", "T" (or "U"), predicted at that position is actually the nucleotide that belongs to that designated position. Typical algorithms for performing the alignment function include the Needleman-Wunsch and Smith-Waterman algorithms. In either case, these algorithms perform sequence alignments between a subject's genomic sequence string and a reference genomic sequence string, thereby comparing selected segments of possible lengths instead of comparing whole-genome sequences to each other.

[00139] Once reads have been assigned positions, for example, relative to a reference genome, which may include determining whether the read belongs to a particular chromosome and/or its offset from the start of that chromosome, the reads can be sorted by position. This may allow subsequent analyzes to take advantage of the oversampling procedures described herein. All reads that overlap a given position in the genome will be adjacent to each other after sorting and can be organized into a pileup and easily examined to determine whether most of them agree with the reference value or not. If not, the option can be flagged.

[00140] For example, in various embodiments, the methods of the present invention may include generating a variant definition file (VCF) identifying one or more (e.g., all) genetic variants in an individual whose DNA/RNA has been sequenced, for example, in accordance with one or more reference genomes. For example, once the actual genome of a sample is known and compared to a reference genome, variations between the two genomes can be determined and a list of all variations/deviations between the reference genomes and the sample genome can be generated, for example a variant definition file can be created. In particular, in one aspect, a variant definition file may be generated containing all variations in a subject's genetic sequence relative to reference sequences.

[00141] As stated above, such variations between two genetic sequences can be due to a number of reasons. Therefore, to generate such a file, the subject's genome must be sequenced and rebuilt before identifying its variants. However, there are several problems that may arise when attempting to form such an assembly. For example, there may be problems with the chemistry, the sequencer, and/or human errors that occur during the sequencing process. In addition, there may be genetic artifacts that make such reconstruction problematic. For example, a typical problem in performing such assemblies is that there are sometimes huge portions of the genome that repeat themselves, such as long sections of the genome that include the same strings of nucleotides. Consequently, since any genetic sequence is not unique everywhere, it may be difficult to determine where in the genome an identified read is mapped and aligned. In addition, single nucleotide polymorphisms (SNPs) are possible, for example where one base in a subject's genetic sequence has been replaced by another; more extensive substitutions of multiple nucleotides are possible; an insertion or deletion is possible, such as when one or more bases are added to or removed from a subject's genetic sequence; and/or there may be a structural variant, such as one caused by the crossing of two chromosome legs, and/or there may simply be a misalignment resulting in a shift in sequence.

[00142] Accordingly, there are two possibilities for variation. First, there is actual variation at that particular location, for example, when the human genome at a particular location actually differs from the reference, for example, there is natural variation due to a SNP (single base substitution), insertion or deletion (one or more nucleotides long) , and/or there is a structural variant, such as when DNA material from one chromosome crosses over another chromosome or stem, or when a particular region occurs twice in the DNA. Or the variation may be caused by a problem in the read data, due to chemistry or machine, sequencer or aligner, or other human error. The methods described herein can be used to compensate for these types of errors and, in particular, to distinguish errors in variation due to chemistry, machine or human, from actual variation in the sequenced genome. More specifically, the systems, devices, and methods for using them described herein are designed to clearly distinguish between these two different types of variations and, therefore, to better ensure the accuracy of any generated variation files to correctly identify true variations.

[00143] Therefore, specific embodiments provide a platform of technologies for performing genetic analyses, where the platform may include performing one or more of the functions of: mapping, alignment, sorting, local realignment, duplicate flagging, base quality assessment recalibration, variant detection, compression and/or decompression. For example, in accordance with various aspects, a pipeline may be provided that includes performing one or more analytical functions, as described herein, on the genomic sequence of one or more individuals, for example, on data obtained in an image file and/or digital file in FASTQ or BCL format from an automated sequencer. A typical pipeline to be performed may include one or more sequencing of genetic material, such as part or all of a genome, of one or more subjects, wherein the genetic material may comprise DNA, ssDNA, RNA, rRNA, tRNA, and the like. and/or, in some cases, the genetic material may represent encoded or non-encoded regions, such as exomes and/or episomes of DNA. The pipeline may include one or more of performing an image processing procedure, a base determination operation, and/or error correction, such as in digitized genetic data, and/or may include one or more of performing a mapping, alignment, and/or sorting function on genetic data. In certain cases, the pipeline may include performing one or more of realignment, deduplication, base quality assessment recalibration, reduction, and/or compression and/or decompression on the digitized genetic data. In certain cases, the pipeline may include running a variant detection operation, such as a Hidden Markov Model, on the genetic data.

[00144] Accordingly, in certain cases, an implementation of one or more of these platform functions is designed to perform one or more of determining and/or reconstructing a consensus genomic sequence of a subject, comparing the genomic sequence of the subject with a reference sequence, e.g., a reference or model genetic sequence, determining how a subject's genomic DNA or RNA differs from a reference, e.g., identifying variants, and/or to perform tertiary analysis on the subject's genomic sequence, e.g., genome-wide variation analysis, gene functional analysis, protein functional analysis, e.g. protein binding, as well as for various diagnostic and/or prophylactic and/or therapeutic evaluation assays.

[00145] As stated above, in accordance with one aspect, one or more of these platform functions, e.g., mapping, alignment, sorting, re-alignment, duplicate marking, base quality assessment recalibration, variant detection, compression and/or decompression functions, are performed with the possibility of implementation in software. In accordance with some aspects, one or more of these platform functions, e.g., mapping, alignment, sorting, local realignment, duplicate marking, ground quality assessment recalibration, decompression, variant detection, compression, and/or decompression functions, are configured to be implemented in hardware, such as firmware. In accordance with certain aspects, these genetic analysis technologies may use improved algorithms that may be implemented in software that execute with less processing intensity and/or less time and/or a higher percentage of accuracy, such as hardware-implemented functionality faster, require less intensive processing and more accurate.

[00146] For example, certain embodiments provide improved algorithms for performing such primary, secondary, and/or tertiary processing as described herein. Improved algorithms aim to more efficiently and/or more accurately perform one or more of the functions of mapping, alignment, sorting and/or variant detection, for example, on an image file and/or digital representation of DNA/RNA sequence data obtained from a sequencing platform, e.g. , in FASTQ or BCL file format obtained from an automated sequencer such as one described above. In specific embodiments, improved algorithms may be designed to more efficiently and/or more accurately perform one or more of the functions of local realignment, duplicate flagging, base quality score recalibration, variant detection, compression, and/or decompression. Additionally, as described in more detail below herein, in certain embodiments, these genetic analysis technologies may use one or more algorithms, such as enhanced algorithms, that may be implemented using one or more of software and/or hardware, which are performed with less processing intensity, and/or less time, and/or a higher percentage of accuracy than various traditional software implementations for doing the same. In various cases, improved algorithms are provided for implementation on a quantum processing platform.

[00147] Therefore, in accordance with various aspects, systems, devices and methods are provided herein for implementing bioinformatics protocols, such as for performing one or more genetic data analysis functions, such as genomic data, for example, through one or more optimized algorithms and/ or one or more integrated and/or quantum circuits, for example on one or more hardware processing platforms. In one case, systems and methods are provided for implementing one or more algorithms, for example, in software and/or firmware, and/or using a quantum processing circuit, to perform one or more steps of analyzing genomic data in bioinformatics protocols, for example, when the steps may include performing one or more of: mapping, alignment, sorting, local realignment, duplicate labeling, ground quality assessment recalibration, variant identification, compression and/or decompression; and may also include one or more steps on a tertiary processing platform. Accordingly, in certain cases, methods are provided herein that include software, firmware, hardware, and/or quantum processing algorithms for performing the methods, where the methods include executing an algorithm, such as an algorithm for implementing one or more genetic analysis functions such as mapping, alignment, sorting, re-alignment, duplicate marking, base quality assessment recalibration, variant detection, compression, decompression, and/or one or more tertiary processing protocols, where the algorithm, for example including firmware, is optimized according with the way in which it should be implemented.

[00148] In particular, when an algorithm is to be implemented in a software solution, the algorithm and/or its supporting processes are optimized so that they run faster and/or with higher accuracy when executed by that environment. Likewise, when the functions of an algorithm must be implemented in a hardware solution, such as firmware, the hardware is designed to perform those functions and/or the processes that serve them in an optimal manner to perform faster and/or with higher accuracy when executed by that environment. Additionally, when an algorithm is to be implemented in a quantum processing solution, the algorithm and/or its supporting processes are optimized so that they run faster and/or with higher accuracy when executed by that environment. These methods, for example, can be used, for example, in iterative mapping, alignment, sorting, variant determination and/or tertiary processing procedures. Alternatively, systems and methods are provided for implementing the functions of one or more algorithms for performing one or more steps of genomic data analysis in a bioinformatics protocol as defined herein, the functions being implemented on a hardware and/or quantum accelerator, which may or may not be connected. coupled to one or more general purpose processors and/or supercomputers and/or quantum computers.

[00149] More specifically, in some cases, methods and/or equipment are provided for implementing these methods for the purpose of performing secondary analytics on data related to the genetic makeup of a subject. In one case, the analytics to be performed may include a reference-based reconstruction of the subject's genome. For example, reference-based mapping involves the use of a reference genome, which may be generated by sequencing the genome of one or more individuals, or it may be a pool of DNA/RNA belonging to different individuals that are combined to create a prototypic standard reference genome. against which the genetic material, such as DNA/RNA, of any individual can be compared, for example, to determine and reconstruct the individual's genetic sequence and/or to determine the difference between their genetic makeup and this standard reference, for example, to identify variants.

[00150] Specifically, the reason for performing a secondary analysis on a subject's sequenced DNA/RNA is to determine how the subject's DNA/RNA differs from the reference DNA/RNA to determine one, many, or all differences in the subject's nucleotide sequence from the reference nucleotide sequence . For example, differences between the genetic sequences of any two randomly selected individuals occur once in about 1,000 base pairs, which, given the over 3 billion base pairs in the entire genome, amounts to a variation of up to 3,000,000 different base pairs per person. Determining these differences may be useful, for example, in a tertiary analysis protocol, for example, to predict the possibility of a disease state, for example, due to a genetic disorder, and/or the likelihood of success of a prophylactic or therapeutic intervention, for example, based on how the prevention or treatment is expected to interact therapy with the subject's DNA or proteins formed in this process. In various cases, it may be useful to perform either de novo or reference-based reconstruction of a subject's genome to reinforce the results of one with those of the other, and to improve the accuracy of the variant detection protocol, if required.

[00151] Accordingly, according to one aspect, in various embodiments, after a subject's genome has been reconstructed and/or a VCF file has been generated, such data may then be subjected to tertiary processing for the purpose of interpretation, for example, to determine what the data means with in terms of identifying diseases that the subject may or may not be exposed to, and/or to identify therapies or lifestyle changes that the subject may wish to take advantage of in order to eliminate and/or prevent the disease state. For example, a subject's genetic sequence and/or its variant definition file may be analyzed to determine clinically relevant genetic markers that indicate the presence or potential of a disease condition and/or the effectiveness with which a recommended therapeutic or prophylactic regimen may affect the subject. These data may then be used to provide the subject with one or more therapeutic or prophylactic regimens in order to improve the subject's quality of life, eg, treat and/or prevent a disease state.

[00152] In particular, once one or more genetic variations of an individual have been determined, such variant definition file information may be used to prepare medically useful information, which in turn may be used to determine, e.g. using known statistical analysis models, health-related data and/or medically useful information, e.g., for diagnostic purposes, e.g., diagnosing a disease or its potential, clinical interpretation (e.g., finding markers that represent a disease variant), determining whether a subject should be included in or excluded from various clinical trials, and other such purposes. More specifically, in various cases, the generated genomics and/or bioinformatics data may be used to perform one or more tertiary genomics and/or bioinformatics protocols, such as a microarray protocol, a genomic analysis protocol, e.g., a genome-wide analysis, a genotyping analysis protocol , exome analysis protocol, epigenome analysis protocol, metagenome analysis protocol, microbiome analysis protocol, genotyping analysis protocol including co-genotyping, variant assay protocols including structural variants, somatic variants, and GATK, as well as RNA sequencing protocols and other genetic assay protocols.

[00153] Because there are a finite number of disease states that are caused by genetic disorders, in tertiary processing, variants of a particular type, for example, known to be associated with disease conditions, can be refined, for example, by determining whether one or more genetic markers of the disease into the subject's variant definition file. Therefore, in various cases, the methods described herein may include analyzing, for example, scanning, VCF and/or generated sequence for known disease-associated sequence variants, for example, present for this reason in a genomic marker database, to identify presence of a genetic marker in the VCF and/or generated sequence, and if present, check for the presence or possibility of a genetically determined disease state. Because there are a huge number of known genetic variations and a huge number of individuals suffering from diseases caused by such variations, in some embodiments, the methods described herein may include the formation of one or more databases linking whole genome sequenced data and/or associated with them a file identifying variants, for example, from one or multiple individuals with a disease condition, and/or searching generated databases to determine whether a particular subject has a genetic makeup that predisposes him to having such disease condition. Such a search may involve comparing one complete genome with one or more others, or a fragment of a genome, such as a fragment containing only variations, with one or more fragments of one or more other genomes, for example, in a database of reference genomes or fragments thereof.

[00154] Therefore, in various cases, the pipeline of the present invention may comprise one or more modules, where the modules are configured to perform one or more functions, such as image processing or base detection, and/or error correction operation, and/or mapping, and /or alignment, eg gapped and non-gapped alignment, and/or genetic data sorting function, eg sequenced genetic data. And in various cases, the pipeline may comprise one or more modules, where the modules are configured to perform one or more of local re-alignment, duplicate removal, ground quality score recalibration, variant detection, e.g., HMM, reduction, and/or decompression on genetic data. In addition, the pipeline may comprise one or more modules, where the modules are configured to implement a tertiary analysis protocol, such as microarray analysis protocols, genome analysis protocols, e.g., whole genome analysis, genotyping analysis protocols, exome analysis protocols, epigenome analysis protocols, analysis protocols metagenome, microbiome analysis protocols, genotyping analysis protocols including co-genotyping analysis protocols, variant analysis protocols including co-genotyping, variant analysis protocols including structural variants, somatic variants, and GATK protocols, as well as RNA sequencing and other genetic analysis protocols.

[00155] Many of these modules may be executed by either software or hardware, locally or remotely, for example through software or hardware, say, on a cloud, for example, on a remote server and/or a bank of servers, such as a quantum computing cluster. Additionally, many of these modules and/or pipeline stages are optional and/or can be arranged in any logical order and/or omitted entirely. For example, the software and/or hardware described herein may or may not include image processing and/or base detection or a sequence correction algorithm, for example where there may be concerns that such functions may lead to statistical bias error. Therefore, the system may or may not include base detection and/or sequence correction functionality, respectively, depending on the level of accuracy and/or efficiency required. And, as stated above, one or more functions of the pipeline may be used in generating the genomic sequence of a subject, for example, by reconstructing the genome from a reference. Additionally, in certain cases, the output from the secondary processing pipeline may be a variant definition file (VCF, gVCF) indicating partial or complete variants in or part of the genome.

[00156] In particular, once reads have been assigned positions relative to a reference genome, which may include determining which chromosome the read belongs to and/or its offset from the start of that chromosome, duplicates can be removed from them and/or sorted, for example by position. This allows subsequent analyzes to take advantage of the various oversampling protocols described herein. All reads that overlap a given position in the genome can be located next to each other after sorting, and they can be accumulated, for example to form a cluster, and easily examined to determine whether most of them agree with the reference value or not. If not, as stated above, the option can be checked.

[00157] Accordingly, as discussed above with respect to mapping, the image file, BCL file and/or FASTQ file obtained from the sequencer consists of many, for example millions to billions or more, reads consisting of short strings of nucleotide sequence data, representing part and or the entire genome of an individual. For example, the first step in the secondary analysis pipelines described herein is the ingestion of genomic and/or bioinformatics data, for example, from a genomic data generating device such as a sequencer. Typically, the data produced by a sequencer, such as a next-generation sequencer, may be in a BCL file format, which in some cases may be converted to a FASTQ file format, before or after transfer to, for example, a secondary processing platform described herein. In particular, when sequencing the human genome, it is necessary to identify a subject's DNA and/or RNA, base by base, and the result of such sequencing is a BCL file. A BCL file is a binary file that contains base definitions and quality scores made for each base of each sequence of a collection of sequences that constitutes at least a portion or the entire genome of a subject.

[00158] Traditionally, the BCL file generated by the sequencer is converted into a FASTQ file, which can then be transferred to a secondary processing platform, such as the one described herein, for further processing, for example, to determine genomic variation from it. A FASTQ file is a text format file for transmitting and storing both a biological sequence (such as a nucleotide sequence) and its corresponding quality scores, where each letter of the sequence, such as A, C, G, T and/or U, and the quality score may be encoded with a single ASCII character for brevity. Accordingly, in this and other systems, it is the FASTQ file that is used for further processing. Although using a FASTQ file for genomic processing is useful, converting the generated BCL file to a FASTQ file, which is implemented in the sequencer, is time-consuming and inefficient. Therefore, in accordance with one aspect, devices and methods are provided for directly converting a BCL file to a FASTQ file and/or directly inputting such data into pipelines of the present platform, as described herein.

[00159] For example, in various embodiments, a next-generation sequencer or sequencer-on-a-chip technology may be configured to perform a sequencing operation on incoming genetic data. For example, as shown in FIG. 1A, the genetic data 6a may be linked to a platform 6 for insertion into a next-generation sequencer for iterative sequencing such that each sequence is built up by incrementally adding one nucleotide after another. Namely, the sequencing platform 6 may contain a number of template nucleotide sequences 6a of a subject, which are arranged in a grid pattern to form tiles 6b on the platform 6, the template sequences 6a of which are to be sequenced. The platform 6 may be added to a flow cell 6c of a sequencer that is configured to perform sequencing reactions.

[00160] As sequencing reactions occur, at each step a nucleotide having a fluorescent tag 6d is added to the platform 6 of the flow cell 6c. If a hybridization reaction occurs, fluorescence is observed and an image is taken, then the image is processed and the appropriate base determination is made. This is repeated one base after another until all the template sequences, such as the entire genome, have been sequenced and converted into reads, thereby creating the system's read data. Thus, upon completion of sequencing, the generated data, such as reads, must be transferred from the sequencing platform to the secondary processing system. For example, this image data is typically converted into a BCL and/or FASTQ file, which can then be transferred to the system.

[00161] However, in various cases this conversion and/or transfer process can be made more efficient. Specifically, this document presents methods and architectures for rapidly converting a BCL file into files that can be quickly processed by a secondary processing system. More specifically, in specific cases, instead of transferring raw BCL or FASTQ files, generated images representing each tile of a sequencing operation can be transferred directly into the system and prepared for mapping, alignment, etc. For example, tiles can be streamed through a suitably configured PCIe card to an ASIC, FPGA, or CPU, where read data can be directly extracted from them and the reads promoted to mapping and alignment engines and/or other processing engines.

[00162] In particular, with regard to transferring data from tiles obtained by the sequencer to the FPGA/CPU/GPU/CPU, as shown in FIG. 1A, the sequencing platform 6 can be depicted as a 3-dimensional cube 6c within which growing sequences 6a can be generated. Essentially, as shown in FIG. 1B, the sequencing platform 6 may consist of 16 lanes, 8 in the front and 8 in the back, which can be configured to form about 96 tiles 6b. Within each tile 6b are a plurality of template sequences 6a to be sequenced to thereby form reads, each representing a nucleotide sequence for a given region of the subject's genome, each column representing one file, and digitally encoded representing 1 byte for each file, 8 bits each per file, for example, where 2 bits represent the base found and the remaining 6 bits represent the quality score.

[00163] More specifically, with regard to next generation sequencing, it is typically performed on glass plates 6 forming flow cells 6c that are introduced into an automated sequencer for sequencing. As shown in FIG. 1B, flow cell 6c is a platform 6 consisting of 8 vertical columns and 8 horizontal rows (front and back), together forming 16 lanes, where each lane is sufficient for sequencing a complete genome. The subject's DNA and/or RNA 6a to be sequenced is linked at designated positions between the fluid-impermeable intersections of the columns and rows of the platform 6 to form tiles 6b, where each tile contains the template genetic material 6a to be sequenced. Therefore, the sequencing platform 6 contains a plurality of subject template nucleotide sequences, with the sequences arranged in a grid of tiles on the platform (see FIG. 1B). The genetic data 6 is then sequenced in an iterative manner, with each sequence being incrementally introduced one nucleotide at a time into the flow cell, with each incremental step representing a sequencing cycle.

[00164] As stated, after each step, an image is obtained, and the growing sequence, for example, of images, forms the basis with which the BCL file is generated. As shown in FIG. 1C, reads from a sequencing run can form clusters, and it is these clusters that form the theoretical 3-dimensional cube 6c. Accordingly, within this theoretical 3-dimensional cube, each base of each growing nucleotide strand that is sequenced will have an x dimension and a y dimension. Image data, or tiles 6b, from this 3-dimensional cube 6c can be extracted and assembled into a 2-D map from which a matrix can be formed, as shown in FIG. 1D. This matrix is formed from sequencing runs, which represent the horizontal axis, and read IDs, which represent the vertical axis. Accordingly, as shown in FIG. 1C, the sequenced reads form clusters in the flow cell 6c, where the clusters can be defined using a vertical and horizontal axis, run by run and base by base, and data from each run for each read can be inserted into the matrix shown in FIG. 1D, for example, in streaming and/or pipeline mode.

[00165] Namely, each cycle represents the potential gain of each read in the flow cell by the addition of a single nucleotide, which when sequencing one or more human genomes can represent a gain of about 1 billion or more reads per lane. The increment of each read, for example due to the addition of a nucleotide base, is identified by iteratively imaging the tiles 6b of the flow cell 6c between incremental steps. Using these images, bases are determined, quality scores are determined, and the virtual matrix shown in FIG is formed. 1D. Accordingly, both the base definition and the quality score will be entered into the matrix, with each tile from each cycle representing a separate file. It should be noted that when sequencing is performed on an integrated circuit, the electronic readout data may replace the image data.

[00166] For example, as shown in FIG. 1D, the matrix itself will iteratively grow as images are acquired and processed, bases are determined, and scores are determined for each read, cycle by cycle. This is repeated for each base in the read for each flow cell tile. For example, the clusters of reads shown in FIG. 1C can be numbered and entered into the matrix as a vertical axis. Likewise, the cycle number can be entered as the horizontal axis, and then the base definition and quality rating can be entered to fill the matrix column by column, row by row. Accordingly, each read will be represented by a number of bases, for example from about 100 or 150 to 1000 or more bases per read depending on the sequencer, and there may be up to 10 million or more reads per tile. So if there are about 100 tiles, each with 10 million reads, the matrix will contain about 1 billion reads that need to be organized and streamed to the secondary processor.

[00167] Accordingly, such an organization is fundamental to fast and efficient data processing. Therefore, in accordance with one aspect, methods are provided herein for transporting data represented by a virtual sequencing matrix such that the data can be efficiently streamed into pipelines of a system described herein. For example, generating sequencing data that is represented by the star-shaped cluster depicted in FIG. 1C are largely unorganized, which creates difficulties in terms of data processing. In particular, as data is generated by a sequencing operation, it is organized into one file per run, which means that upon completion of the sequencing operation, millions and millions of files are generated, as shown in FIG. 1E with data in columns delimited by solid lines. However, for purposes of secondary and/or tertiary processing as described herein, the file data must be reorganized into the read data delimited by the dotted lines in FIG. 1E.

[00168] More specifically, to more efficiently stream sequencer-generated data to secondary data processing, the data represented by the virtual matrix should be rearranged, for example, by reorganizing the file data from a column-by-column construction of tiles in each cycle to a row-by-row construction, identifying the bases each read. Namely, the data structure of the generated files that form a matrix, as they are created by the sequencer, is organized cycle by cycle, column by column. Using the processes described herein, this data can be rearranged, for example, substantially simultaneously so that it is represented, as shown in a virtual matrix, read by read, row by row, where each row represents a separate read and each read is represented by the serial number of base definitions and quality scores, thus identifying both the sequence for each read and its reliability. Thus, in a permutation operation as described herein, data in memory may be reorganized, for example, in a virtual matrix, from a column-by-column arrangement representing the data input order to a row-by-row arrangement representing the data output order, changing the the most order of data from a vertical organization to a horizontal organization. Additionally, while this process can be implemented efficiently in software, it can be done even more efficiently and quickly by implementing it in hardware and/or using a quantum processor.

[00169] For example, in various cases this rearrangement process can be accelerated by implementation in hardware. For example, in one implementation, in a first step, software on a host device, such as a sequencer, may write input data to memory associated with the FPGA, column by column, such as in input order. Namely, as data is generated and stored in associated memory, the data can be organized into files, cycle by cycle, where the data is stored as its own individual files. This data can be represented by the 3-cube shown in FIG. 1A. This generated data, which is organized into columns, can then be queued and/or streamed, for example in real time, to hardware where specialized processing engines will queue up the columnarized data and rearrange that data from column-by-column configuration in loop order to a row-by-row configuration in read order in a manner as described above herein, for example by converting 3-dimensional tile data into a 2-dimensional matrix, thereby column data can be reorganized into row data , for example, read by read. This rearranged data can then be stored in memory in a more strategic order.

[00170] For example, host software may be configured to write input data to a memory associated with a chip, such as an FPGA, for example, column by column in input order, and like hardware may be configured to queue the data such such that they are read into memory in a strategic manner, for example, as shown in FIG. 1F. Specifically, the hardware may include an array of registers 8a into which loop files can be distributed and reorganized into individual read data, for example by writing one radix from a column to registers that are organized into rows. More precisely, as shown in FIG. 1G, a hardware device 1 including a permutation processing engine 8 may include a DRAM port 8a that can queue data to be swapped, the port being operatively coupled to a memory interface 8b that is coupled to a plurality of registers and/or external memory 8c, and is configured to process an increased number of transactions per cycle, where queued data is transmitted in batches.

[00171] In particular, this shuffling may occur one data segment at a time, for example when the memory access queue is organized to take full advantage of DDR transfer rates. For example, in the case of DRAM, this means that the minimum length of a DDR packet can be, for example, 64 bytes. Accordingly, columnarized data stored in main memory can be accessed in such a way that each memory access results in a column with a total size corresponding to, for example, 64 bytes of data. Therefore, in a single memory access, it is possible to access a portion of a tile, for example representing the corresponding "64" loops or files, column by column.

[00172] However, as shown in FIG. 1F, although data in main memory is available as column data, when transferred to hardware, it can be loaded into associated smaller memories, such as registers, in a different order, and thereby the data can be converted into bytes, for example, 64 bytes , the data is read row by row, for example, in accordance with the minimum transfer rate of the DDR packet, to form the corresponding "64" memory units or blocks for each memory access. This is illustrated by the example of the virtual matrix shown in FIG. 1D, where multiple reads, for example 64, are accessed block by block and read into memory in the form of segments, as shown in FIG. 1E, for example, when there is a specific read per register or flip-flop, for example 64 cycles × 64 reads × 8 bits per read = 32K flip-flops. Namely, this can be achieved in all sorts of different ways in hardware, such as when the input data record is organized in column order and the output data record is organized in row order. Therefore, in this configuration, the hardware can be configured to read from and/or write to "64" different addresses per cycle.

[00173] More specifically, the hardware may be coupled to an array of registers such that each read base is directed to and written to a single register (or multiple registers arranged in a row), for example, when each block is completed, the row's reordered data may be transferred into memory as output data, such as FASTQ data, line by line. The FASTQ data may then be accessed by one or more further processing engines of the secondary processing system for further processing, for example, using a mapping, alignment and/or variant detection engine as described herein. It should be noted that, as described herein, the rearrangement is performed in small blocks, however, depending on the circumstances, the system may also be configured to process larger blocks.

[00174] As stated, after the BCL file is converted to a FASTQ file as described above, and/or the BCL or FASTQ file is otherwise received by the secondary processing platform, a mapping operation can be performed on the received data. Mapping typically involves mapping reads to all locations in the reference genome where there is a match. For example, depending on the size of the read, there may be one or many locations where the read substantially matches the corresponding sequence in the reference genome. Therefore, the mapping and/or other functions described herein can be performed to determine which of all possible locations where one or more reads may match a reference genome is actually the true location to which they are mapped.

[00175] For example, in various cases, an index of a reference genome may be generated or otherwise provided so that reads or portions of reads, for example in a lookup table (LUT), can be searched by reference to the index, thereby retrieving location locators in the mapping reference reads on the reference. Such a reference index can be constructed in various forms and accessed in various ways. In some methods, the index may contain a tree of prefixes and/or suffixes. In particular methods, the index may be obtained from the reference using the Burrows-Wheeler transform. Thus, as an alternative to or in addition to using a tree of prefixes or suffixes, a Burrows-Wheeler transform can be performed on the data. For example, the Burrows-Wheeler transform can be used to store a tree data structure, abstractly equivalent to a tree of prefixes and/or suffixes, in a compact format, such as the space allocated for storing a reference genome. In various cases, the data is not stored in a tree structure, but rather the reference sequence data is presented in a linear list, which can be scrambled into a different order to transform it in a very special way so that the accompanying algorithm allows reference searching by reference to the sample reads. to essentially move around the "tree".

[00176] Additionally, in various cases, the index may contain one or more hash tables, and the methods described herein may include a hash function that can be performed on one or more portions of the reads for the purpose of mapping the reads to a reference , for example, to the reference index. For example, as an alternative to or in addition to using one or both of a prefix/suffix tree and/or a Burrows-Wheeler transform on the reference genome and a subject's sequence data to find locations where they map to each other, another such method includes creating a hash table index and/or executing a hash function. A hash table index can be a large reference structure that is built from reference genome sequences, which can then be compared to one or more parts of the read to determine where they might match each other. Similarly, a hash table index can be built from parts of the reads, which can then be compared to one or more reference genome sequences and thereby used to determine where they may match each other.

[00177] Implementing a hash table is a fast way to perform pattern matching. Each search takes a nearly constant amount of time to complete. This method can be contrasted with the Barroos-Wheeler method, which may require many samples (the number may vary depending on how many bits are required to find a unique pattern) per sequence of interest to find a match, or the binary search method, which requires log ₂ (N ) samples, where N is the number of seed samples in the table. Moreover, even though a hash function can break a reference genome into segments of seeds of any given length, e.g., 28 base pairs, and can then convert the seeds into a digital, e.g., binary, 56-bit representation, access all 56 bits simultaneously or the same way is not required. For example, the hash function may be implemented such that the address for each seed is designated by a number of less than 56 bits, such as about 18 to about 44 or 46 bits, such as about 20 to about 40 bits, such as about 24 to about 36 bits, including about 28 to about 32 or 30 bits can be used as the initial key or address to access the hash table. For example, in certain cases, about 26 to about 29 bits may be used as the primary access key for a hash table, leaving about 27 to about 30 that can be used as a means to double check the first key, such as if the first and the second keys arrive at the same time in the same cell in the hash table, after which it is quite obvious that the specified location is the place where they belong.

[00178] For example, a first portion of the digital seed, for example about 26 to about 32, say about 29 bits, may form the primary access key and be hashed and searched for in a first step. And in a second step, the remaining about 27 to about 30 bits, such as a secondary access key, can be inserted into a hash table, such as a hash chain, as a means to verify the first pass. Accordingly, for any seed, its initial address bits can be hashed in the first stage, and the secondary address bits can be used in the second stage, the confirmation stage. In such a case, the first portion of the seeds may be inserted at the primary record location, and the second portion may be inserted into the table at the location of the second record chain. And, as stated above, in various cases these two different record locations may be positionally separated, such as by a chained format record.

[00179] In specific cases, brute-force linear scanning can be used to compare a reference with a read or its parts. However, using a brute-force linear search to scan the reference genome for locations where the primer matches may have to examine over 3 billion locations. Such a search may be performed in accordance with the methods described herein in software or hardware. However, using the hashing-based approach as set forth herein, each seed search may take an approximately constant amount of time. Often, a location can be revealed in several, for example, one, memory accesses. However, in cases where multiple seeds map to the same location in the table, for example they are not entirely unique, several additional memory accesses may be made to find the current seed being searched. Therefore, even though there may be 30M or more possible locations where a 100-bp read matches the reference genome, the hash table and hash function can quickly determine where in the reference genome that read might appear. Therefore, when using a hash table index to determine where reads are mapped and aligned, there is no need to search for the entire reference genome, for example by brute force.

[00180] In view of the above, any suitable hash function can be used for this purpose, however, in various cases, the hash function used to determine the table address for each seed may be a cyclic redundancy check (CRC), which may be based on on a 2k-bit primitive polynomial as above. An alternative implementation could use a trivial hash function matcher, for example by simply discarding some of the 2k bits. However, in various cases, CRC may be a stronger hash function that can better separate similar seeds while avoiding table overflow. This may be particularly useful in the absence of a speed penalty when calculating the CRC, as is the case with the specialized software described in this document. In such cases, the hash entry populated for each seed may include the reference position where that seed occurs and a flag indicating whether it was back-complemented before hashing.

[00181] The result returned after executing the mapping function may be a list of candidate locations where one or more, for example, each read is mapped to one or more reference genomes. For example, the output of each mapped read could be a list of possible locations where the read could be mapped to a matching sequence in the reference genome. In various embodiments, an exact reference match may be searched for at least a portion, such as a seed read, if not the entire read. Accordingly, in various cases, an exact match of all parts of all reads with all parts of the reference genome is not required.

[00182] As described herein, all of these operations may be performed in software or may be implemented in hardware, for example, in an integrated circuit such as a chip, for example as part of a printed circuit board. For example, the functions of one or more algorithms may be built into a chip, such as an FPGA (Field Programmable Gate Array) or ASIC (Application Application-Specific Integrated Circuit) circuit, and may be optimized to operate more efficiently by implementing them in such hardware. In addition, one or more, e.g., two or all three, of these mapping functions may form a module, such as a mapping module, which may form part of a system (e.g., a pipeline) used in the process of determining the actual complete genomic sequence of an individual or its parts.

[00183] The advantage of implementing a hash module in hardware is that processes can be accelerated and therefore run much faster. For example, when software may include various instructions to perform one or more of these various functions, implementation of such instructions often requires storing and/or recalling and/or reading and/or interpreting the data and instructions, for example, before execution. However, as stated above and described in detail herein, it is possible to hardwire a chip to perform these functions without having to call, interpret, and/or execute one or more of the instruction sequences. Rather, the chip can be wired to perform such functions directly. Accordingly, in various aspects, the invention relates to a custom hard-wired machine that can be configured such that the mapping, e.g., hashing module described above is partially or fully implemented by one or more network circuits hard-wired on a chip, such as an FPGA or an ASIC circuit.

[00184] For example, in various cases, construction of the hash table index and execution of the hash function may be performed on-chip, and in other cases, the hash table index may be generated off-chip, such as by software executed by the host CPU, but after generation the index is loaded into or otherwise made available to the hardware and used by the chip, for example during operation of the hash module. In particular, in various cases, a chip such as an FPGA may be configured to be hard coupled to a host CPU, for example, through a low latency interconnect such as a QPI interconnect. More specifically, the chip and the CPU may be configured to hardwire a connection together so as to share one or more memory resources, such as DRAM, in a cache coherent configuration. In such a case, the main memory may construct and/or include a reference index, such as a hash table, that may be stored in the main memory but be readily accessible to the FPGA array, such as for use when performing a hashing or other mapping function. In particular implementations, one or more of the CPU and FPGA may contain one or more caches or registers that may be connected together to form a coherent configuration such that data stored in one cache can be substantially duplicated by another cache.

[00185] Accordingly, in view of the above, at run time, one or more pre-built hash tables, for example containing an index of a reference genome, or a hash table being created or to be created, may be loaded into on-chip memory or may at least be made available for the main application, as described in more detail later in this document. In such a case, reads, for example stored in the FASTQ file format, can be sent by the host application to built-in processing engines, such as memory or cache or other register associated with them, for example, to use mapping and/or alignment engines and/ or sorting, for example, where their results can be sent to and used by the variant determination function. In this regard, as stated above, in various cases, a cluster of overlapping primers can be generated, for example, using a primer generation function, and extracted from the sequenced reads, or pairs of reads, and after the primers are formed, they can be hashed, for example, relative to index, and search the hash table to identify candidate read mapping positions in the reference.

[00186] More specifically, a mapping module may be provided in various cases, for example, when the mapping module is configured to perform one or more mapping functions, for example, in a hard-wired configuration. Namely, the hard-wired mapping module may be configured to implement one or more functions typically performed by one or more algorithms running on a CPU, for example, functions typically implemented in a software-based algorithm that generates a prefix tree and/or suffixes, performs a Burrows-Wheeler transform, and/or performs a hash function, such as a hash function that uses or otherwise relies on indexing a hash table of, say, a reference, such as a reference genomic sequence. In such cases, the hash function may be structured to implement a strategy, such as an optimized mapping strategy, which may be implemented to minimize the number of memory accesses performed, such as large DMAs, thereby maximizing improve the utilization of on-chip or otherwise associated memory bandwidth, which may be significantly limited, such as by space within the chip architecture.

[00187] Additionally, in certain cases, to make the system more efficient, the main CPU/GPU/CPU may be hard coupled to associated hardware, such as an FPGA, for example, through a low latency interface such as a Quick Path Interconnect (“ QPI") to allow integrated circuit processing engines to easily access main memory. In specific cases, the interconnect between the main CPU and the connected chip and its associated associated memories, such as one or more DRAM devices, may be configured to maintain cache coherence. Thus, in various embodiments, an integrated circuit may be provided that is pre-configured, e.g., pre-wired, to include one or more digital logic circuits, which may be in a mounting configuration and may be interconnected, e.g. via a plurality of physical electrical interconnections, and in various embodiments, hard-wired digital logic circuits may be organized into one or more processing engines to form one or more modules, such as a mapping module.

[00188] Accordingly, in various cases, a mapping module may be provided, for example, in a first pre-configured mounting, eg, hard-wired configuration, where the mapping module is configured to perform various mapping functions. For example, the mapping module may be configured to access at least some of the nucleotide sequence in a read of a plurality of reads obtained from a sequenced genetic sample of a subject, and/or a genetic reference sequence, and/or an index of one or more genetic reference sequences in memory or a cache associated with it, for example, via a memory interface such as a process interconnect, such as Quick Path Interconnect, etc. The mapping module may also be configured to map a read to one or more segments of said one or more genetic reference sequences, for example based on an index. For example, in various specific embodiments, the algorithm and/or mapping module provided herein may be used to construct or otherwise create a hash table that can be used to compare a read, or portion thereof, of a subject's sequenced genetic material with one or more more reference genome segments to obtain mapped reads. In such a case, alignment can be performed upon completion of the mapping.

[00189] For example, once it has been determined where all the possible primer matches to the reference genome are, it is necessary to determine which of all those possible locations where a given read could possibly match is actually the correct position to which it is aligned. Thus, after mapping, there may be many positions where one or more reads appear to match the reference genome. Therefore, there may be multiple primers that appear to indicate exactly the same thing, for example they may match exactly the same position on the reference given the position of the primer in the read. Therefore, for any given read, the true alignment must be determined. This determination can be made in several different ways.

[00190] In one case, all reads can be evaluated to determine their correct alignment to a reference genome based on the positions indicated by each seed from the read that returned position information during a mapping process, such as a hashed search. However, in various cases, it is possible to perform a seed chain filter function on one or more seeds before performing the alignment. For example, in certain cases, primers associated with a given read that appear to map to the same general location in the reference genome may be aggregated into a single strand that references the same general region. All seeds associated with a single read can be grouped into one or more seed chains, so that each seed is included in only one chain. It is these chains that then lead to the alignment of the read with each specified position in the reference genome.

[00191] In particular, in various cases, all seeds that have the same supporting data indicating that they belong to the same general locations in the reference can be collected together to form one or more chains. Therefore, primers that cluster together or at least give the impression that they will end up next to each other in the reference genome, for example within a certain band, will be grouped into a chain of primers, and those outside that band will be converted into another a chain of seeds. Once these different seeds are aggregated into one or more different seed strands, it can be determined which strand actually represents the correct strand to be aligned. This can be done, at least in part, by using a filtering algorithm, which is a heuristic algorithm designed to eliminate weak seeds that have a high probability of not being true.

[00192] The result of performing one of these mapping, filtering and/or editing functions is a list of reads, which for each read contains a list of all possible locations at which the read could match the reference genome. Therefore, the mapping function can be performed to quickly determine where reads from an image file, a BCL file, and/or a FASTQ file obtained from a sequencer map to a reference genome, for example, where different reads map in the complete genome. However, if there is an error in any of the reads or genetic variation, there may not be an exact match to the reference and/or there may be multiple locations that one or more reads appear to match. Therefore, it is necessary to determine where different reads are actually aligned relative to the genome as a whole.

[00193] Accordingly, after mapping and/or filtering and/or editing, location positions have been determined for a large number of reads, with multiple location positions defined for some of the individual reads, and it is now necessary to determine which of all possible locations are actually true or the most likely locations to which different reads are aligned. Such alignment can be performed using one or more algorithms, such as a dynamic programming algorithm, which maps mapped reads to a reference genome and performs an alignment function on it. As an example, the alignment function compares one or more, for example, all reads to a reference, say, by overlaying one on top of another in a graphical mode, for example, in a table such as a virtual array or matrix, where the sequence of one of the reference genomes or mapped reads is placed on one dimension or axis, such as the horizontal axis, and the other is placed on the opposite dimensions or axis, such as the vertical axis. An imaginary scoring wavefront is then passed over the array to determine the alignment of the reads relative to a reference genome, for example by calculating alignment scores for each cell in the matrix.

[00194] The evaluation wavefront represents one or more, for example, all, cells of the matrix or portions of those cells that can be evaluated independently of each other and/or simultaneously in accordance with the rules of dynamic programming applicable in the alignment algorithm, for example Smith-Waterman or Needleman-Wunsch or related algorithms. Alignment scores can be compared sequentially or in other orders, for example by calculating all scores in the top row from left to right, then all scores in the next row from left to right, etc. In this approach, a diagonally unfolding diagonal wavefront represents an optimal sequence of batches of estimates computed simultaneously or in parallel in a series of wavefront stages.

[00195] For example, in one embodiment, a reference genome window containing the segment to which the read has been mapped may be placed on the horizontal axis, and the read may be placed on the vertical axis. In a similar way, an array or matrix is formed, for example, a virtual matrix, as a result of which the nucleotide at each position in the read can be compared with the nucleotide at each position in the reference window. As the wave front passes through the array, all potential ways to align the read with the reference window are considered, including whether any changes need to be made to one sequence to make the read match the reference sequence, for example, by replacing one or more nucleotides of the read with other nucleotides , or the insertion of one or more nucleotides into one sequence, or the deletion of one or more nucleotides from one sequence.

[00196] An alignment estimate representing the extent of changes that would be required to achieve accurate alignment is generated, which estimate and/or other associated data may be stored in these array cells. Each array cell corresponds to the probability that a nucleotide at its position on the read axis is aligned with a nucleotide at its position on the reference axis, and the score generated for each cell represents the partial alignment ending with the cell's positions in the read and reference window. The highest score generated in any cell represents the best overall alignment of the read relative to the reference window. In various cases, the alignment may be global, where the entire read must be aligned to some part of the reference window, for example using the Needleman-Wunsch algorithm or a similar algorithm; or in other cases the alignment may be local, where only part of the read may be aligned to part of the reference window, for example using the Smith-Waterman algorithm or similar algorithm.

[00197] Accordingly, in various cases, the alignment function can be performed, for example, on data obtained from the mapping module. Thus, in various cases, the alignment function may form a module, such as an alignment module, which may form part of a system (e.g., a pipeline) that is used, for example, in addition to the mapping module, in the process of determining the actual complete genomic sequence of an individual or part thereof . For example, the result returned after executing a mapping function, such as from a mapping module, such as a list of candidate locations where one or more or all reads map to one or more positions in one or more reference genomes, can be used by the alignment functions to determine the actual sequence alignment of the subject's sequenced DNA.

[00198] Such an alignment function can always be useful because, as described above, often, for a number of different reasons, sequenced reads do not always match exactly with the reference genome. For example, one or more reads may contain an SNP (single nucleotide polymorphism), for example, the replacement of one nucleotide by another at one position; may be an "indel", an insertion or deletion of one or more bases in one or more read sequences, and this insertion or deletion is not present in the reference genome; and/or there may be a sequencing error (eg, errors in sample preparation and/or sequencer read and/or sequencer output, etc.) causing one or more of these apparent variations. Accordingly, when a read differs from the reference, for example due to a SNP or an indel, the reason may be that the reference differs from the true sample DNA sequence, or that the read differs from the true sample DNA sequence. The challenge is to figure out how to properly align the reads to the reference genome, given the fact that there will likely be many different differences between the two sequences.

[00199] In various cases, the input to an alignment function, for example from a mapping function such as a prefix/suffix tree, or a Burrows-Wheeler transform, or a hash table and/or a hash function, may be a list of possible locations where one or more reads may be mapped to one or more positions of one or more reference sequences. For example, any given read may match any number of positions in the reference genome, such as 1 location, or 16, or 32, or 64, or 100, or 500, or 1000 or more locations where a given read maps to the genome. However, any individual read was obtained, for example sequenced, from only one specific part of the genome. Therefore, to find where a given read came from, one can perform an alignment function, such as a Smith-Waterman gapped and non-gapped alignment, a Needleman-Wunsch alignment, etc., to determine where in the genome one was actually obtained or more reads, for example, by comparing all possible locations where a match occurs and determining which of all possible options is the most likely location from which the read was sequenced based on the highest location alignment score.

[00200] As stated, an algorithm is typically used to perform such an alignment function. For example, the Smith-Waterman and/or Needleman-Wunsch alignment algorithm can be used to align two or more sequences to each other. In this case, they can be used in such a way that, for any given position where a read is mapped to a reference genome, the probabilities are determined that mapping is actually performed at the position where the read originates. Typically, these algorithms are implemented in software, however, in various cases, such as those presented herein, one or more of these algorithms may be implemented in hardware, as described in more detail below herein.

[00201] In particular, the alignment function is used at least in part to align one or more, e.g., all, reads to a reference genome despite the presence of one or more mismatch positions, e.g., SNPs, insertions, deletions, structural artifacts, etc. etc. to determine where these reads are likely to fit correctly into the genome. For example, one or more reads are compared to a reference genome and the best possible match of the read to the genome is determined, taking into account substitutions and/or indels and/or structural variants. However, to better determine which modified version of a read fits best into the reference genome, the putative changes need to be taken into account, and therefore a scoring function can also be performed.

[00202] For example, it is possible to perform an evaluation function, for example as part of an overall alignment function, in which the alignment module performs its function and introduces one or more changes to a sequence being compared with another sequence, for example, to achieve a better or better match between by them, and for each change made to achieve a better alignment, a number is subtracted from the initial score, for example, either from the ideal score or from the zero initial score, so that when the alignment is performed, the assessment of this alignment is also determined, for example, When matches are found, the score is increased, and each change made is penalized, so that the best possible match for possible alignments can be determined by identifying, from all possible modified reads, the read that has the highest score for its match to the genome. Accordingly, in various cases, the alignment function may be configured to determine the best combination of changes to make to the read(s) to achieve the alignment with the highest score, and then that alignment may be determined to be the correct or most likely alignment.

[00203] Therefore, in view of the above, there are at least two purposes that can be achieved by performing the alignment function. One is a report of the best alignment, including a position in the reference genome and a description of the changes that are necessary for the read to match the reference segment at that position, and the other is an assessment of the quality of the alignment. For example, in various cases, the output from the alignment module may be a Compact Idiosyncratic Gapped Alignment Report, such as a CIGAR string, where the CIGAR string output is a report detailing all the changes that were made to the reads to achieve the most appropriate alignment for them, e.g. , detailed alignment instructions indicating how the sequence of interest actually aligns with the reference. The output of such a CIGAR string can be useful in subsequent processing steps to better determine that, for a given individual's genomic nucleotide sequence, the predicted variations relative to the reference genome are indeed true variations and not simply due to machine, software, or human error.

[00204] As discussed above, in various implementations, alignment is typically performed sequentially, with the algorithm and/or firmware receiving read sequence data (e.g., from a mapping module) belonging to the read, and one or more possible locations where that read can potentially be mapped to one or more reference genomes, and also receives genomic sequence data (e.g., from one or more memories, such as linked DRAMs) relating to one or more positions in one or more reference genomes that can be mapped to read. Specifically, in various embodiments, the mapping module processes reads, for example from a FASTQ file, and maps each of them to one or more positions in the reference genome to which they may be aligned. The aligner then takes these predicted positions and uses them to align the reads to a reference genome, for example by constructing a virtual array against which the reads can be compared to the reference genome.

[00205] In performing this function, the aligner evaluates each mapped position for each individual read and, in particular, evaluates those reads that are mapped to many possible locations in the reference genome, and for each position evaluates the possibility that it is correct. It then compares the best estimates, such as the two best estimates, and makes a decision about where a particular read actually aligns. For example, when comparing the first and second best alignment scores, the aligner checks the difference between the scores, and if the difference between them is large, then the confidence rating that the position with the higher score is correct will be high. However, if the difference between them is small, for example zero, then the confidence level of choosing one of the two positions as the correct position from which the read was obtained is low and additional processing may be useful to clearly determine the true location in the reference genome , from which the read was obtained.

[00206] Therefore, the aligner specifically looks for the largest difference between the first and second best confidence scores to decide that a given read maps to a given location in the reference genome. Ideally, the score of the best possible alignment is significantly higher than the score of the second best alignment for a given sequence. There are many different ways to implement an alignment evaluation method, for example, you can evaluate each cell of an array or a subset of cells, for example, in accordance with the methods described herein. In various cases, the scoring parameters for nucleotide matches, nucleotide mismatches, insertions, and deletions may have any number of different positive, negative, or zero values. In various cases, these evaluation parameters may be modified based on available information. For example, accurate alignments can be achieved by changing scoring parameters, including any or all of nucleotide match scores, nucleotide mismatch scores, gap penalties (insertion and/or deletion), gap opening penalties, and/or gap extension penalties, according to the base quality score associated with the current nucleotide or read position. For example, scoring bonuses and/or penalties may be reduced when the base quality score indicates a high likelihood of sequencing or other errors. The ground quality-sensitive assessment can be implemented, for example, using a fixed or configurable lookup table accessible by the ground quality score, which returns the corresponding assessment parameters.

[00207] When implemented in hardware in an integrated circuit such as an FPGA or ASIC, the evaluation wavefront may be implemented as a linear array of evaluation cells, such as 16 cells, or 32 cells, or 64 cells, or 128 cells, etc. .P. Each of the evaluation cells can be constructed from digital gates in a wiring configuration to compute alignment estimates. Thus, for each wavefront stage, such as each clock cycle or some other fixed or variable time unit, each of the evaluation cells, or subset of cells, calculates the evaluation or evaluations required for a new cell in the virtual alignment matrix. Theoretically, different evaluation cells are considered to be at different positions in the alignment matrix corresponding to the front of the evaluation wave, as noted herein, for example, along a straight line extending from the lower left to the upper right of the matrix. As is known from the field of digital logic device design, the physical evaluation cells and their constituent digital logic do not have to be physically located in this manner on an integrated circuit.

[00208] Accordingly, as the wave front sweeps step by step through the virtual alignment matrix, the imaginary positions of the evaluation cells update each cell accordingly, for example, speculatively "moving" a step to the right or, for example, a step down in the alignment matrix . All evaluation cells undergo the same relative imaginary movement, maintaining the order of the diagonal wave front. Whenever a wave front moves to a new position, for example by a step in the vertical direction down or a step in the horizontal direction to the right in the matrix, the scoring cells arrive at the new imaginary positions and compute alignment scores for the virtual alignment matrix cells they entered . In such an implementation, adjacent scoring cells in a linear array are connected to exchange test (belonging to a read) nucleotides, reference nucleotides, and previously calculated alignment scores. The reference window nucleotides can be sequentially applied to one end of the wave front, for example, the top right score cell in a linear array, and can be sequentially shifted from there down along the wave front so that at any given time a segment of reference nucleotides equal in length to the number of score cells is was present in these cells, one successive nucleotide in each successive cell.

[00209] For example, whenever the wave front moves one step in the horizontal direction, the next reference nucleotide is fed into the upper right cell, and other reference nucleotides are moved down the left along the wave front. This shift of reference nucleotides may be a real reflection of the imaginary movement of the wavefront of the score cells to the right in the alignment matrix. Thus, the nucleotides of the read can be sequentially fed to the opposite end of the wave front, for example, into the lower left evaluation cell in a linear array, and can be sequentially shifted from there upward along the wave front so that at any given time a segment of the nucleotides under study equal in length to the number of cells estimated to be present in these cells, one successive nucleotide in each successive cell. Likewise, whenever the wave front moves one step in the vertical direction, the next nucleotide of interest is fed into the lower left cell, and other nucleotides of interest are moved up to the right along the wave front. This shift of the test nucleotides may be a real reflection of the imaginary downward movement of the wavefront of the score cells in the alignment matrix. Accordingly, by issuing a command to shift reference nucleotides, one can move the wave front one step in the horizontal direction, and by issuing a command to shift the test nucleotides, one can move the wave front one step in the vertical direction. Thus, to obtain an overall diagonal movement of the wavefront, for example to follow a typical alignment of the test and reference sequences without insertions or deletions, the wavefront can be commanded alternately to move the wavefront one step in the vertical and horizontal directions.

[00210] Accordingly, adjacent scoring cells in a linear array can be connected to exchange previously computed alignment scores. In various alignment estimation algorithms, such as the Smith-Waterman or Needleman-Wunsch algorithm, etc., the scores in each cell of the virtual alignment matrix can be calculated using previously calculated scores in other matrix cells, for example, in three cells located directly to the left of the current cell, above the current cell, and up to the left diagonally from the current cell. When a score cell computes new scores for another matrix position in which it is included, it must retrieve such previously computed scores corresponding to such other matrix positions. These previously computed scores may be obtained from a store of previously computed scores within the same cell and/or from a store of previously computed scores in one or two adjacent score cells in the linear array. The point is that the three scoring-contributing positions in the virtual alignment matrix (directly left, top, and top left diagonally) could have been scored either by the current score cell or by one of its neighboring score cells in the linear array.

[00211] For example, a cell immediately to the left in the matrix could be scored by the current evaluation cell if the most recent wavefront step was in the horizontal direction (to the right), or could be scored by an adjacent cell to the bottom left in a linear matrix if the most recent wavefront step was in the vertical (down) direction. Similarly, a cell immediately above in the matrix could be scored by the current evaluation cell if the most recent wavefront step was in the vertical (down) direction, or could be scored by the adjacent cell on the top right in a linear matrix if the most recent the wavefront step was in the horizontal direction (to the right). Specifically, the top-left diagonal cell in the matrix could be scored as the current scoring cell if the two most recent wavefront steps were in different directions, for example, down then right, or right then down , or could be scored by a top-right neighbor cell in a line array if the two most recent wavefront steps were both in the horizontal direction (to the right), or could be scored by a bottom-left neighbor cell in a line array if the two most recent wavefront steps were both in the vertical direction (down).

[00212] Accordingly, given information about the directions of the last one or two steps of the wave front, an estimation cell can select the appropriate previously calculated estimates by accessing them within itself and/or in neighboring evaluation cells using the connection between adjacent estimates. As an option, invalid zero or minimum value estimates can be hardwired at the outer evaluation inputs of the evaluation cells at the two ends of the wave front so that they do not affect new evaluation calculations in those outer cells. Thanks to the wavefront implemented in this way in a linear array of evaluation cells with such a connection for shifting the reference and test nucleotides across the array in opposite directions, in order to speculatively move the wavefront step by step in the vertical and horizontal direction, for example, diagonally, and a connection for accessing the estimates, previously calculated by neighboring cells, in order to calculate alignment estimates at new positions of the cells of the virtual matrix in which the wave front enters, one can accordingly estimate a strip of cells in the virtual matrix as wide as the wave front, for example, by issuing commands to move the wave front sequentially step by step, so that it rolled across the matrix.

[00213] Therefore, to align the new read and the reference window, the wave front can start from inside the evaluation matrix, or, advantageously, can gradually enter the evaluation matrix from the outside, starting, for example, from the left, or from the top, or diagonally from the left and from the top left corner of the matrix. For example, the wave front may begin with its top left evaluation cell located immediately to the left of the top left cell of the virtual matrix, and then the wave front may roll to the right of the matrix through a series of horizontal steps, evaluating a horizontal strip of cells in the top left region of the matrix. When the wavefront reaches the predicted alignment relationship between the reference and test sequences, or when a match is detected based on increasing alignment scores, the wavefront can begin to sweep diagonally down to the right by alternating vertical and horizontal steps, evaluating a diagonal stripe of cells in the middle. When the lower left wavefront evaluation cell reaches the bottom of the alignment matrix, the wavefront may begin to roll to the right again through successive horizontal steps, evaluating a horizontal strip of cells in the lower right region of the matrix, until some or all of the wavefront cells exceed the boundaries alignment matrices.

[00214] One or more such alignment procedures may be performed using any suitable alignment algorithm, such as the Needleman-Wunsch alignment algorithm and/or the Smith-Waterman alignment algorithm, which may be modified to conform to the functionality described herein document. In general, both of these and similar algorithms essentially work, in some cases, in a similar way. For example, as stated above, these alignment algorithms typically construct a virtual array in a similar manner such that in various cases, a horizontal top border may be configured to represent a genomic reference sequence that may be laid out across the entire top row of the array according to its base pair compositions. Likewise, the vertical border may be designed to represent sequenced and mapped sequences of interest that have been arranged in order downward along the first column such that their nucleotide sequence order is substantially the same as the nucleotide sequence of the reference to which they are mapped. Then the intermediate cells can be filled with estimates of the probability that the corresponding base of the sequence under study at a given position is located at that location relative to the reference. When performing this function, the capture strip can move diagonally across the matrix, filling in the scores within the intermediate cells, and starting from a specified position, the probability for each base of the sequence being examined can be determined.

[00215] With respect to the Needleman-Wunsch alignment function, which creates optimal global (or semi-global) alignments aligning all read sequences onto some segment of the reference genome, control of the direction of the wave front can be configured so that it generally rolls away from the the top edge of the alignment matrix, to the very bottom edge. Upon completion of the wave front rolling, the maximum score is selected at the bottom edge of the alignment matrix (corresponding to the end of the read), and the alignment is tracked back to the cell at the top edge of the matrix (corresponding to the start of the read). In various cases described herein, reads can be of any length, can be of any size, and extensive parameters are not required to describe how the alignment is performed, for example, in various cases, the length of a read can be the same as a chromosome. However, in this case, memory size and chromosome length may be a limiting factor.

[00216] Similar to the Smith-Waterman algorithm, which generates optimal local alignments by aligning all or part of a read sequence to some segment of a reference genome, this algorithm can be configured to search for the best possible estimate based on a full or partial read alignment. Therefore, in various cases, the band estimated by the wavefront may not reach the top and/or bottom edges of the alignment matrix, for example, if a very long read has only seeds when mapping its middle to the reference genome, but usually the wavefront can still estimate from the top to bottom of the matrix. Local alignment is usually achieved by two adjustments. First, the alignment scores are prohibited from falling below zero (or some other lower threshold), and if the cell's calculated score would otherwise be negative, it is replaced with a zero score representing the start of a new alignment. Secondly, the maximum alignment score obtained in any cell of the matrix, not necessarily along the bottom edge, is used as the final alignment point. The alignment is tracked backwards from this maximum score up and to the left throughout the matrix until the zero score is used as the starting position of the local alignment, even if it is not in the top row of the matrix.

[00217] In view of the above, there are several possible paths through the virtual array. In various implementations, the wave front begins at the top left corner of the virtual array and moves down toward the maximum score identifiers. For example, the results of all possible alignments can be collected, processed, correlated, and scored to determine a maximum score. When the end of the boundary or the end of the array is reached and/or the computation that led to the highest score for all processed cells is determined (eg, the overall highest score is identified), you can trace back to find the path that led to that highest score being reached. For example, a path that leads to a predicted maximum score can be found, and an audit can then be performed to determine how that maximum score was arrived at, for example by moving backwards, following the alignment arrows with the best score indicating the path backwards. , which led to the achievement of the identified maximum score, for example, calculated using the wave front score cells.

[00218] This backward reconstruction or backtracking involves starting a movement at a certain maximum score and working back through previous cells, tracing a path through the cells having the scores that led to the maximum score, from the very top of the table and back to the starting boundary , such as the start of the array or zero evaluation in the case of local alignment. During backtracking, after reaching a particular cell in the alignment matrix, the next backtracking step is to the adjacent cell immediately to the left, or above, or diagonally up to the left, which contributed to the best estimate selected to construct the estimate in the current cell. In this way, the evolution of the maximum score can be tracked, thereby revealing how the maximum score was achieved. The backtracking may end at a corner, or at an edge, or at a boundary, or may end at a zero score, such as the top left corner of an array. Accordingly, it is this backtracing that determines the correct alignment and thereby produces a CIGAR output string that shows how the genomic sequence, or part thereof, from a sample taken from an individual matches or otherwise aligns with the genomic sequence of the reference DNA.

[00219] Once it has been determined where each read maps to, and it has also been determined where each read is aligned, for example, each corresponding read is given a position and a quality score reflecting the likelihood that that position is the correct alignment, so that the nucleotide sequence for Since the subject's DNA is known, the order of the various reads and/or the genomic sequence of the subject's nucleic acids can be verified, for example, by determining the identity of each nucleic acid in its correct order in the genomic sequence of the sample. Therefore, in accordance with some aspects, the present invention relates to a backtracking function, such as one that is part of an alignment module that performs both an alignment function and a backtracking function, such as a module that may be part of a pipeline of modules, e.g., a pipeline that is designed to for receiving raw sequence read data, for example in the form of an individual's genomic sample, and mapping and/or aligning this data, which can then be stored.

[00220] To facilitate the backtracking operation, it is useful to store a score vector for each scored cell in an alignment matrix encoding the score selection decision. In the case of classical interpretations of the Smith-Waterman and/or Needleman-Wunsch estimator with linear gap penalties, the estimator vector can encode four possibilities, which can optionally be stored as a 2-bit integer from 0 to 3, for example: 0 = new alignment (zero rating selected); 1 = vertical alignment (score selected from cell above, modified due to gap penalty); 2 = horizontal alignment (score selected from cell on left, modified due to gap penalty); 3 = diagonal alignment (score selected from cell above and left, modified by nucleotide match or mismatch score). Optionally, one can also store the calculated scores for each matrix cell evaluated (in addition to the maximum alignment score achieved, which is usually stored), but this is not necessary for backtracking, and this information can take up huge amounts of memory. The backtracking execution then reduces to the following evaluation vectors; when backtracking has reached a given cell in the matrix, the next backtracking step is determined by the stored score vector for that cell, for example: 0 = end backtracking; 1 = backward tracking up; 2 = back tracking left; 3 = reverse tracking up left.

[00221] Such scoring vectors may be stored in a two-dimensional table arranged according to the dimensions of the alignment matrix, which may be populated only with entries corresponding to the cells being scored by the wavefront. In an alternative implementation, to save memory, make it easier to record score vectors as they are generated, and more easily accommodate alignment matrices of different sizes, the score vectors can be stored in a table, the size of each row of which is determined by the stored score vectors from one wavefront of the score cells. for example, 128 bits to store 64 2-bit wavefront estimate vectors consisting of 64 cells, and the number of rows is equal to the maximum number of wavefront steps in the alignment operation. In addition, for this option, it is possible to record the directions of the various wavefront steps, for example, by storing an additional, for example, 129th, bit in each row of the table, encoding, for example, with 0 the vertical wavefront step preceding a given wavefront position, and using 1, the horizontal step of the wave front preceding a given position of the wave front. This extra bit can be used during backtracking to control which positions of the virtual score matrix the score vectors in each row of the table correspond to, so that the correct score vector can be retrieved after each subsequent step. If the backtrack step is vertical or horizontal, the next estimate vector should be retrieved from the previous row of the table, but if the backtrack step is diagonal, the next estimate vector should be retrieved two rows above, since the wave front must take two steps to move from the estimate of any one cell to the score of the cell below it to the right.

[00222] In the case of affine gap estimation, the estimation vector information can be expanded, for example, to 4 bits per cell being estimated. In addition to, for example, a 2-bit evaluation selection direction indicator, two 1-bit flags, a vertical extension flag and a horizontal extension flag, may be added. According to the methods of affine gap estimation extensions to Smith-Waterman, Needleman-Wunsch, or similar algorithms, for each cell, in addition to the primary alignment score representing the best-scoring alignments ending at that cell, a "vertical score" should be generated corresponding a maximum alignment score achieved by that cell using the final vertical step, and a “horizontal score” corresponding to the maximum alignment score achieved by that cell using the final horizontal step; and when calculating any of the three estimates, the vertical step per cell can be calculated using either the primary estimate from the cell above minus the gap opening penalty, or the vertical estimate from the cell above minus the gap extension penalty, whichever is greater; and the horizontal step per cell can be calculated using either the primary estimate from the cell on the left minus the gap opening penalty, or the horizontal estimate from the cell on the left minus the gap extension penalty, whichever is greater. In the case of choosing a vertical estimate minus the penalty for gap extension, the vertical extension flag must be set in the evaluation vector, for example, “1”, and otherwise it must be cleared, for example, “0”.

[00223] In the case of selecting a horizontal score minus gap extension penalty, the horizontal extension flag should be set in the score vector, for example, "1", and otherwise it should be cleared, for example, "0". During backtracking in the case of affine gap evaluation, whenever backtracking takes a vertical step up from a given cell and the vertical extension flag in that cell's evaluation vector is set, the next backtracking step must also be vertical, regardless of the evaluation vector for the cell above. Likewise, whenever backtracking takes a horizontal step to the left from a given cell and the horizontal extension flag in that cell's evaluation vector is set, the next backtracking step must also be horizontal, regardless of the evaluation vector for the cell to the left. Accordingly, such a table of estimation vectors, having, for example, 129 bits per row for 64 cells when using linear gap estimation, or 257 bits per row for 64 cells when using affine gap estimation, and a certain number, NR, of rows, is sufficient to support the reverse tracking after completion of the alignment evaluation, where the evaluation wave front completes NR steps or less.

[00224] For example, when aligning reads of 300 nucleotides, the number of wavefront steps may always be less than 1024, so the table may occupy 257×1024 bits, or approximately 32 kilobytes, which in many cases may be reasonable local memory within the integrated circuit. But if very long reads need to be aligned, for example 100,000 nucleotides, the memory requirements for the score vectors can be quite large, for example 8 megabytes, which may be too expensive to include as local memory inside the integrated circuit. To support such support, the evaluation vector information can be written to a large capacity off-chip memory such as DRAM, but then the bandwidth requirements, such as 257 bits per clock cycle for each equalizer, may become prohibitively expensive, which may result in congestion and abrupt reduce the performance of the equalizer. Accordingly, it would be desirable to have a method for allocating score vectors prior to completion of the alignment to keep their memory requirements limited, for example, to perform incremental backtraces generating incremental partial CIGAR strings, for example, from the initial portions of the history of the alignment score vectors, such that the initial parts of the evaluation vectors could be discarded. The problem is that backtracking is supposed to start from the final alignment point, the cell with the maximum score, which is unknown until the alignment evaluation is complete, so any backtracking that starts before alignment is complete may start from the wrong cell , not along the possible final optimal alignment path.

[00225] Therefore, a method is provided for performing incremental backtracking based on partial alignment information, for example, containing partial information about the evaluation vectors for the alignment matrix cells currently evaluated. Based on the boundary of the alignment performed so far, for example, a specific position of the wave front, backtracking begins from all cell positions on the boundary. Such backtracking from all boundary cells may be performed sequentially or, advantageously, especially in the case of a hardware implementation, all backtracking may be performed together. It is not necessary to extract symbolic alignment records, such as CIGAR strings, from these multiple backtraces; one only needs to determine which alignment matrix positions they pass through during backtracking. When implementing simultaneous backtracking from a boundary, it is possible to use initialized, eg set to "1", 1-bit registers in a number corresponding to the number of alignment cells that indicate whether any of the backtracking passes through the corresponding position. For each simultaneous backtracking step, the score vectors corresponding to all the current "1s" in those registers can be examined, for example from one row of the score vectors table, to determine the next backtracking step corresponding to each "1" in the registers leading to the next position for each "1" in the registers, for the next step of simultaneous backtracking.

[00226] It is important to note that it is very likely that multiple "1's" in the registers will merge into common positions corresponding to multiple simultaneous backtraces merging into common backtracking paths. Once two or more simultaneous tracks are merged, they will remain merged forever since they will use the score vector information from the same cell from that point on. It has been observed, empirically and theoretically, that there is a high probability that all simultaneous tracking will merge into a singular backtracking route in a relatively small number of backtracking steps, the number of which, for example, can be a multiple of the number of estimation cells in the wave front by a small factor, e.g. , 8-fold. For example, in the case of a 64-cell wavefront, there is a high probability that all backtracks from a given wavefront boundary will merge into a single backtrack route within 512 backtrack steps. Or it is also possible, and not uncommon, that all backtracks will complete within this number, for example 512 backtrack steps.

[00227] Accordingly, multiple simultaneous backtraces can be performed from an estimation boundary, such as an estimated wavefront position, far enough away that they all either complete or merge into a single backtracking path, for example, in 512 steps or less. If they all merge into a singular backtracking path, then starting from the location in the matrix where they merged, or any distance further along the singular backtracking path, incremental backtracking is possible based on partial alignment information. Backtracking is then started from the merge point or any distance further back using conventional singular backtracking techniques, including writing an appropriate symbolic notation, such as a partial CIGAR string. This incremental backtracking and, for example, the partial CIGAR row, must be part of any eventual final backtracking, and, for example, the full CIGAR row, that will result upon completion of the alignment, unless such final backtracking ends before reaching the estimate boundary where the simultaneous backtracks, since if it reaches the evaluation boundary, it must follow one of the simultaneous tracking routes and merge with the singular backtracking route now allocated incrementally.

[00228] Consequently, all score vectors for matrix regions corresponding to incrementally allocated backtracking, for example, in all table rows for wavefront positions preceding the start of the allocated singular backtracking, can be safely discarded. When executing a trailing backtrack from a cell with a maximum score, if it ends before reaching the score boundary (or if it ends before reaching the start of the allocated singular backtrack), the incremental symbolic record, such as a partial CIGAR row, can be discarded. If the trailing backtrack continues before the start of the dedicated singular backtrack, then its alignment symbolic record may be incremented by an incremental symbolic alignment record, such as a partial CIGAR string. Additionally, for very long alignments, the process of performing simultaneous backtracking from an estimate boundary, such as the estimated wavefront position, until all backtracks are completed or merged, followed by a singular backtracking highlighting the symbolic alignment record, can be repeated many times. from various successive assessment boundaries. An incremental symbolic alignment record, such as a partial CIGAR string, from each successive incremental backtracking may then be incremented by the accumulated previous symbolic alignment entries, unless a new simultaneous backtracking or singular backtracking completes first, in which case the accumulated previous symbolic alignment entries can be discarded. Likewise, a possible terminating backtrack builds its symbolic record onto the most recently accumulated symbolic alignment records to complete a backtrack description, such as a CIGAR string.

[00229] Accordingly, with such a method, the memory for storing the evaluation vectors may remain limited under the assumption that simultaneous backtraces are always merged in a limited number of steps, for example 512 steps. In rare cases where simultaneous backtraces fail to merge or complete within a limited number of steps, various exceptional actions may be taken, including aborting the current alignment due to failure or retrying it with a higher limit or no limit, perhaps in a different or traditional way , such as storing all score vectors for full alignment, for example, in external DRAM. Alternatively, it may be advisable to terminate such an alignment due to failure, as this is extremely rare, and even more rarely, such a failed alignment will have a better-scoring alignment included in the alignment report.

[00230] Optionally, the score vector store can be divided, physically or logically, into a number of different blocks, for example, 512 rows each, and the ending row in each block can be used as a score boundary to begin simultaneous backtracking. Optionally, simultaneous backtracking may be required to complete or merge within a single block, for example in 512 steps. Optionally, if simultaneous backtracks are merged in fewer steps, the joint backtrack can still be continued throughout the block before proceeding to extract the singular backtrack in the previous block. Accordingly, after the evaluation vectors are completely written to block N and writing to block N+1 begins, simultaneous backtracking can be started in block N, followed by singular backtracking and symbolic alignment record extraction in block N-1. If the speed of simultaneous backtracking, singular backtracking, and alignment evaluation are similar or identical, and these operations can be performed simultaneously, for example in parallel hardware in an integrated circuit, then singular backtracking in block N-1 can occur simultaneously with the block evaluation vectors populating N+2, and when it is time to fill block N+3, block N-1 can be freed and reused.

[00231] Therefore, with such an implementation, a minimum of 4 blocks of evaluation vectors can be used, replacing each other in a cycle. Therefore, the total storage of evaluation vectors for the equalizer module could be, for example, 4 blocks of 257x512 bits each, or approximately 64 kilobytes. Alternatively, if the current maximum alignment estimate corresponds to an earlier block than the current wavefront position, that block and the previous block may be held rather than reused so that the final backtracking can be started from that position if it remains the position with maximum rating; holding an extra 2 blocks this way will give you at least 6 blocks, for example.

[00232] In another embodiment, additional blocks, e.g., 1 or 2 additional blocks, e.g., a total of 8 blocks, can be used to support overlapping alignments, where the evaluation wavefront gradually moves from one alignment matrix to the next, as described above, e.g. approximately 128 kilobytes. Accordingly, by cycling such a limited number of blocks, such as 4 blocks or 8 blocks, it is possible to align and backtrack long reads, such as 100,000 nucleotides, or an entire chromosome, without using external memory for the score vectors. It should be understood that in relation to the above, although in some cases the mapping function may have been referred to in the descriptions as, for example, a mapper, and/or in some cases the alignment function may have been referred to as, for example, an aligner, these different functions may be performed simultaneously by the same architecture , which is commonly referred to in the art as a leveler. Accordingly, in various cases, the mapping function and the alignment function as described herein may be performed by a common architecture, which can be understood as an aligner, especially in cases where in order to perform the alignment function, the mapping function must first be performed.

[00233] In various cases, the devices, systems, and methods for using them of the present invention may be configured to perform one or more of gapped and/or ungaped alignments of a full read, which can then be evaluated to determine the appropriate alignment for the reads in the set data. For example, in various cases, a gapless alignment procedure may be performed on the data to be processed, and the gapless alignment procedure may be followed by one or more gap alignments and/or a selective Smith-Waterman alignment procedure. For example, at the first stage an alignment chain without gaps can be formed. As described herein, such gapless alignment functions can be performed quickly without the need to consider gaps, where the first step of performing gapless alignment can be followed by performing gap alignment.

[00234] For example, an alignment function can be performed to determine how any given nucleotide sequence, such as a read, aligns to a reference sequence, without the need to insert gaps in one or more reads and/or reference. An important part of performing such an alignment function is to determine where and what mismatches exist in the sequence of interest compared to the reference genome sequence. However, due to the enormous homology in the human genome, any given nucleotide sequence, theoretically, tends to match to a significant degree with a representative reference sequence. The mismatches that exist are likely to be due to a single nucleotide polymorphism, which is relatively easy to detect, or they will be caused by an insertion or deletion in the sequence of interest, which is much more difficult to detect.

[00235] Therefore, when performing an alignment function, in most cases the sequence of interest will tend to match the reference sequence, and if there is a mismatch due to an SNP, this will be easily determined. Therefore, performing such analysis does not require a relatively large amount of computing power. However, if there are insertions or deletions in the test sequence compared to the reference sequence, difficulties arise, since such insertions and deletions lead to gaps in the alignment. Such gaps require a more extensive and complex processing platform to correctly determine the alignment. However, since the percentage of indels will be small, a relatively small percentage of gap alignment protocols will need to be performed compared to the millions of gapless alignments performed. Therefore, only a small percentage of all non-gapped alignment functions will result in the need for further processing due to the presence of an indel in the sequence and therefore require a gap-aligned alignment.

[00236] When an indel is specified in a gapless alignment procedure, only those sequences are passed to an alignment engine for further processing, for example, a processing engine configured to perform an advanced alignment function such as a Smith-Waterman alignment (SWA). Therefore, since alignment must be performed either without gaps or with gaps, the devices and systems described herein are a much more efficient use of resources. More specifically, in certain embodiments, alignments can be performed on a given set of sequences either without gaps or with gaps, for example, one after the other, then the results for each sequence are compared and the best result is selected. Such a configuration may be implemented, for example, when increased accuracy is desired and an increased amount of time and resources are available to perform the required processing.

[00237] In particular, in various cases, the first stage of alignment can be performed without involving the Smith-Waterman function, which requires intensive processing. Therefore, multiple gapless alignments can be performed with fewer resource requirements and less time overhead, and since fewer resources are required, less on-chip space must be allocated for such processing. In this way, more processing can be done using fewer processing elements requiring less time, so more alignments can be performed and higher accuracy can be achieved. More specifically, performing Smith-Waterman alignments requires fewer chip resources using less chip area because the processing elements required to perform gapless alignments do not require as much chip area as to perform gapped alignments. As chip resource requirements are reduced, more processing can be performed in a shorter period of time, and with more processing that can be performed, higher accuracy can be achieved.

[00238] Accordingly, in such cases, a gapless alignment protocol can be used, for example, which must be executed by suitably configured gapless alignment resources. For example, as described herein, in various embodiments an alignment processing engine is provided, for example where the processing engine is configured to receive digital signals, for example, representing one or more reads of genomic data, such as digital data expressing one or more nucleotide sequences , from an electronic data source, and mapping and/or aligning that data to a reference sequence, e.g. by first performing a gapless alignment function on the data, where the gapless alignment function may be followed by a gap alignment function if necessary, e.g. Smith-Waterman alignment protocol.

[00239] Therefore, in various cases, the non-gapped alignment function is performed on a solid portion of the read, for example, using a non-gapped aligner, and if the non-gapped alignment runs from the beginning to the end, for example, the read is complete, the gap alignment is not performed. However, if the results of an alignment without gaps indicate the presence of an indel, such as the read being truncated or otherwise incomplete, an alignment with gaps can be performed. Thus, the results of a gapless alignment can be used to determine whether a gap alignment is necessary, for example when a gapless alignment has reached the gap region but has not extended to the full length of the read, for example when a read may have been trimmed, such as lightly trimmed, and where it is cut, alignment can then be done with the gaps.

[00240] Therefore, in various implementations based on completeness and alignment assessments, a gapped alignment is only used if a non-gapped alignment ends up being trimmed, eg, does not go from start to finish. More specifically, in various embodiments, the best estimate of the alignment without and/or with gaps that can be determined can be evaluated and used as a cutoff line for deciding whether the estimate is good enough to justify further analysis, for example by performing alignment with gaps. Thus, alignment completeness and its score can be used such that a high score indicates that the alignment is complete and therefore has no gaps, and a low score indicates that the alignment is not complete and an alignment with gaps needs to be performed. Therefore, when a high score is reached, no gap alignment is performed, and gap alignment is performed only when the score is low enough. Of course, in various cases a brute-force alignment approach can be applied, for example when a number of gapped and/or non-gapped aligners are deployed in a chip architecture to allow more alignments to be performed and thus more data can be examined.

[00241] More specifically, in various embodiments, each mapping and/or alignment engine may include one or more, such as two, Smith-Waterman aligner modules. In certain cases, these modules may be configured to support global (end-to-end) alignment without gaps and/or local (truncated) alignment with gaps, perform affine scoring of gaps, and may be configured to generate score bonuses for not clipping on each end. It is also possible to support a quality-sensitive basis for assessing agreement and disagreement. When two equalizer modules are included, for example as part of an integrated circuit, for example, each Smith-Waterman equalizer can be constructed as an anti-diagonal wavefront of evaluation cells, with this wavefront "moving" across the virtual equalization rectangle, evaluating the cells through which it rolls.

[00242] However, in the case of long reads, the Smith-Waterman wavefront may also be configured to support automatic control of the direction of movement of the wavefront to track better alignment through accumulated indels to ensure that the alignment wavefront and the cells being evaluated do not fall outside the evaluation band . Logic engines can be configured to check current estimates of the wave front, find the maximum, mark subsets of cells at a threshold distance below the maximum, and target the midpoint between two extreme flags in the background. In such a case, the automatic control of the direction of movement may be configured to travel diagonally when the target is in the center of the wave front, but may be configured to travel straight horizontally or vertically when necessary to return the target to the center if it moves, for example due to the presence of indels.

[00243] The output from the alignment module is a SAM (text) or BAM file (e.g., a binary version of a SAM file) along with a Mapping Quality Assessment (MAPA), which reflects the confidence that the predicted and aligned location of the read relative to the reference is actually the same one. where did the read come from? Accordingly, once it is determined where each read is mapped, and it is also determined where each read is aligned, for example, each corresponding read is given a position and a quality score reflecting the likelihood that that position is a correct alignment, so that the nucleotide sequence for the DNA When a subject's DNA is known, and how the subject's DNA differs from a reference (eg, a CIGAR string is defined), the various reads representing the subject's genomic nucleic acid sequence can be sorted by chromosomal location so that the precise chromosomal location of the read can be determined. Therefore, in accordance with some aspects, the present invention relates to a sorting function, for example, that can be performed by a sorting module, which can be part of a pipeline of modules, for example, a pipeline designed to receive raw sequence read data, for example, in the form of a genomic sample of an individual, and mapping and/or alignment of this data, which can then be stored.

[00244] More specifically, once reads have been assigned positions, for example, relative to a reference genome, which may include determining which chromosome the read belongs to and/or its offset from the start of that chromosome, the reads can be sorted by position. Sorting can be useful, for example, in subsequent analyzes, since with its help all reads that overlap a given position in the genome can be formed into a cluster so that they are located next to each other, for example, after processing by the sorting module, as a result of which you can easily determine whether the majority of reads agree with the reference value or not. Thus, if the majority of reads do not agree with the reference value, the variant definition can flag. Therefore, sorting may involve one or more sortings of reads that are aligned to the same position, e.g., the same chromosome position, to create a cluster so that all reads that cover the same location are physically grouped together; and may also include analysis of reads in the assemblage to determine where the reads may indicate an actual variant in the genome compared to a reference genome, which variant can be distinguished, for example, by assemblage reconciliation, from an error such as a machine read error or an error in sequencing methods that may occur in a small portion of the reads.

[00245] Once the data is received, there are one or more other modules that can be used to clean up the data. For example, one module that may be included in, for example, a sequence analysis pipeline, such as for determining the genomic sequence of an individual, may be a local realignment module. For example, it is often difficult to identify insertions and deletions that occur at the end of a read. The reason is that the Smith-Waterman algorithm or similar alignment process lacks context about the indel to allow evaluation to detect its presence. Therefore, the actual indel may be reported as one or more SNPs. In such a case, the accuracy of the predicted location for any given read can be improved by performing local realignment on the mapped and/or aligned and/or sorted read data.

[00246] In such cases, pipelines can be used to assist in figuring out the true alignment, for example, when the position in question is at the end of any given read, that same position is likely to be in the middle of some other read in a given cluster. Accordingly, when performing local realignment, various reads in a cluster can be analyzed to determine whether some reads in the cluster indicate the presence of an insertion or deletion at a given position where another read does not contain an indel or rather has a substitution at that position, then one can insert an indel, for example, into a reference where it is not present, and one can re-align reads in a local cluster that overlaps that region to see if this achieves a better aggregate score than if it were not there insertions and/or deletions. If there is an improvement, the entire set of reads in the cluster can be revised, and if the score of the entire set has improved, then it becomes clear that there was indeed an indel at that position. In a similar way, one can compensate for the lack of sufficient context to more accurately align the read at the end of the chromosome for any individual read. Therefore, when performing a local realignment, one or more clusters that may contain one or more indels are examined and it is determined whether the overall alignment score can be improved by adding an indel at any given position.

[00247] Another module that may be included, for example, in a sequence analysis pipeline, such as for determining the genomic sequence of an individual, may be a duplicate tagging module. For example, the duplicate flagging function may be configured to compensate for chemistry errors that may occur during the sequencing phase. For example, as described above, during some sequencing procedures, nucleic acid sequences are attached to beads and built on that basis using labeled nucleotide bases. Ideally, you will get one read per bead. However, sometimes multiple reads are attached to a single bead, resulting in an excessive number of copies of the attached reads. This phenomenon is known as read duplication.

[00248] After performing the alignment and obtaining the results and/or performing the sort, local re-sort and/or deduplication function on the resulting data, the variant detection function can be used. For example, a typical variant determination function, or portion thereof, may be implemented in a software and/or hardware configuration, such as an integrated circuit. Specifically, variant detection is a process that involves positioning all reads aligned to a given reference location into groupings such that all overlapping regions from all the different aligned reads form a “cluster.” The cluster of reads covering a given region of the reference genome is then analyzed to determine what the individual's actual DNA/RNA probe content is most likely to be in that region. This is then repeated in stages for each region in the genome. From the received content, it generates a list of differences called “variations” or “variants” of the reference genome, indicating for each of them the corresponding level of confidence and other metadata.

[00249] The most common variants are single nucleotide polymorphisms (SNPs), in which one base differs from the reference. SNPs occur approximately 1 time per 1000 positions in the human genome. The next most common are insertions (insertions in the reference) and deletions (omissions in the reference). They are most often found in short lengths, but can be of any length. However, additional difficulties arise because, due to the random selection of sequenced segments (“reads”), some regions will have deeper coverage than others. There are also many complex variants that involve multiple base substitutions and combinations of indels and substitutions that can be perceived as length-altering substitutions. Standard software-based variant identifiers have difficulty identifying all of this and have varying limitations on variant lengths. More specialized variant detectors in software and/or hardware implementation are needed to detect longer variations and many variations of exotic “structural variants” involving large chromosomal changes.

[00250] However, identifying variants is complex to implement in software and orders of magnitude more complex to deploy in hardware. To account for and/or detect these types of errors, standard variant determiners may perform one or more of the following tasks. For example, they might start with a set of hypothetical genotypes (the content of one or two chromosomes at a locus) and use Bayesian calculations to estimate the posterior probability that each genotype is true in light of the observed supporting evidence, and report the most likely genotype along with the level of confidence. Because of this, option qualifiers can be simple or complex. Simple variant detectors look only at the columns of bases in the cluster of aligned reads at the exact position where the detection is made. More advanced variants of determinants are "haplotype-based determinants", which can be made to take into account context, for example in a window around the definition being made.

[00251] A "haplotype" is the specific DNA content (nucleotide sequence, list of variants, etc.) in one common "strand", such as one or two diploid strands in a region, and the haplotype determiner takes into account Bayesian implications of what differences are related by appearing in the same read. Accordingly, the variant detection protocol that is presented herein may implement one or more functions, such as those performed by the Genome Analysis Tool Kit (GATK) haplotype determiner and/or the Hidden Markov Model (HMM) tool, and/or the graph function de Bruijn, for example, where one or more of these functions typically performed by the GATK haplotype finder and/or the HMM and/or the de Bruijn graph function may be implemented in software and/or hardware.

[00252] More specifically, in accordance with the implementations herein, various different variant determination operations may be capable of being implemented in software or hardware, and may include one or more of the following steps. For example, the variant detection function may include hot region detection, for example, to identify locations where multiple reads do not match the reference, and to form a window around the identified hot region so that only that region can be selected for further processing. In addition, localized haplotype assembly is possible, for example when, for each given active region, all overlapping reads can be assembled into a “De Bruijn graph” (DBG) matrix. From this DBG, different routes can be extracted through the matrix, where each route forms candidate haplotype, for example, hypotheses about what the true DNA sequence may be on at least one strand. In addition, haplotype alignment is possible, for example where each isolated candidate haplotype can be aligned, for example using the Smith-Waterman algorithm, back to the reference genome to determine which reference variations it represents. In addition, a read likelihood calculation can be performed, for example where a read can be tested against each haplotype or hypothesis to estimate the likelihood of observing a read under the assumption that the haplotype was derived from the true source DNA.

[00253] With regard to these processes, calculating the read likelihood will typically be the most resource-intensive and time-consuming operation, often requiring pairwise HMM estimation. Additionally, constructing de Bruijn graphs for each cluster of reads, together with the associated operations of local or global identification of unique K-mers, as described below, can also be resource-intensive and/or time-consuming. Accordingly, in various embodiments, one or more of the various calculations related to performing one or more of these steps may be capable of being implemented in an optimal manner in software or hardware, for example, capable of being implemented in an accelerated manner by integrated circuitry, as described in this document.

[00254] As discussed above, in various embodiments, the haplotype determiner of the present invention, implemented in software and/or hardware or a combination thereof, may be configured to include one or more of the following operations: active region detection, localized assembly haplotype, haplotype alignment, read likelihood calculation and/or genotyping. For example, the devices, systems and/or methods of the present invention may be configured to perform one or more of mapping, alignment and/or sorting operations on data obtained from a subject's sequenced DNA/RNA to form mapped, aligned and/or sorted these results. This result data may then be cleaned, for example by performing a duplicate removal operation on it, and/or the data may be passed to one or more specialized haplotype determiner processing engines to perform a variant determination operation, including one or more of the above steps, on these results data to thereby generate a variant definition file. Therefore, all reads that have been sequenced and/or mapped and/or aligned to specific positions in the reference genome can be further processed to determine how a particular sequence differs from the reference sequence at any given point in the reference genome.

[00255] Accordingly, in various embodiments, an apparatus, system, and/or method of use thereof that are described herein may include a variant or haplotype detector system that is implemented in software and/or hardware configuration for performing an active region detection operation on the obtained results data. Hot spot detection involves identifying and defining locations where multiple reads, such as in a read cluster, are inconsistent with a reference, and also involves forming one or more windows around the differences (“hot spots”) so that the area within the window could be selected for further processing. For example, during the mapping and/or alignment step, identified reads are mapped and/or aligned to the region in the reference genome from which they are expected to come from the subject's genetic sequence.

[00256] However, because sequencing is performed in such a way as to oversample sequenced reads for any given region of the genome, at any given position in the reference sequence one can see a cluster of any and/or all sequenced reads that are anchored and aligned to that region. All of these reads that align and/or overlap at a given region or cluster position can be entered into the variant finder system. Thus, any given read being analyzed can be compared to a reference in its putative region of overlap, and that read can be compared to the reference to determine whether its sequence has any differences from the known reference sequence. If the read fits onto the reference without any insertions or deletions and all bases are the same, then the alignment is defined as good.

[00257] Therefore, any given mapped and/or aligned read may have bases that differ from the reference, for example, a read may contain one or more SNPs forming a position where the base does not match; and/or the read may have one or more insertions and/or deletions, for example creating a gap in the alignment. Accordingly, in any of these cases there will be one or more mismatches that need to be taken into account in further processing. However, to save time and improve efficiency, such further processing should be limited to cases where the detected discrepancy is non-trivial, e.g., non-interfering, difference. When determining the significance of a discrepancy, places where multiple reads in a cluster do not match the reference can be identified as an active region, then a window around this active region can be used to select a divergence locus, which can then be further processed. However, the difference must be non-trivial. This can be determined in a variety of ways, for example, for each locus considered, it can be calculated that it is likely not to belong to the reference, for example, by analyzing a quality score based on base matches and mismatches such that exceeding a given threshold is considered a sufficiently significant indicator that these reads are not consistent with reference to a large extent.

[00258] For example, if 30 of the mapped and/or aligned reads stack and/or overlap to form a cluster at a given position in the reference, for example in the active region, and only 1 or 2 of these 30 reads do not agree with the reference, then it can be considered that the minimum threshold for further processing is not satisfied and discordant reads can be ignored, given the fact that 28 or 29 reads are consistent. However, if 3, or 4, or 5, or 10, or more reads in a cluster are discordant, then the discordance can be considered statistically significant enough to justify further processing, and an active region can be identified around the identified areas of difference. In such a case, a hot-region window defining the grounds surrounding the difference can be considered to improve the context of the region surrounding the difference, and additional processing steps can be taken such as applying a Gaussian distribution and summing the non-reference probabilities distributed over all adjacent parts to further examine and process this region to determine whether an active region needs to be declared and, if so, what variations relative to the reference are actually present in this region, if any. Therefore, hot spot detection identifies areas where additional processing may be needed to determine whether variation is actually occurring or an error has occurred.

[00259] In particular, since in many cases it is not desirable to subject every region in a cluster of sequences to further processing, it is possible to define an active region that contains only those regions where further processing may be required to clearly determine whether it is indeed a variation or an error has occurred , which may be determined to require further processing. And, as stated above, the size of the active region window can be determined by the size of the expected variation. For example, in various cases, the active window boundaries may vary from 1 or 2, or from about 10 to 20, or even from about 25 to about 50, or from about 200 to about 300, or from about 500 to about 1000 bases or more, Moreover, further processing occurs only within the boundaries of the active window. Of course, the active window size can be any suitable length such that it provides context for determining the statistical significance of the difference.

[00260] Therefore, if there are only one or two isolated differences, the active window may need to span one or more of several dozen bases in the active region in order to have enough context to statistically determine the presence of a valid variant. However, if there is a cluster or group of differences, or if there are indels that require more context, the window can be made larger. In any case, it may be necessary to analyze any and all differences that may exist in clusters to analyze them all in one or more active windows, as this can provide supporting information about each individual difference and will save processing time by reducing the number of active windows affected. In various cases, the boundaries of the active region can be defined by active probabilities that satisfy a given threshold, such as from about 0.00001, or about 0.00001, or about 0.0001 or less, to about 0.002, or about 0.02. or about 0.2 or more. And if the active region is longer than this threshold, for example, about 300-500 bases or 1000 bases or more, then this region can be divided into subregions, for example, into subregions defined by the locus with the lowest active probability score.

[00261] In various cases, after identifying the active region, a localized haplotype assembly procedure can be performed. For example, in each active region, all clustered and/or overlapping reads can be collected into a “de Bruijn graph” (DBG). The DBG may be a directed graph based on all reads that overlap a selected active region, where the active region may be about 200 bases long, or about 300 bases, or about 400 bases, or about 500 bases or more, and within that active region the presence and /or identity of options. In various cases, as noted above, the active region may be extended, for example by including about 100 more or about 200 or more bases in each direction of the locus of interest to form an extended active region, for example when additional surrounding context may be required. Accordingly, it is in the active region window, elongated or not, that all reads that have parts that overlap the active region are collected into a cluster, for example, to create a cluster, the overlapping parts are identified and the read sequences are sent to the haplotype determiner system and thereby collected together as Count de Bruijn, which is much like putting together a jigsaw puzzle.

[00262] Accordingly, in any given active window there will be reads that form a cluster such that the cluster as a whole contains a sequence path whereby overlapping regions of the various overlapping reads in the cluster cover the entire sequence within the active window. Thus, at any given locus in the active region there will be many reads that overlap that locus, although any given read may not extend across the entire active region. As a result, different regions of different reads within a cluster are used in DBG to determine whether there is actually a variant for any given locus in the sequence within the active region or not. Since this determination is made within the active window, it is the parts of any given read within the boundaries of the active window that are taken into account, and the parts outside the active window can be discarded.

[00263] As stated, it is those sections of reads that overlap the reference within the active region that are fed into the DBG system. The DBG system then assembles the reads, like a mosaic, into a graph, and then, for each position in the sequence, based on the collection of overlapping reads for that position, it determines whether there is a match or a mismatch for any given read, and if there is a mismatch, what is the probability of that mismatch. For example, when there are discontinuous places where segments of reads in a cluster overlap each other, they can be aligned with each other based on their match areas, and by stringing or stitching together matching reads based on their match points, for each position within that segment, it is possible to determine whether, and to what extent, the reads at any given position match or mismatch. Therefore, if two or more reads overlap and identically coincide with each other for some time, a single-line graph is obtained; however, once two or more reads hit a point where they differ, a branch is formed in the graph and two or more diverging rows are obtained until the two or more reads match again.

[00264] Consequently, paths through the graph are often not a straight line. For example, when the k-mers of a read differ from the k-mers of the reference and/or the k-mers of one or more overlapping reads, such as in a cluster, a “bubble” is formed in the graph at the point of difference, resulting in two divergent rows that continue along two different paths until the two sequences match again. Each vertex can be assigned a weighted score indicating how many times the corresponding k-mers overlap across all reads in the cluster. In particular, each path passing through the generated graph from one side to the other can be assigned a counter. And when the same k-mers are formed from multiple reads, for example when each k-mer has the same sequence structure, they can be counted in the graph by incrementing the counter for that path, where the k-mer overlaps an already existing path k -measure. Thus, when the same k-mer is formed from many overlapping reads having the same sequence, the structure of the path through the graph will be repeated over and over again, and the counter for that path through the graph will be incremented accordingly. In this case, the structure is recorded only for the first instance of the k-mer, and the counter is incremented for each k-mer that repeats this structure. In this mode, it is possible to obtain different reads in a cluster to determine what variations occur and where.

[00265] Similarly, a graph matrix can be formed by taking all possible N-base k-mers, for example 10-base k-mers, which can be formed from any given read by sequentially traversing the entire length of the read in ten-base segments, where the start every new ten bases is offset by one base from the last 10-base segment formed. This procedure can then be repeated, doing the same for every other read in the cluster within the active window. The formed k-mers can then be aligned with each other so that regions of identical matches between the formed k-mers coincide with the regions where they overlap, thereby constructing a data structure, such as a graph, which can then be scanned and the percentage of match determined and mismatches. In particular, the reference and any previously processed k-mers aligned to it can be scanned against the next generated k-mer to determine whether the current generated k-mer matches and/or overlaps with any part of the previously generated k-mer , and where a match is found, the current k-mer being generated can be inserted into the graph at the appropriate position.

[00266] Once the graph is constructed, it can be scanned and based on this comparison, it can be determined whether any SNP data and/or indels relative to the reference may be an actual variation in the subject's genetic code, or whether they are the result of a processing error or other error. For example, if all or a significant portion of the k-mers of all or a significant portion of the reads in a given region contain the same mismatch in the form of an SNP and/or indel, but differ from the reference in the same way, then it can be determined that there is indeed a variation in the subject's genome in the form of SNPs and/or indels compared to the reference genome. However, if only a limited number of k-mers from a limited number of reads exhibit an artifact, this is likely due to machine and/or processing error and/or other error and does not indicate the presence of true variation at that position of interest.

[00267] As stated, where variation is suspected, a bubble will be formed in the graph. Namely, where all k-mers in the entire given region of reads correspond to the reference, they will line up in such a way that they form a linear graph. However, where there is a difference between bases at a given locus, the graph will branch at that locus with a difference. This branch can be at any position within the k-mer, and hence at this point the 10-base k-mer difference that includes this difference will deviate from the rest of the k-mers in the graph. In this case, a new node will be formed, forming a different path through the graph.

[00268] Therefore, where everything can be consistent, for example, the sequence in a given new k-measure being plotted on the graph is the same as the sequence it is aligned to in the graph up to the point of difference, the path for that k-measure will match the path for the graph as a whole will be linear, but after the point of dissimilarity a new path through the graph will appear to accommodate the dissimilarity represented by the k-mer sequence again plotted on the graph. This divergence will be represented by a new node in the graph. In such a case, any new k-mers to be added to the graph that coincides with the newly diverging path will increment the counter at that node. Therefore, for each read that supports an edge, the counter will be incremented.

[00269] In various such cases, the k-mer and/or the read it represents will someday begin to match again, for example after a point of divergence, so that there will now be a convergence point where the k-mer begins to coincide with the main path through the graph , represented by k-mers of the reference sequence. For example, naturally, after some time, reads that support a forked node must rejoin the graph over time. Thus, over time, the k-mers for a given read will rejoin the main path. More specifically, in the case of an SNP at a given locus within a read, the k-mer starting at that SNP will deviate from the main path and remain separated for approximately 10 nodes, since there are 10 bases in the k-mer that overlap this mismatch locus between read and reference. Thus, in the case of a SNP at position 11, the k-mers covering that locus within the read will rejoin the main pathway as the exact match is resumed. Therefore, for the k-mers of a read that has a SNP at a given locus, ten shifts will be required to rejoin the main graph represented by the reference sequence.

[00270] As stated above, there is usually one main path or line or spanning tree, i.e., the reference path, and when there is a divergence, a bubble is formed at the node where there is a difference between the read and the spanning graph. Thus, there are some reads that deviate from the backbone and form a bubble, and this divergence may indicate the presence of a variant. As the graph is processed along the reference skeleton, bubbles within bubbles within bubbles can form so that they form a stack and multiple paths through the graph can be created. In such a case, there may be a main path represented by the reference skeleton, one path of the first divergence, and another path of the second divergence within the first divergence, all within a given window, and each path through the graph may represent a real variation or may be an artifact, for example caused by sequencing error, and/or PCR error, and/or processing error, etc.

[00271] Once such a graph has been constructed, it is necessary to determine which paths through the graph represent actual variation within the sample's genome and which are merely artifacts. While it is expected that reads containing processing or machine errors will not be supported by the majority of reads in a sample aggregation, this is not always the case. For example, errors in PCR processing may typically result from a cloning error that occurs during DNA sample preparation, with such errors typically resulting in the addition of an insertion and/or deletion to the cloned sequence. Such indel errors can be more consistent between reads and can be twisted to form multiple reads that have the same error due to a given error in PCR cloning. Therefore, such errors may result in a higher count line for such a point.

[00272] Thus, once a graph matrix with multiple paths through the graph has been generated, the next stage is to traverse, and thereby isolate, all paths through the graph, for example, from left to right. One path will be the reference skeleton, but there will be other paths that follow different bubbles along the path. It is necessary to go through all the paths and enter their counters into the table. For example, if a graph contains a path with a two-level bubble in one place and a three-level bubble in another place, there will be (2x3) ⁶ paths through that graph. Therefore, each of the paths must be isolated separately, and the selected paths are called candidate haplotypes. Such candidate haplotypes represent hypotheses as to what might actually represent the subject's actual DNA that has been sequenced, and subsequent processing steps, including one or more of haplotype alignment, read likelihood calculation, and/or genotyping, can be used to test these hypotheses to find the probabilities that any and/or each of these hypotheses is true. Therefore, implementing de Bruijn graph reconstruction provides a way to reliably isolate a good set of hypotheses to test.

[00273] For example, when performing a variant detection function as described herein, a hot region detection operation may be implemented, for example, to identify locations where multiple reads in a cluster within a region are not consistent with the reference, and to form a window around the identified active area so that only this area can be selected for further processing. In addition, localized haplotype assembly can take place, for example when, for any given active region, all overlapping reads in a cluster can be assembled into a "de Bruijn graph" (DBG) matrix. From this DBG, different routes can be extracted through the matrix, where each route forms a candidate haplotype, for example, a hypothesis for which the true DNA sequence may be on at least one strand.

[00274] In addition, haplotype alignment is possible, for example, where each isolated candidate haplotype can be aligned, for example, using the Smith-Waterman algorithm, back to the reference genome to determine which variations of the reference it represents. In addition, a read likelihood calculation can be performed, for example where a read can be tested against each haplotype to estimate the likelihood of observing a read suggesting that the haplotype was derived from the true source DNA. Finally, a genotyping operation can be implemented and a variant finding file can be created. As stated above, any or all of these operations can be implemented in an optimized manner in software and/or hardware, and in various cases due to the resource-intensive and time-consuming nature of constructing a DNG matrix and isolating candidate haplotypes from it and/or Due to the resource-intensive and time-consuming nature of performing haplotype alignment and/or read likelihood calculation, which may involve the use of Hidden Markov Model HMM estimation), these operations (e.g., localized haplotype assembly, and/or haplotype alignment, and/or read likelihood calculation) or portions thereof may be configured to implement one or more of the functions of these operations in hardware, for example, to perform them in an accelerated manner by an integrated circuit as described herein. In various cases, these tasks may be accomplished by one or more quantum circuits, such as a quantum computing device.

[00275] Accordingly, in various cases, devices, systems and methods for using them may be configured to perform haplotype alignment and/or read likelihood calculations. For example, as noted, each isolated haplotype can be aligned, for example using the Smith-Waterman algorithm, back to the reference genome to determine which reference variations it represents. In various example cases, evaluation may be performed, for example, according to the following example evaluation parameters: compliance = 20.0; mismatch = -15.0; opening gap = -26.0; and gap extension = -1.1; other evaluation parameters may be used. Accordingly, a CIGAR string can be generated in this manner and associated with a haplotype to create an assembled haplotype, where the assembled haplotype can ultimately be used to identify variants. Accordingly, the likelihood of a given read associated with a given haplotype can be calculated in a similar way for all read/haplotype combinations. In such cases, the likelihood can be calculated using a hidden Markov model (HMM).

[00276] For example, different assembled haplotypes can be aligned according to a dynamic programming model, similar to Smith-Waterman alignment. In such a case, a virtual matrix can be formed, for example, where a candidate haplotype, for example generated by DBG, can be located on one axis of the virtual array, and a read can be located on the other axis. The matrix can then be populated with scores generated by walking the highlighted paths through the graph and calculating the probabilities that a given path is the true path. Therefore, in such a case, the difference between this alignment protocol and a typical Smith-Waterman alignment protocol is that to find the most likely path through the array, a maximum likelihood computation is used, such as that performed by an HMM model, which is designed to provide full probability for alignment of reads with a haplotype. Therefore, there is no need to perform actual CIGAR string alignment in this case. Instead, all possible alignments are considered and their probabilities are summed. Evaluating a pairwise HMM is resource-intensive and time-consuming, and therefore implementing its operations in a hardware configuration on an integrated circuit or using quantum circuits on a quantum computing platform is of great advantage.

[00277] For example, each read with each potential haplotype can be tested to estimate the probability of observing the read under the assumption that the haplotype is a true representation of the original DNA sample. In various cases, this calculation can be done by estimating a "pairwise Hidden Markov Model" (HMM), which can be done to model the various possible ways in which a candidate haplotype could have been altered, such as PCR or sequencing errors, etc. and the variation introduced into the observed read. In such cases, HMM estimation can be performed using a dynamic programming technique to calculate the overall probability of any sequence of transitions between Markov states reaching the observed read, in the form of the possibility that any discrepancy in the read could be the result of an error model. Accordingly, such HMM calculations can be made to analyze all possible SNPs and indels that may have been introduced into one or more reads, for example due to amplification and/or sequencing artifacts.

[00278] Specifically, the pairwise HMM takes into account in a virtual matrix all possible alignments of a read to candidate reference haplotypes along with the probability associated with each of them, where all probabilities are summed. To get one overall probability for each read, add up all the probabilities of all options along a given path. This process is then performed for each haplotype and read pair. For example, if there is a cluster of six clusters overlapping a given region, such as a region of six candidate haplotypes, and if the cluster contains about a hundred reads, 600 HMM operations would be required. More specifically, if there are 6 haplotypes, then there must be 6 branches along the path, and the probability must be calculated that each of them is the correct path that matches the subject's actual genetic code for that region. Therefore, you need to consider each path for all reads, and you need to calculate the probability for each read that you will arrive at that given haplotype.

[00279] A pairwise hidden Markov model is an approximate model of how a true haplotype in a DNA sample can evolve into a possible different detectable read. It has been observed that these types of transformations are a combination of SNPs and indels that were introduced into the genetic sample set by the PCR process, one or more other sample preparation steps, and/or error caused by the sequencing process, etc. As shown in FIG. 2, to account for these types of errors, an underlying 3-state base model can be used where: (M = alignment match, I = insertion, D = deletion), and where any transition other than I <-> D is possible.

[00280] As shown in FIG. 2, containing the transitions of the 3-state base model are not made in a time sequence, but rather in a progression sequence through the candidate haplotype and read sequences, starting from position 0 in each sequence, where the first base is at position 1. Transition to M means position +1 in both sequences; going to I means position +1 of the read sequence only; and going to D means position +1 in the haplotype sequence only. The same 3-state model can also be configured as the basis for the Smith-Waterman and/or Needleman-Wunsch algorithms described herein. Accordingly, such a 3-state model as set forth herein can be used in the Smith-Waterman and/or Needleman-Wunsch process to thereby provide an affine gap (indel) estimate, which assumes that the opening of a gap (Introduction I or D) it will be less likely that the gap will be extended (continuing to remain in state I or D). Thus, in this case, the pairwise HMM can be thought of as an alignment, and a CIGAR string can be created to encode the sequence of different state transitions.

[00281] In various cases, the basic 3-state model can be complicated by allowing the probabilities to vary depending on the position. For example, the probabilities of all M transitions can be multiplied by the non-priori probabilities of observing the next base of a read, determined by its estimate of the quality of the base and the corresponding next base of the haplotype. In this case, base quality estimates can be converted into the probability of SNP error during sequencing. When two bases coincide, the prior probability is taken as one minus the probability of this error, and when they do not coincide, the probability of the error divided by 3 is taken, since there are 3 possible results of the SNP.

[00282] The abstract “Markov” model is discussed above. In various cases, a maximum likelihood sequence of transitions, referred to herein as alignment, may also be determined and may be performed using the Needleman-Wunsch algorithm or other dynamic programming algorithm. But, in various embodiments, when performing the variant determination function as described herein, the maximum likelihood alignment or any particular alignment should not be of paramount importance. Rather, one can calculate the total probability, for example, by calculating the total probability of observing a read of a given haplotype, which is the sum of the probabilities of all possible paths through the graph from the zero read position at any haplotype position to the final read position at any haplotype position, where the probability of each component path is simply the product of the probabilities of the various component transitions.

[00283] Finding the sum of path probabilities can also be done using a virtual array and a dynamic programming algorithm as described above, for example, in each cell of the matrix (0 ... N) × (0 ... M) there are three calculated probability values corresponding to the transitions in states M, D and I (or, what is the same, there are 3 matrices). The top row (zero read position) of the matrix can be initialized with probability 1.0 in states D and probability 0.0 in states I and M; and the rest of the left column (the zero haplotype position) can be initialized to all zeros. (In software, the initial probabilities D can be set close to the maximum double precision value, such as 2^1020, to avoid going to machine zero, but this factor can be normalized later.)

[00284] This 3-to-1 calculation dependency limits the order in which cells can be calculated. They can be calculated from left to right in each row, moving from top to bottom, or from top to bottom in each column, moving to the right. In addition, they can be calculated in antidiagonal wavefronts, where the next step is to calculate all cells (n, m), and n+m is equal to the incremental step number. The advantage of this wavefront order is that all cells on the antidiagonal can be calculated independently of each other. Then the bottom row of the matrix at the end position of the read can be configured to represent the calculated alignments. In this case, the haplotype determiner will operate by summing the probabilities I and M of all cells in the bottom row. In various implementations, the system can be configured to disable D transitions in the bottom row, or use a 0.0 D transition probability there to avoid double counting.

[00285] As described herein, in various cases, each HMM score may operate on a pair of sequences, for example, a pair of a candidate haplotype and a read. For example, within a given active region, each set of haplotypes can be estimated using an HMM relative to a set of reads. In such a case, the bandwidth at the input of the software or hardware can be reduced and/or minimized by transmitting the read set and the haplotype set once and allowing the software and/or hardware to generate N×M pairs of operations. In certain cases, the Smith-Waterman estimator may be configured to queue individual HMM operations, each with its own copy of the read and haplotype data. The Smith-Waterman (SW) alignment module may be configured to perform pairwise HMM computation in linear space or may operate in logarithmic probability space. This is useful for maintaining precision across the huge range of probability values represented by fixed-point numbers. However, in other cases floating point operations may be used.

[00286] There are three parallel multiplications (eg summations in logarithmic space), then two sequential summations (~5-6 step approximation pipelines), then another multiplication. In this case, the complete pipeline can be about L = 12-16 cycles long. I&D calculations can be about half the length. The pipeline may be fed multiple input probabilities, for example 2, or 3, or 5, or 7, or more input probabilities in each cycle, for example, from one or more already calculated neighboring cells (M and/or D on the left, M and/ or I top, and/or M and/or I and/or D top left). It may also contain in each cycle one or more haplotype bases and/or one or more read bases, for example, together with associated parameters such as preprocessed parameters. It outputs a final set of M&I&D for one cell per cycle after end-to-end delay.

[00287] As stated above, when performing the variant determination function as described herein, a de Bruijn graph can be constructed, and when all reads in a cluster are identical, the DBG will be linear. However, when there are differences, "bubbles" will form in the graph, which indicate areas of difference, resulting in multiple paths diverging from the alignment match to the reference, and then later reconnecting at the alignment match. From this DBG, it is possible to extract various pathways that generate candidate haplotypes, for example, hypotheses as to what the true DNA sequence on at least one strand might be, and these hypotheses can be tested by performing an HMM, or modified HMM, operation on these data. Moreover, the genotyping function can be used, for example, where diploid combinations of candidate haplotypes can be generated, and for each of them the conditional probability of observing the entire cluster of reads can be calculated. These results can then be fed into a Bayesian formula module to calculate the unconditional probability that each genotype is true, given that the entire population of reads is observed.

[00288] Therefore, in accordance with the devices, systems and methods of using them described herein, in various cases, a genotyping operation can be performed, and this genotyping operation can be performed in an optimized manner in software and/or hardware software, and/or a quantum processing device. For example, one can generate possible diploid combinations of candidate haplotypes and, for each combination, calculate the conditional probability of observing the entire cluster of reads, for example, using the composite probabilities of observing each given read of each haplotype based on the pairwise HMM estimate. The results of these calculations are fed into the Bayesian formula module to calculate the unconditional probability that each genotype is true, given that the entire population of reads is observed.

[00289] Accordingly, in various aspects, the present invention relates to a system for performing the operation of determining haplotypes or variants on generated and/or provided data, thereby creating a variant definition file. Namely, as described above, in some cases the variant definition file may be a digital or other similar file that encodes the differences between one sequence and another, for example, the difference between a sample sequence and a reference sequence. In particular, in various cases, a variant definition file may be a text file that states or otherwise details genetic and/or structural variations in the genetic makeup of an individual compared to one or more reference genomes.

[00290] For example, a haplotype is a set of genetic, eg DNA and/or RNA, variations, such as polymorphisms, that are present on a person's chromosomes and as such can be passed on to a descendant and thereby inherited. In particular, a haplotype may refer to a combination of alleles, such as one of many alternative forms of a gene that may arise as a result of mutation, the allelic variations typically being found at the same location on the chromosome. Therefore, when determining the identity of an individual's genome, it is important to know which form of the many different possible alleles a particular individual's genetic sequence encodes. In specific cases, a haplotype may refer to one or more, for example, a set, of nucleotide polymorphisms (eg, SNPs) that can be found at the same position on the same chromosome.

[00291] Typically, in various embodiments, a software-based algorithm, such as an algorithm using a haplotype determination program, such as GATK, can be used to determine the genotype, e.g., allelic haplotypes, for a subject, as described above herein. for simultaneous determination of SNPs and/or insertions and/or deletions, i.e. indels, in an individual's genetic sequence. In particular, the algorithm may include one or more haplotype assembly protocols, for example, to locally assemble a haplotype from the beginning in one or more active regions of the genetic sequence being processed. Such processing typically involves deploying a processing function called a hidden Markov model (HMM), which is a stochastic and/or statistical model used to implement randomly changing systems, for example, under the assumption that future states in the system depend only on the present state, and not from a series of events preceding it.

[00292] In such cases, the system being modeled has the characteristics of, or is otherwise assumed to be, a Markov process with unobservable (hidden) states. In specific cases, the model may contain a simple dynamic Bayesian network. In particular, with regard to the definition of genetic variation in its simplest form, there is one of four possibilities for identifying any given base in a processed sequence, for example, by comparing a segment of a reference sequence, such as a hypothetical haplotype, and a segment of a subject's DNA or RNA sequence, such as a read. obtained from the sequencer. However, to identify such variation, the subject's DNA/RNA must first be sequenced, for example using a next-generation sequencer (NGS), to create a read, or “reads,” that define the subject's genetic code. Next, after a subject's genome has been sequenced to obtain one or more reads, the various representations representing the subject's DNA and/or RNA need to be mapped and/or aligned, as described in more detail hereinbefore. The next step of processing is then to determine how the subject's genes that have just been identified, eg, mapped and/or aligned, differ from those of the prototypical reference sequence. Therefore, when performing such an analysis, it is assumed that the read potentially representing a given gene in a subject is a representation of the prototypical haplotype, albeit with different SNPs and/or indels that have been identified to date.

[00293] Namely, in accordance with specific aspects, devices, systems and/or methods for their practical application are provided, for example, to perform a haplotype determination function and/or variants, for example, deploying an HMM function, for example, in an accelerated haplotype determiner. In various cases, to overcome these and other similar problems known in the art, the HMM accelerator presented herein may be configured to perform operations in such a way that it can be implemented in software, implemented in hardware, or implemented and/or otherwise controlled in a combination of part software and/or part hardware, and/or with the possibility of using quantum computing implementations. For example, in accordance with a particular aspect, the invention relates to a method by which data relating to the DNA and/or RNA sequence identity of a subject can be determined, and/or how the subject's genetic information may differ from a reference genome.

[00294] In such a case, the method may be implemented by implementing a haplotype or variant detection function, for example, using the HMM protocol. In particular, the HMM function may be implemented in software, hardware, or through one or more quantum circuits, for example, on an accelerated device, in accordance with the method described herein. In such a case, the HMM accelerator may be configured to receive and process the sequenced, mapped and/or aligned data to process it, for example, to create a variant definition file, as well as to transmit the processed data back throughout the system. Accordingly, the method may include deploying a system in which data can be sent from a processor, such as a software-controlled CPU or GPU or even CPU, to a haplotype determiner implementing an accelerated HMM, which haplotype determiner may be deployed on a microprocessor chip, such as an FPGA, ASIC, or structured ASIC, or may be implemented using one or more quantum circuits. The method may also include data processing steps to obtain resultant HMM data, which results may then be fed back to the CPU and/or GPU and/or GPU.

[00295] Specifically, in one embodiment, as shown in FIG. 3A, a bioinformatics pipeline system incorporating an HMM accelerator is proposed. For example, in one case, the bioinformatics pipeline system may be implemented as a variant determination system 1. The system in the figure is depicted in hardware implementation, but could also be implemented by one or more quantum circuits, such as a quantum computing platform. Namely, in FIG. 3A is a high-level view of the HMM interface structure. In specific embodiments, the variant determination system 1 is configured to speed up at least a portion of a variant determination operation, such as an HMM operation. Therefore, in this document, in various cases, the variant determination system may be referred to as HMM system 1. System 1 includes a server having one or more processing devices 1000 (CPU/GPU/CPU) configured to execute one or more standard programs related to sequencing and/or processing genetic information, for example, for comparing a sequenced genetic sequence with one or more more reference sequences.

[00296] In addition, the system 1 includes a peripheral device 2, such as an expansion card, which contains a chip 7, such as an FPGA, ASIC or sASIC. In some cases, one or more quantum circuits may be provided capable of performing various operations specified herein. It should also be noted that the term ASIC can equally mean structured ASIC (sASIC), where appropriate. Peripheral device 2 includes an interconnect 3 and a bus interface 4, such as a parallel or serial bus, that connects the CPU/GPU/CPU 1000 to the chip 7. For example, device 2 may include a peripheral component interconnect such as PCI, PCI-X, PCIe, or QPI (Quick Access Interconnect), and may include a bus interface 4 that is configured to functionally and/or communicatively connect the CPU/GPU/CPU 1000 to the device 2, for example for high-speed, low-latency data transfer. Accordingly, in specific cases, the interface may be a peripheral component interconnect express (PCIe) interface 4, which is coupled to a chip 7, the chip including an HMM accelerator 8. For example, in specific cases, the HMM accelerator 8 is configured to implement the HMM function, for example, where the HMM function in certain embodiments may be at least partially implemented in FPGA, AISC, or sASIC hardware or through one or more suitably configured quantum circuits.

[00297] Specifically, in FIG. 3A is a high-level view of an HMM accelerator 8 with an exemplary arrangement of one or more engines 13, such as a plurality of _m+1 engines 13a-13, to perform one or more case determination function processes, eg, including an HMM task. Accordingly, the HMM accelerator 8, which may consist of a data distributor 9, such as CentCom, and one or a plurality of _n+1 processing clusters 11-11, which may be organized as, or otherwise include, one or more instances 13, for example, when each instance can be implemented in the form of processing engines, such as the small-sized engine 13a-13 _m+1 . For example, the distributor 9 may be configured to receive data, for example, from the CPU/GPU/CPU 1000, and distribute or otherwise transmit this data to one or more of a plurality of HMM processing clusters 11.

[00298] In particular, in certain embodiments, the distributor 9 may be located logically between the embedded PCIe interface 4 and the HMM accelerator module 8, for example, when the interface 4 communicates with the distributor 9, for example, over an interconnect or other suitably configured bus 5, for example, via the PCIe bus. The distributor module 9 may be configured to communicate with one or more HMM accelerator clusters 11, for example, over one or more cluster buses 10. For example, the HMM accelerator module 8 may be configured as, or otherwise include, an array of clusters 11a- 11 _n+1 , for example, where each HMM cluster 11 may be configured as or otherwise include a cluster hub 11 and/or may include one or more instances 13, wherein the instance may be a processing engine 13 that is executed with the ability to perform one or more operations on the data received by it. Accordingly, in various embodiments, each cluster 11 may be configured as, or otherwise include, a hub 11a-11 _n+1 , where each of the hubs may be configured to operably communicate with a plurality of engine instances 13a-13 _m+1 HMM accelerator, for example where each cluster hub 11 may be configured to route data to a plurality of _m+1 processing engines 13a-13 within the cluster 11.

[00299] In various cases, the HMM accelerator 8 is configured to compare each base of a subject's sequenced genetic code, for example in read format, with various known or generated candidate reference sequence haplotypes and determine the probability that any given base at the position being studied is either the same , or does not coincide with the corresponding haplotype, for example, the read contains an SNP, insertion or deletion, which leads to a variation of the base at the position being studied. Specifically, in various embodiments, the HMM accelerator 8 is configured to assign transition probabilities for a sequence of base reads occurring between each of these states, match (“M”), insertion (“I”), or deletion (“D”), as is described in more detail later in this document.

[00300] More specifically, depending on the configuration, the HMM acceleration function may be implemented in software, for example, by the CPU/GPU/GPU 1000 and/or chip 7, and/or may be implemented in hardware and may be represented by in a chip 7, for example, located on a card or peripheral expansion board 2. In various implementations, this functionality may be implemented partly in software, such as executed by the CPU/GPU/CPU 1000, and partly in hardware implemented on chip 7 or through one or more quantum processing circuits. Accordingly, in various embodiments, chip 7 may be present on the CPU/GPU/CPU motherboard 1000, or it may be part of a peripheral device 2, or both. Accordingly, HMM accelerator module 8 may comprise or otherwise be associated with various interfaces, for example, 3, 5, 10 and/or 12, to enable efficient transfer of data to and from processing engines 13.

[00301] Accordingly, as shown in FIG. 2 and 3, in various embodiments, a chip 7 is provided, configured to perform the function of determining variants, for example haplotypes. The chip 7 can be connected to the CPU/GPU/CPU 1000, for example, directly coupled with them, for example, installed on a computer motherboard, or indirectly connected to them, for example, be part of a peripheral device 2, which is configured to be functionally connected to the CPU /GPU/CPU 1000, for example, through one or more interconnects, for example 3, 4, 5, 10 and/or 12. In this case, the chip 7 is present on the peripheral device 2. It must be understood that although the accelerator is in the form of a chip, it may also be embodied in the form of one or more quantum circuits of a quantum processing device, wherein the quantum circuits are embodied in the form of one or more processing engines for performing one or more of the functions described herein.

[00302] Therefore, the peripheral device 2 may include a parallel or serial expansion bus 4, for example, to connect the peripheral device 2 to the central processing unit (CPU/GPU/CPU) 1000 of a computer and/or server, for example, through an interface 3, such as DMA . In specific cases, the peripheral device 2 and/or serial expansion bus 4 may be a Peripheral Component Interconnect Express (PCIe) interface that is configured to communicate with the chip 7, for example via connection 5. As described herein, the chip 7 may , at least in part, be configured as, or otherwise include, the HMM accelerator 8. The HMM accelerator 8 may be implemented as part of the chip 7, for example, in hardware implementation and/or as code designed to be executed along with it, and is configured to perform a variant determination function, for example, to perform one or more hidden Markov model operations. , on the data supplied to the chip 7 using the CPU/GPU/CPU 1000, for example, via PCIe interface 4. Likewise, after one or more case determination functions are performed, for example, one or more HMM operations are performed, the results thereof may be transferred from the HMM accelerator 8 of the circuit 7 via bus 4 to the CPU/GPU/GPU 1000, for example via connection 3.

[00303] For example, in specific cases, a CPU/GPU/CPU 1000 is provided for processing and/or transmitting information and/or executing instructions along with a chip 7 that is only partially implemented as an HMM accelerator 8. The CPU/GPU/CPU 1000 communicates with the chip 7 via an interface 5 that is configured to facilitate communication between the CPU/GPU/CPU 1000 and the HMM accelerator 8 of the chip 7 and can therefore communicate with the CPU/GPU/CPU. 1000 with an HMM accelerator 8, which is part of the chip 7. To facilitate this function, the chip 7 includes a distributor module 9, which may be a CentCom module configured to communicate data to a plurality of HMM engines 13, for example through one or more clusters 11, where each engine 13 is configured to receive and process data, for example, by executing the HMM protocol on them, calculating the final values, outputting its results, and repeating these actions. In various cases, execution of the HMM protocol may include determining one or more transition probabilities, as described later herein. In particular, each HMM engine 13 may be configured to perform tasks, including one or more formations and/or evaluations of the virtual HMM matrix, thereby creating and outputting a final summary value, wherein the final summary value expresses the probable likelihood that a particular base is the same as or different from a corresponding base in the sequence of a hypothetical haplotype, as described below herein.

[00304] In FIG. 3B is a detailed view of the HMM cluster 11 shown in FIG. 3A. In various embodiments, each HMM cluster 11 contains one or more HMM instances 13. One or multiple clusters may be provided, for example, according to the amount of resources provided, for example, on a chip or quantum computing processor. In particular, an HMM cluster may be provided, which is configured as a cluster hub 11. The cluster hub 11 receives data related to one or more jobs 20 from the distributor 9 and is further connected to communicate with one or more, e.g., multiple, HMM instances 13, e.g., via one or more HMM instance buses 12 to which cluster hub 11 transmits job data to 20.

[00305] The bandwidth for transmitting data throughout the system may be a relatively low-bandwidth process, and once job 20 is received, system 1 may be configured to execute that job, for example, without requiring chip 7 to access memory . In various implementations, at any given time, one job 20a is sent to one processing engine 13a, and multiple jobs 20 _an can be distributed by the cluster hub 11 to multiple processing engines 13a-13 _m+1 , for example, when each of the processing engines 13 is work on one task 20, for example one comparison between one or more reads and one or more haplotype sequences, in parallel and at high speeds. As described below, performing such task 20 may typically include generating a virtual matrix whereby the subject's read sequences can be compared to one or more, eg, two, hypothetical haplotype sequences to determine differences between them. In such cases, one job 20 may include processing one or more matrices having multiple cells that need to be processed for each comparison performed, eg, base by base. Since the human genome is about 3 billion base pairs long, analyzing the human genome with 30 times excess coverage (equivalent to about 20 trillion cells in the matrices of all associated HMM jobs) would require running on the order of 1 to 2 billion different jobs.

[00306] Accordingly, as described herein, each HMM instance 13 may be configured to implement an HMM protocol, such as generating and processing an HMM matrix, on sequence data, such as data received from the CPU/GPU/GPU 1000. For example, As explained above, when sequencing a subject's genetic material, such as DNA or RNA, the DNA/RNA is broken down into segments, for example, up to about 100 bases in length. These 100-base segments are then checked for identity, for example using an automated sequencer, and "read" into a FASTQ text file or other format that stores the identity of each base of the read along with a Phred quality score (for example, this is typically a number from 0 to 63 per logarithmic scale, where a score of 0 indicates the minimum amount of confidence that a particular basis is true, and scores of 20 to 45 are generally considered acceptable as being relatively accurate).

[00307] In particular, as stated above, the Phred quality score is a quality indicator that measures the quality of identification of nucleotide identities generated by a sequencing processor, such as an automated DNA/RNA sequencer. Therefore, each read base contains its own quality score, such as Phred, based on how the sequencer judged the quality of that particular identification. Phred represents the confidence with which the sequencer judges that it has correctly determined the identity of the base. This Phred score is then used by HMM Module 8, as detailed below, to further determine the accuracy of each identified base in a read compared to the haplotype to which it was mapped and/or aligned, such as determining its probabilities of transitioning to "match" states. "insertion" and/or "deletion", for example into and out of the "match" state. It should be noted that in various embodiments, system 1 may modify or otherwise adjust the initial Phred score before executing the HMM protocol using it, for example, by taking into account neighboring bases/scores and/or neighboring DNA fragments and assuming the influence of such factors on the Phred score base, for example, the cell under study.

[00308] In such cases, as shown in FIG. 4, system 1, e.g., computer/quantum software, can determine and identify various active regions 500 _n within the sequenced genome, which can be used and/or otherwise subjected to further processing as described herein, and which can be is broken down into 20 _n jobs that can be parallelized among the various cores and available threads 1007 throughout the system 1. For example, such active regions 500 can be identified as sources of variation between the sequenced and reference genome. In particular, the CPU/GPU/CPU 1000 may have multiple threads 1007 running, identifying active regions 500a, 500b and 500c, compiling and aggregating all sorts of different jobs 20 _n to be executed, for example, through a suitably configured aggregator 1008, based on the active areas 500a-c currently being explored. In order for system 1 to operate efficiently, any suitable number of threads 1007 may be used, for example, the more threads, the less active time spent waiting.

[00309] Upon completion of identification, compilation and/or aggregation, threads 1007/1008 will transmit active jobs 20 to data distributor 9, e.g., CentCom, HMM module 8, e.g., via PCIe interface 4, e.g., in “homing” mode, and then will move on to other processes while waiting for the HMM module 8 to send the output back to align it back with the corresponding active region 500 to which it is mapped and/or aligned. After this, the data distributor 9 distributes the jobs 20 to various different HMM clusters 11, for example, job by job. If everything is running efficiently, it could be a first come, first served format, but it doesn't have to be. For example, in various implementations, raw job data and processed job results may be sent throughout the system as they become available.

[00310] Specifically, in FIG. 2, 3 and 4, the various data of job 20 can be aggregated into pages of 4K bytes, which can be sent via PCIe 4 to and through CentCom 9 and onwards to processing engines 13, for example, via clusters 11. The amount of data sent can be greater than or equal to less than 4K bytes, but will typically contain 100 HMM jobs per 4K data page (e.g. 1024). In particular, this data is then digested by the data distributor 9 and sent to each cluster 11, for example, one 4K page is sent to one cluster 11. However, this is not necessary, since each individual job 20 can be sent to any given cluster 11 depending on which cluster becomes available and when.

[00311] Accordingly, the cluster-based approach 11 as presented here distributes incoming data to processing engines 13 efficiently and at high speed. Specifically, as data arrives on PCIe interface 4 from the CPU/GPU/CPU 1000, for example via DMA connection 3, the received data is then sent via PCIe bus 5 to CentCom distributor 9 of variant determiner chip 7. The distributor 9 then sends the data to one or more HMM processing clusters 11, for example, on one or more dedicated cluster buses 10, which cluster 11 can then send this data to one or more processing instances 13, for example, on one or more buses 12 instances, for example, for processing. In this case, the PCIe interface 4 is configured to supply data through the peripheral expansion bus 5, the distributor 9, and/or the cluster and/or instance buses 10 and/or 12 at a fast speed, for example, one that can keep one or more busy. for example, all _a-(m+1) HMM accelerator instances 13 within one or more, for example all, _a-(n+1) HMM clusters 11, for example, over a long period of time, for example full time, and for During the period during which the system 1 operates, jobs 20 are processed, and at the same time the system manages to output HMM data to be sent back to one or more CPUs 1000 via PCIe interface 4.

[00312] For example, any inefficiency of interfaces 3, 5, 10 and/or 12 that results in a period of downtime of one or more HMM accelerator instances 13 may increase the overall processing time of system 1. In particular, when analyzing the human genome, there may be on the order of two or more than 20 billion different jobs that must be distributed to different 11 HMM clusters and processed over some period of time, for example, less than 1 hour, less than 45 minutes, less than 30 minutes, less than 20 minutes, including 15 minutes, 10 minutes, 5 minutes or less.

[00313] Accordingly, in FIG. 4 provides an overview of an example of data flow throughout the software and/or hardware of system 1, as described generally above. As shown in FIG. 4, the system 1 can be configured, in part, to transfer data, for example, between the PCIe interface 4 and a distributor 9, such as CentCom, for example, over a PCIe bus 5. In addition, the system 1 may also be configured, in part, to transmit received data, for example, between the distributor 9 and one or more HMM clusters 11, for example, via one or more cluster buses 10. Thus, in various embodiments, the HMM accelerator 8 may comprise one or more clusters 11, for example, one or more clusters 11 configured to implement one or more HMM function processes. In such a case, there is an interface, such as a cluster bus 10, that connects the CentCom 9 to the HMM cluster 11.

[00314] For example, in FIG. 5 is a high-level diagram showing an input and output interface of the HMM module 8, such as the input and output of a cluster module. As shown in FIG. 6, each HMM cluster 11 may be configured to communicate, such as receiving data from and/or sending resulting data, such as summary data, with the CentCom data distributor 9 via the cluster bus 10. In particular, any suitable interface may be provided 5 if it allows PCIe interface 4 to communicate with data distributor 9. More specifically, bus 5 may be an interconnect containing interpretation logic useful for telling data distributor 9 what interpretation logic may be configured to adapt to any protocol used to provide that functionality. Namely, in various cases, the interconnect can be implemented as a 5 PCIe bus.

[00315] In addition, cluster 11 may be configured to utilize one or more clock regions and, therefore, one or more clock generators may be present within cluster 11. In specific cases, multiple clock frequency regions may be provided. For example, a slower clock may be provided, for example, for data transfers, say, to and from cluster 11. In addition, a faster, eg, high-speed, clock generator may be provided that can be used by the HMM instances 13 when performing the various state calculations described herein.

[00316] Specifically, in various embodiments, as shown in FIG. 6, the system 1 may be installed such that, in the first case where the data distributor 9 uses an existing CentCom IP, a collar such as an O-ring may be provided, the O-ring being configured to transmit a signal to the CentCom interface 5 from the interface or bus 10 HMM clusters and vice versa. For example, the HMM cluster bus 10 may communicate communicatively and/or functionally between the CPU/GPU 1000 and various clusters 11 of the HMM accelerator module 8. Thus, as shown in FIG. 6, structured written and/or read data for each haplotype and/or each read can be sent throughout the system 1.

[00317] Following a task 20 entered into the HMM engine, the HMM engine 13 may typically begin work either: a) immediately if it is in an idle state (IDLE), or b) upon completion of the currently assigned task. It should be noted that each HMM accelerator engine 13 can process input signals in a staggered manner (e.g., can work on one set of data while loading another), thereby minimizing wasted time between jobs. Accordingly, the HMM cluster cuff 11 may be configured to automatically receive an input job 20 sent by the data distributor 9 and assign it to one or more HMM engine instances 13 in the cluster 11, which can accept the new job. There is no control required on the software side that can select a specific HMM engine instance 13 for a specific job 20. However, in various cases, the software may be configured to control such instances.

[00318] Accordingly, in view of the above, the system 1 can be rationalized in sending result data back to the CPU/GPU/CPU, and due to this efficiency, there is not much data that needs to be returned to the CPU/GPU/CPU to benefit from the results. This allows the system to achieve a variant determination operation time of about 30 minutes or less, for example about 25, or about 20 minutes or less, for example about 18 or about 15 minutes or less, including about 10 or about 7 minutes or less, even about 5 or about 3 minutes or less depending on system configuration.

[00319] In FIG. 6 is a high-level view of various functional blocks of an example HMM engine 13 within a hardware accelerator 8 on an FPGA or ASIC 7. Specifically, within a hardware HMM accelerator 8 there are a plurality of clusters 11, and within each cluster 11 there are a plurality of engines 13. In FIG. Figure 6 shows an example of a 13 HMM engine. As shown in FIG. 6, engine 13 may include an instance bus interface 12, a plurality of memories, such as HMEM 16 and RMEM 18, various other components 17, HMM control logic 15, and a result output interface 19. In particular, on the engine side, the HMM instance bus 12 is configured to be operatively connected to the memories, HMEM 16 and RMEM 18, and may contain interface logic that communicates with the cluster hub 11, wherein the cluster hub communicates with the allocator 9, which, in in turn communicates with PCIe interface 4, which communicates with variant determination software executed by the CPU/GPU and/or server 1000. Thus, HMM instance bus 12 receives data from CPU 1000 and loads it into one or more memories, for example HMEM and RMEM. This configuration can also be implemented in one or more quantum circuits and adapted accordingly.

[00320] In these cases, sufficient memory must be allocated to allow at least two or more haplotypes, such as two haplotypes, to be loaded into HMEM 16 for a given read sequence that is loaded into, for example, RMEM 18, which When loading multiple haplotypes, it reduces the load on PCIe bus 5 bandwidth. In specific cases, two haplotypes or two read sequences can be loaded into their respective memories, allowing the four sequences to be processed together in all relevant combinations. In other cases, for example, four, or eight, or sixteen, pairs of sequences may be downloaded and similarly processed in combination, for example, to further lighten the bandwidth load if required.

[00321] In addition, enough memory may be reserved to allow a toggling structure to be implemented so that once the memories are loaded with a new job 20a, for example on one side of the memory, a new job is signaled and the logic control 15 could begin processing a new job 20a, for example, by generating a matrix and performing the necessary calculations, as described below herein. Accordingly, this leaves the other side of the memory available so that another job 20b can be loaded into it, which can be loaded there while the first job 20a is being processed, and upon completion of job 20a, processing of job 20b can immediately begin using the control logic 15 .

[00322] In such a case, the matrix for job 20b can be processed with virtually no overhead, such as one or two clock cycles, between the completion of processing of the first job 20a and the start of processing of the second job 20b. Therefore, when using both sides of the toggled memory structure, HMEM 16 can typically store 4 haplotype sequences, eg two fragments, and RMEM 18 can typically store 2 read sequences. This alternating switching configuration is useful because it requires only a little extra memory, but allows double the throughput of the 13 engine.

[00323] During and/or after processing, the memories 16, 18 provide data to a transition probability calculator and lookup table (LUT) unit 17a, which is configured to calculate various information related to the "Prior" values, as explained below, and which, in turn, supplies the Prior results data to the M, I and D state calculator block 17b for use in calculating the transition probabilities. In addition, one or more flash RAMs 17c may be included, for example, to store the states M, I and D at the edge of the capture strip, for example, the values of the bottom line of the processing capture strip, which, as noted, in various cases, can be any suitable number of cells, for example about 10 cells, in length, to be commensurate with the length of the 35 strip.

[00324] Additionally, a separate result output interface block 19 may be included so that upon completion of the summations they, for example 4 32-bit words, can be immediately transferred back to the CPU/GPU/CPU variant determination software 1000. It should be noted that that this configuration can be configured so that system 1, namely the M, I and D calculator 17b, does not wait for the output interface 19 to clear, for example, until it clears the results after it has completed the job 20. Therefore, in a given configuration, there may be three pipeline stages operating in unison to create a system-wide pipeline such as loading memory, performing MID calculations, and outputting results. Additionally, it should be noted that any given HMM engine 13 is one of many with its own output interfaces 19, however they may share a common interface 10 to return to the data distributor 9. Therefore, the cluster hub 11 will include management capabilities to control the transfer (“xfer”) of information through the HMM accelerator 8 to avoid collisions.

[00325] Accordingly, the following describes in detail the processes performed in each module of the HMM engines 13 as it receives haplotype and read data, processes it, and outputs the resulting data relating thereto, as largely described above. Namely, high-bandwidth computations in the HMM engine 13 within the HMM cluster 11 are aimed at calculating and/or updating the values of the insertion (M), insertion (I) and deletion (D) states, which are used in determining whether a particular the read being studied with the reference haplotype, and also to what extent it matches, as described above. In particular, the read, along with the Phred score and the gap opening penalty (GOP) value for each base in the read, is transmitted to the cluster 11 from the allocator 9 and thereby assigned to a specific processing engine 13 for processing. This data is then used by the M, I and D calculator 17 of the processing engine 13 to determine whether a particular base in the read is more or less likely to be correct and/or match its corresponding base in the haplotype, or be the result of a variation, e.g. insertion or deletion; and/or if there is variation, whether the variation appears to be caused by actual variation in the haplotype, or rather by an artifact due to error in the sequence generation and/or mapping and/or alignment systems.

[00326] As stated above, part of such analysis includes a MID calculator 17 that determines the probabilities of a base-to-base transition in a read moving from one M, I, or D state to another, compared to a reference, such as a match state. to another match state, or from a match state to either insertion or deletion states. In making such determinations, each of the associated transition probabilities is determined and taken into account when assessing whether the observed transition between read and reference is a true variation and not simply some kind of machine or processing error. For these purposes, it is useful to use the Phred score for each base considered when determining the probabilities of transitions into and out of a match state, for example, from a match state to an insertion or deletion state, such as a gap, in a comparison. In a similar way, the probabilities of transition of gap extension or exit from a state with a gap, for example, an insertion or deletion state, back to a match state are also determined. In particular cases, the probabilities of entering and exiting a deletion or insertion state, such as exiting a gap extension state, may be fixed values, and may be referred to herein as the gap extension probability or the gap extension penalty. However, in various cases, such gap extension penalties may be floating and therefore subject to change depending on the accuracy requirements of the system configuration.

[00327] Accordingly, as shown in FIG. 7 and 8, for each possible pair of read and haplotype, the value of each of the states M, I and D is calculated. In such a case, a virtual matrix 30 can be formed from cells containing the read being evaluated on one axis of the matrix and the associated haplotype sequence on the other axis of the matrix, such that such that each cell in the matrix represents the position of a base in the read and reference haplotype. Therefore, if the read and haplotype sequences are each 100 bases long, the matrix 30 will contain 100 by 100 cells, this portion of which will need to be processed to determine the likelihood and/or degree to which each particular read matches that particular reference. Therefore, once the virtual matrix 30 is generated, it can be used to determine the various state transitions that occur when moving from one base in the read sequence to another, and compare with the same haplotype sequence transitions, for example, as depicted in FIG. 7 and 8. Namely, the processing engine 13 is configured to process multiple cells in parallel and/or sequentially while passing through the matrix using logic 15. For example, as shown in FIG. 7, the virtual processing capture strip 35 extends and moves across and to the bottom of the matrix 30, for example, from left to right, processing individual cells of the matrix 30 diagonally downward from right to left.

[00328] More precisely, as shown in FIG. 7, each individual virtual cell within the matrix 30 contains a state value M, I and D that must be calculated to evaluate the nature of the identity of the found base and, as depicted in FIG. 7, the dependencies between these data for each cell in this process can be clearly seen. Therefore, to determine a given state M of the current cell being processed, it is necessary to move the match, insertion and deletion states of the cell located diagonally above it into the current cell and use it in calculating the state M of the cell being calculated at the moment (for example, thus promoting diagonally downward along the matrix indicates a match),

[00329] However, to determine the I state into the current cell being processed, only the match and insertion states from the cell immediately above the current cell being processed need to be moved (thus, when moving to the insertion state, we get a vertical downward “gapped” movement). Similarly, to determine the D state, only the match and deletion states from the cell immediately to the left of the current cell need to be moved into the current cell being processed (thus, when moving to the deletion state, we get a horizontal advancement across the “gap”). As shown in FIG. 7, after the computation of cell 1 (the shaded cell in the topmost row) begins, processing of cell 2 (the shaded cell in the second row) can also begin processing without expecting any results from cell 1, because between that cell in line 2 and the cell in line 1 where processing begins, there is no data dependency. The result is a reverse diagonal of 35, where the progression continues down and to the left, as shown by the red arrow. This reverse diagonal 35 forward approach improves processing efficiency and throughput of the entire system. Likewise, data generated in cell 1 can be immediately advanced to the cell below and directly to the right of the topmost cell 1, thereby advancing capture strip 35.

[00330] For example, in FIG. 7 illustrates an example of an HMM matrix structure 35 showing a hardware processing flow. Matrix 30 includes a haplotype base index, for example, containing 36 bases arranged from left to right along the top edge of the horizontal axis, and also includes a read base index, for example, containing 10 bases, arranged from top to bottom along the side edge of the vertical axis such that that to form a structure of cells, where the selected cell can be filled with probability states M, I and D and the probabilities of transition from the current state to the neighboring state. In such a case, as described in more detail above, moving from a match state to a match state results in moving forward diagonally across the matrix 30, while moving from a match state to an insertion state results in moving vertically downward to form a gap, and moving from a match state into the deletion state leads to horizontal movement with the formation of a gap. Therefore, as shown in FIG. 8, for a given cell, the match, insertion, and deletion probabilities of its three adjacent cells are used to determine the match, insertion, and deletion states for each cell.

[00331] The downward arrow in FIG. 7 represents the parallel and serial nature of the processing engines, which are configured to create a swath 35 or wave that moves translationally along the virtual matrix in accordance with data dependencies (see FIGS. 7 and 8) to determine the states M, I and D for each specific cells in structure 30. Accordingly, in certain cases it may be necessary to calculate the identities of each cell in the downward and diagonal direction, as explained above, rather than simply calculating each cell solely along the vertical or horizontal axis, although this can be done if required. This is due to the increased latency, eg latency, that would be required when processing the virtual cells of the matrix 30 individually and sequentially along only the vertical or horizontal axis, eg by hardware configuration.

[00332] For example, in such a case, when moving linearly and sequentially through the virtual matrix 30, for example, row by row or column by column, in order to process each new cell, the calculations of each previous cell must be completed, which increases the overall latency. However, by propagating the probabilities M, I, D of each new cell downward and diagonally, system 1 does not have to wait for its predecessor cell, such as row one, to complete processing before processing the adjacent cell in row two of the matrix. This allows parallel and sequential processing of diagonal cells, and also allows the various pipeline computation delays associated with calculations of the M, I, and D states to be hidden. Accordingly, as the capture strip 35 moves across the matrix 30 from left to right, the computational processing moves diagonally downward, e.g. , to the left (as shown by the arrow in FIG. 7). This configuration can be particularly useful for hardware and/or quantum-based implementations, for example, when memory latency and/or clock-to-clock latency are of paramount importance.

[00333] In such configurations, the truly valuable result of each call to the HMM engine 13, for example, upon completion of the calculation of the entire matrix 30, may be the bottom row (for example, row 35 in FIG. 21) containing the states M, and D, where the states M and I can be summed (the D states can be ignored at this stage since they have already served their function in processing the calculations above) to obtain a final summed value, which can be a single probability estimating for each read index and haplotype the probability of observing the read, e.g. , assuming that the haplotype was actually taken from the original DNA.

[00334] In particular, the result of processing the matrix 30, for example, shown in FIG. 7, there may be a single value representing the probability that a read is an actual representation of a given haplotype. This probability is a value between 0 and 1 and is obtained by summing all M and I states from the bottom row of cells in the 30 HMM matrix. Essentially, what is being assessed is the possibility that some error occurred in the sequencer or in the associated DNA preparation methods prior to sequencing, resulting in the incorrect occurrence of a mismatch, insertion, or deletion in the read that is not actually present in the subject's genetic sequence. In such a case, the read is not a true reflection of the subject's actual DNA.

[00335] Therefore, by accounting for such production errors, it is possible to determine what any given read actually represents relative to the haplotype, allowing the system to better determine how a subject's genetic sequence, for example, as a whole, may differ from the reference sequence. For example, one can compare multiple haplotypes to multiple read sequences, generating scores for all of them and determining, based on which hits have the best scores, what the actual identity of an individual's genomic sequence is, and/or how it actually differs from the reference genome.

[00336] More specifically, in FIG. 8 is an enlarged view of the HMM state matrix 30 shown in FIG. 7. As shown in FIG. 8, given the internal composition of each cell in matrix 30, as well as the structure of the matrix as a whole, the probability of states M, I and D for any given “new” calculated cell depends on the states M, I and D of several surrounding neighbors that have already been calculated . In particular, as shown in more detail in FIG. 1 and 16, in the configuration example, the probability of transitioning from a match state to another match state may be approximately 0.9998, and the probability of transitioning from a match state to an insertion or deletion state, such as a gap, (penalty for opening a gap) is only 0. 0001. In addition, when in a gap state due to an insertion or in a gap state due to a deletion, the probability of remaining in the gap state (extension or continuation penalty) may be only 0.1, while the probability of returning to the match state is 0.9. It should be noted that, in accordance with this model, all probabilities of entering or exiting a given state must add up to one. In particular, the processing of matrix 30 revolves around calculating the transition probabilities, taking into account the various penalties for opening a gap or continuing a gap, finally calculating the sum.

[00337] Therefore, these calculated state transition probabilities are derived primarily from their immediately adjacent cells in the matrix 30, for example, from the cells that are immediately to the left, above, and diagonally to the left of a given calculated cell, as shown in FIG. 16. Additionally, state transition probabilities can be derived in part from the “Phred” quality score that is attached to each read base. Therefore, these transition probabilities are useful in calculating the values of the M, I, and D states for a particular cell and, similarly, for any associated new cell being calculated. It should be noted that, as described herein, the gap opening and gap continuation penalties may be fixed values, however, in various cases, the gap opening and gap continuation penalties may be variable and therefore can be programmed into the system, albeit with using additional hardware resources specifically designed to determine such variable transition probability calculations. Such cases can be useful if increased accuracy is required. However, when such values are assumed constant, it is possible to reduce resource usage and/or chip size, resulting in increased processing speed, as explained below.

[00338] Accordingly, when each new state value is obtained, M, I, and D use a variety of calculations and/or other mathematical operations, such as multiplications and/or additions. In such a case, to maximize computational throughput, the primitive mathematical operations used in each computation of the transition state M, I, and D can be organized as a pipeline. Such pipeline organization can be designed in such a way that the corresponding clock frequencies are high, but the depth of the pipeline can be non-trivial. Additionally, such a pipeline can be designed to have a finite depth, and in such cases, operations may require more than one clock cycle to complete.

[00339] For example, these calculations may be performed at high speeds within processor 7, such as about 300 MHz. This can be achieved by carefully pipelining the FPGA or ASIC using registers so that little math is done between each flip-flop switch. This pipeline structure results in multiple latency cycles from match state input to output, but given the reverse diagonal computation structure shown in FIG. 7 above, these delays may be hidden throughout the HMM matrix 30, for example, when each cell represents one clock cycle.

[00340] Therefore, the number of calculations of the M, I and D states may be limited. In such a case, the processing engine 13 may be configured such that a grouping, e.g., capture strip 35, of cells in a plurality of rows of the matrix 30 may be processed as a group (e.g., diagonally down to the left, as shown in FIG. 7) before proceed to processing the second stripe of captures below, for example, when the second stripe of capture contains the same number of cells in rows to be processed as the first. Likewise, the hardware implementation of accelerator 8 described herein may be configured to improve overall system efficiency as described above.

[00341] Specifically, in FIG. 9 shows an example of a computational structure for performing various state computations described herein. More specifically, in FIG. 9 shows three dedicated logic blocks 17 of the processing engine 13 for performing state calculations used in generating each state value M, I and D for each specific cell or grouping of cells processed in the HMM matrix 30. These logic blocks may be implemented in hardware, but in some cases may be implemented in software, such as those executed by one or more quantum circuits. As shown in FIG. 9, the match state calculation 15a involves more operations than the insertion or deletion calculations 15b 15b because when the match state of the currently processed cell is calculated 15a, the current match calculation includes all previous match, insertion and deletion states of the adjacent cells along with various "Prior" data. (see FIGS. 9 and 10), while calculations of insertion and deletion states include only match and either insertion or deletion states, respectively. Therefore, as shown in FIG. 9, when calculating the match state, three state multipliers are used, as well as two adders and, finally, a final multiplier that takes into account the Prior value, for example, Phred. However, when calculating the I or D state, only two multipliers and one adder are used. It should be noted that in hardware, multipliers are more resource intensive than adders.

[00342] Accordingly, to varying degrees, the values of the M, I, and D states to process each new cell in the HMM matrix 30 use knowledge or pre-computation of the following values, such as the “previous” values of the M, I, and D states to the left, top, and/or diagonally to the left and above the current calculated cell in the HMM matrix. In addition, such values representing previous information or "Prior" values may be based at least in part on the "Phred" quality score and depend on whether the read base and the reference base are the same in a given cell in matrix 30, or they are different. This information is especially useful in determining the match status. Namely, as shown in FIG. 9, in such cases, there are basically seven "transition probabilities" (from M to M, from I to M, from D to M, from I to I, from M to I, from D to D and from M to D), which indicate and/or estimate the probability of observing a gap opening, for example, observing a transition from a match state to an insertion or deletion state; observing the closure of a gap, for example, the transition from an insertion or deletion state back to a match state; and observing the next state remaining in the same state, for example, from match to match, from insertion to insertion, from deletion to deletion.

[00343] State values (e.g., in any cell to be processed in the HMM matrix 30), Prior values, and transition probabilities are values in the range

[0,1]. In addition, the initial conditions for the cells that are on the left and right along the edges of the 30 HMM matrix are also known. As can be seen from logic 15a in FIG. 9, there are four multiplication operations and two addition operations that can be used in computing the particular state M determined for any given cell being processed. Similarly, as seen in Logics 15b and 15c, the computation of each state I and each state D, respectively, involves two multiplications and one addition. Together with the pre-data multiplier, this results in a total of eight multiplications and four additions for computing the M, I and D states of each single cell of the HMM matrix 30 to be processed.

[00344] The result of the final summation, for example, line 34 in FIG. 16, the calculation of matrix 30, for example, for one comparison job 20 of one read with one or two haplotypes, is the sum of the final states M and I across the entire bottom row 34 of matrix 30, which is the total final value output from HMM accelerator 8 and supplied in CPU/GPU/GPU 1000. This final total represents how well the read matches the haplotype(s). This value is a probability, for example, less than one, for a particular job 20a, which can then be compared with the output of another job 20b, for example, belonging to the same active region 500. It should be noted that there are on the order of 20 trillion HMM cells that needs to be estimated in a "typical" human genome with 30-fold coverage, with these 20 trillion HMM cells distributed across 1 to 2 billion 30 HMM matrices of the entire corresponding 20 HMM job.

[00345] The results of such calculations can then be compared with each other to more accurately determine, for example by comparing bases one by one, how the subject's genetic sequence differs from the sequence of one or more reference genomes. For the final sum calculation, it is possible to reuse the adder that has already been used to calculate the M, I and/or D states of the individual cells to calculate the final sum value, for example by including a multiplexer in the selection of reused adders, thereby including one final additional row, for example, to the computation time, into the matrix so that it calculates this final sum, which in the case of a read of 100 bases in length leads to a resource overhead of about 1%. In alternative implementations, dedicated hardware resources may be used to perform such calculations. In various cases, the adder logic for calculating states M and D can be used to calculate the final sum, and the adder of state D can be effectively applied so that it is not otherwise used in the final processing leading to the total values.

[00346] In certain cases, these calculations and associated processes may be capable of matching the output of a given sequencing platform, for example, containing multiple sequencers, which collectively may be able to output (on average) a new human genome with 30-fold coverage every 28 minutes (although they emerge from multiple sequencers in groups of about 150 genomes every three days), In such cases, where ongoing mapping, alignment, and variant identification operations are made to fit into such a sequencing processing technology platform, part of the 28 minutes (e.g. approximately 10 minutes) that it takes to sequence a genome by a sequencing cluster can be used by a suitably configured mapper and/or aligner as described herein to obtain the /BCL/FASTQ image file results from the sequencer and perform the mapping and/or alignment steps genome, for example, post-processing after the sequencer. This leaves about 18 minutes of sequencing time to perform the variant detection step, the main computational part of which is the HMM operation, for example, before the nucleotide sequencer sequences the next genome, for example, within the next 28 minutes. Accordingly, in such cases, 18 minutes can be allocated to calculate the 20 trillion HMM cells that need to be processed according to the genome processing, for example, when the HMM cell processing involves about twelve operations (for example, eight multiplication operations and/or four addition operations ). This throughput enables the following computational dynamics (20 trillion HMM cells) × (12 math operations per cell)/(18 minutes × 60 seconds/minute), which gives a throughput of about 222 billion operations per second when running continuously.

[00347] In FIG. 10 shows the logical block 17 of the processing engine shown in FIG. 9, including M, I and D state update circuits, which are a simplification of the circuit shown in FIG. 9. The system may be configured to have unlimited memory such that a single instance of the HMM engine 13 (for example, which computes all single cells in the HMM matrix 30 at a rate of one cell per clock cycle, on average, plus resource overhead) can be duplicated multiple times. (at least 65~70 times to achieve effective throughput as described above). However, to minimize the size of the hardware, such as the size of chip 2 and/or the use of associated resources, and/or to further attempt to include as many instances of the HMM engine 13 on chip 2 as required and/or possible, simplifications can be made to logic blocks 15a'-c' of processing instance 13 to calculate one or more transition probabilities to be calculated.

[00348] In particular, it can be assumed that the gap opening penalty (GOP) and the gap continuation penalty (GCP), as described above, for example, for insertions and deletions, are the same and the chip configurations are known in advance. This simplification means that the probabilities of transition from I to M and from D to M are identical. In such a case, one or more multipliers, such as those shown in FIG. 9 can be eliminated, for example, by first adding states I and D before multiplying by the total probability of transition from indel to M. For example, in various embodiments, if the calculations of states I and D are assumed to be the same, then the state calculations for the cell can be simplified, as shown in FIG. 10. In particular, if the values of states I and D are the same, then state I and state D can be added and then the sum multiplied by one value, thereby saving the multiplication. This can be done because, as shown in FIG. 10, the penalty for continuing and/or closing the gap for states I and D are the same. However, as stated above, the system may be configured to calculate different transition probabilities for both states I and D, in which case this simplification will not be used.

[00349] Additionally, in the following description, rather than having dedicated circuitry or other computing resources configured in a particular manner to perform a final sum operation at the bottom of the HMM matrix, the present HMM accelerator 8 may be configured to efficiently append one or more additional rows to the HMM matrix 30 in terms of computational time, such as resource overhead, it spends to perform the calculation, and may also be configured to “borrow” one or more adders from the M-state calculation logic 15a and D-state 15c, for example, by multiplexing the final summation values into existing adders as needed to perform the actual final summation calculation. In such a case, the final logic, including blocks of M-state logic 15a, I-state logic 15b, and D-state logic 15c, which together form part of the MID HMM instance 17, may contain 7 multipliers and 4 adders along with the various multiplexing involved.

[00350] Accordingly, in FIG. 10 shows circuits 15a', 15b' and 15c' for updating the states of M, I and D, taking into account simplifying assumptions regarding the transition probabilities, and also taking into account the sharing of various resources M, I and/or D, for example, adder resources, for the final summation operations. In addition, a delay block may be added to the M-state calculating block in the M-state path, as shown in FIG. 10. This delay may be added to compensate for delays in the actual hardware implementation of the multiply and add operations and/or to simplify the control logic, e.g. 15.

[00351] As shown in FIG. 9 and 10, these respective multipliers and/or adders may be floating point multipliers and adders. However, in various cases, as shown in FIG. 11, a logarithmic domain configuration can be implemented, where in such a configuration all the multipliers are converted into adders. In FIG. Figure 12 shows what a log-domain computation would look like if all the multipliers were turned into adders, such as 15a", 15b", and 15c", as happens when using a log-domain computing configuration. In particular, all the multiplier logic turns into an adder, but the adder itself becomes or otherwise includes a function such as: f(a,b) = max(a,b) - log ₂ (1+2^(-[ab])), for example when the logarithmic side of the equation can be maintained in a LUT whose depth and physical size are determined by the required accuracy.

[00352] Given the typical read and haplotype sequence lengths and values typically observed for quality scores (Phred) and associated transition probabilities, the dynamic range requirements for HMM state values can be quite severe. For example, when implementing an HMM in software, various HMM jobs 20 may result in incomplete execution, such as when implemented on single precision (32-bit) floating point state values. This means a dynamic range that is greater than 80 powers of 10, and therefore requires upgrading the case detection software to handle double precision (64-bit) floating point state values. However, full 64-bit double-precision floating point representation has some negative consequences in many cases, for example, if high-speed hardware is to be implemented, the memory and compute pipeline resource requirements will need to be increased, thereby taking up more chip space and/or slowing down coordination over time. In such cases, only a fixed point representation of numbers in the linear domain can be implemented. However, the dynamic range requirements on state values in this implementation make bitwidths less desirable in certain circumstances. Accordingly, in such cases, a logarithmic-domain fixed-point-only representation of numbers may be implemented, as described herein.

[00353] In such a circuit as shown in FIG. 11, instead of representing in memory and computing the actual value of the state, -log in base 2 can be represented. This can provide several advantages, including the use of multiplication operations in linear space, which translates to addition operations in logarithmic space; and/or a given logarithmic domain representation of numbers inherently supports a wider dynamic range with only a small increase in the number of integer bits. These calculations of state updates M, I, D in the logical domain are shown in FIG. 11 and 12.

[00354] As can be seen by comparing the configuration of logic 17 in FIG. 11 and FIG. 9, multiplication operations disappear in the logarithmic region. Rather, they are replaced by addition operations, and the addition operations are converted into a function that can be expressed as a maximum operation followed by the addition of a correction factor, for example by means of a LUT, where the correction factor is a function of the difference between two values summed in the logarithmic domain. This correction factor can either be calculated or generated from a lookup table. What is more effective to use, calculating the correction factor or implementing it using a lookup table, depends on the required precision (bit width) of the difference between the sum values. Therefore, in specific cases, the number of bits to represent state in the logarithmic domain may be approximately 8 to 12 integer bits plus 6 to 24 bits for fractional bits, depending on the required level of quality for any given implementation. This means anywhere from 14 to 36 bits total to represent the state value in the logarithmic domain. In addition, it was determined that there are logarithmic domain fixed-point representations that can provide acceptable quality and acceptable hardware size and speed.

[00355] In various cases, each HMM job 20 typically processes one sequence read, which, as stated, may involve comparison to two haplotype sequences. Similar to the above with respect to the haplotype memory, the read sequence memory 18 may also employ a toggled structure to allow various software functions to write the information of the new HMM job 20b while the current job 20b is still being executed by the HMM engine instance 13. Therefore, the memory for storing the read sequence may require one 1024x32 memory with two ports (eg, one write port, one read port, and/or separate clock generators for the write and read ports).

[00356] In particular, as described above, in various cases, the architecture used by the system 1 is configured to generate a virtual matrix 30 when determining whether a given base in the sequenced genome of a sample matches a corresponding base in one or more reference genomes, where the reference genome is theoretically arranged along the horizontal axis, while the sequenced sequences representing the genome of the sample are theoretically arranged in descending order along the vertical axis. Therefore, when performing an HMM calculation, the HMM processing engine 13, which is described herein, is configured to traverse this virtual HMM matrix 30. Such processing may be depicted as in FIG. 7 as the capture strip 35 moves across the visa diagonal and across the virtual array, performing various HMM calculations for each cell of the virtual array, as shown in FIG. 8.

[00357] More specifically, this theoretical pass involves processing the first grouping of rows of cell 35a from matrix 30 in its entirety, for example, for all haplotype bases and reads within the grouping, before moving down to the next grouping of rows 35b (for example, the next read base group). In such a case, the M, I and D state values for the first grouping are stored at the bottom edge of this initial row grouping so that these M, I and D state values can then be used to feed the top row of the next grouping (swap) down in the matrix 30. In various cases, the system 1 may be configured to feed the HMM accelerator 8 haplotypes and/or reads up to 1008 in length, and since the numeric representation uses W-bits for each state, this results in a memory of 1008 words x W-bits for storage states M, I and D.

[00358] Accordingly, as stated, such a memory may be a single-port memory or a dual-port memory. In addition, cluster-level memory may also be provided, for example to store swath boundary results. For example, in accordance with the above description, the memories discussed are already configured for each engine instance 13. In specific HMM implementations, multiple instances of 13a- _(n+1) engines can be grouped into a cluster 11, which is served by a single connection, for example a 5 PCIe bus, with a 4 PCIe and 3 DMA interface via 9 CentCom. To make more efficient use of PCIe bandwidth using existing CentCom 9 functionality, multiple 11a- _(n+1) cluster instances can be created.

[00359] Therefore, in a typical configuration, anywhere from 16 to 64 engine instances 13 _m are created within a cluster 11 _n , and in a typical FPGA/ASIC HMM module 8 implementation, anywhere from one to four clusters can be created (e.g., depending whether it is a dedicated HMM FPGA for image processing, or whether the HMM must share the FPGA's usable area with the sequencer/matcher/aligner and/or other modules as described herein). In specific cases at the cluster level 11, the HMM hardware may have a small amount of memory used. This memory can be used as a first-in, first-out (“FIFO”) elastic memory to collect output data from the HMM engine instance 13 in the cluster and transfer it to the CentCom 9 for further transmission back to the software on the CPU 1000 via an interface 3 DMA and 4 PCIe. In theory, this FIFO could be very small (on the order of two 32-bit words), since, as a rule, after the FIFO arrives, the data is transferred almost immediately to 9 CentCom. However, to absorb potential discontinuities in the output data path, the size of this FIFO memory can be parameterized. In specific cases, FIFO can be used with a depth of 512 words. So the cluster level memory requirement could be one 512x32 memory with two ports (separate read and write ports, same clock region).

[00360] In FIG. Figure 12 shows various HMM state transitions 17b depicting the relationship between gap opening penalties (GOP), gap extension penalties (GCP), and transition probabilities involved in determining whether and how much a given read sequence matches a particular haplotype sequence. When performing such analysis, the HMM engine 13 includes at least three logic blocks 17b, such as a logic block 15a for determining a match state, a logic block 15b for determining an insertion state, and a logic block 15c for determining a deletion state. These logics 17 for calculating M, I and D states, when properly configured, function efficiently to avoid high bandwidth bottlenecks such as the HMM computation flow. However, once the basic M,I,D,computation architecture is defined, other system improvements,can also be configured and implemented to avoid other,system bottlenecks.

[00361] In particular, the system 1 may be configured to maximize the efficiency of information flow from the computer core 1000 to the option determiner module 2 and vice versa, so as not to create bottlenecks that could limit the overall throughput. One of these blocks, which supplies the logic 17 for calculating the states M, I, D of the HMM core, is the block for calculating transition probabilities and preliminary data. For example, as shown in FIG. 9, each clock cycle uses a representation of seven transition probabilities and one Prior value as input to the M, I, D state calculation block 15a. However, after simplifications that lead to the architecture shown in FIG. 10, for each cycle at the input of the block for calculating the states M, I, D, only four unique transition probabilities and one Prior value are used. Accordingly, in various cases these calculations can be simplified and the resulting values can be generated. This results in increased throughput, increased efficiency and reduced potential for bottlenecks at this stage of the process.

[00362] Additionally, as described above, Prior values are values generated using the quality of a read, eg, a Phred score, a particular basis for the current cell being evaluated in the virtual HMM matrix 30. This relationship can be described using the following equations: First, the Phred read of interest can be expressed as probability = 10^(-(Phred read/10)). Prior can then be calculated depending on whether the base of the read matches the base of the hypothetical haplotype: If the base of the read and the base of the hypothetical haplotype match: Prior = 1 - Phred of the read, expressed as a probability. Otherwise: Prior = (Phred read expressed as probability)/3. The division by three operation in this last equation reflects the fact that there are only four possible bases (A, C, G, T). Therefore, if the read base and the haplotype base do not match, then one of the three remaining bases must match, and each of the three probabilities is modeled as equally plausible.

[00363] The Phred scores for each base are provided to the HMM hardware accelerator 8 as 6-bit values. Therefore, the equations for Prior values have 64 possible results for the “match” case and 64 possible results for the “mismatch” case. This can be effectively implemented in hardware as a 128-word lookup table, where the lookup table address is a 7-bit value formed by concatenating the Phred value with a single bit that indicates whether the base of the read matches the base of the hypothetical haplotype or not .

[00364] Additionally, with respect to determining the probabilities of a match-to-insertion and/or match-to-deletion transition, in various architecture implementations for the HMM hardware accelerator 8, separate gap opening penalties (GOP) can be defined for the match-to-insertion state transition and transition from match to deletion as above. This is equivalent to the fact that the values of M2I and M2D in the state transition diagram shown in FIG. 12, vary. Since the GOP values are obtained from the 8 HMM hardware accelerator as 6-bit Phred values, the transition probabilities to the gap open state can be calculated according to the following equations: Transition probability M2I = 10^(-(GOP(I) read/10) ) and transition probability M2D = 10^(-(GOP(D) read/10)). Similar to retrieving Prior values in hardware, a simple 64-word standby table can be used to obtain M2I and M2D values. If GOP(I) and GOP(D) are fed into the hardware accelerator 8 HMM as potentially different values, then two such lookup tables can be used (or one resource-sharing lookup table, potentially running at twice the clock speed of the rest of the circuit) .

[00365] In addition, regarding the determination of match-to-match transition probabilities, in various cases, the match-to-match transition probability can be calculated as follows: M2M transition probability = 1 - (M2I transition probability + M2D transition probability). If the M2I and M2D transition probabilities can be configured to be less than or equal to the value of ½, then in various implementations the above equation can be implemented in hardware in a manner that improves overall efficiency and throughput, for example by reworking the equation: M2M transition probability = (0.5 - M2I transition probability) + (0.5 - M2D transition probability). This rewrite of the equation allows M2M to be obtained using two 64-element lookup tables followed by an adder where the lookup tables store the results.

[00366] Moreover, with regard to determining the insertion-to-insertion and/or deletion-to-deletion transition probabilities, the transition probabilities I2I and D2D are functions of the gap continuation probability (GCP) values input into the HMM hardware accelerator 8. In various cases, these GCP values may be 6-bit Phred values assigned to each base of the read. The values of I2I and D2D can be obtained as follows: Transition probability I2I = 10^(-(GCP(I) read/10)) and transition probability D2D = 10^(-(GCP(D) read/10)). Similar to some of the other transition probabilities discussed above, I2I and D2D values can be implemented efficiently in hardware and can include two lookup tables (or one lookup table with shared resources), for example, in the same form and with the same context as and the match-to-indel transition tables discussed above. That is, each staging table has 64 words.

[00367] In addition, with regard to determining the probabilities of an insertion and/or deletion transitioning into a match, the transition probabilities I2M and D2M are functions of the gap continuation probability (GCP) values, which can be calculated as follows: Transition probability I2M = 1 - transition probability I2I and the transition probability D2M = 1 is the transition probability of D2D, where the transition probabilities I2I and D2D can be obtained as discussed above. A simple subtraction operation to implement the above equations may be more expensive in hardware resources than simply implementing another 64-word lookup table and using two instances of it to implement I2M and D2M retrieval. In such cases, each staging table has 64 words. Of course, in all relevant embodiments, simple and complex operations can be configured using software suitably configured.

[00368] In FIG. 13 shows a circuit diagram 17a for simplified calculation of HMM transition probabilities and Prior values as described above, which supports the general state transition diagram depicted in FIG. 12. As shown in FIG. 13, in various cases, a simple software accelerator architecture 17a is presented, wherein the accelerator may be configured to include separate GOP values for insertion and deletion transitions and/or may have separate GCP values for insertion and deletion transitions. In such a case, the cost of generating seven unique branch probabilities and one Prior value per clock cycle can be configured as follows: eight 64-word lookup tables, one 128-word lookup table, and one adder.

[00369] In addition, the hardware 2 that is presented herein may be configured to accommodate as many HMM engine instances 13 as can be placed on a target chip (such as an FPGA, sASIC, or ASIC). In this case, the cost of implementing logic 17a for generating transition probabilities and Prior values can be significantly reduced relative to the costs provided by the configurations below. First, instead of supporting a more general version of state transitions such as those shown in FIG. 13, for example, where there may be separate values for GOP(I) and GOP(D), in different cases it can be assumed that the GOP values for insertion and deletion are the same for a given base. This results in several hardware simplifications as stated above.

[00370] In such cases, it is possible to use only one 64-word lookup table to generate an M2Indel value that replaces both the M2I and M2D probability values, whereas in the more general case, two tables would typically be used. Similarly, only one 64-word lookup table can be used to generate the M2M transition probability value, whereas in general two tables and an adder can be used, since M2M can now be calculated as 1-2×M2Indel.

[00371] Second, it can be assumed that the sequencer-dependent GCP value for both insertion and deletion is the same, AND that this value does not change during HMM job 20. This means that: instead of separate I2I and D2D values, one indel-to-indel transition probability can be calculated using a 64-word lookup table instead of two tables; and instead of having separate I2M and D2M values, a single indel-to-match probability can be calculated using one 64-word lookup table instead of two tables.

[00372] In addition, one further simplifying assumption can be made which assumes that the values from insertion to insertion and deletion to deletion (I2I and D2D) and from insertion to match and deletion to match (I2M and D2M) are not only identical between insertion and deletion transitions, but can be static for a specific job 20 HMM. Thus, in a more general architecture with transition probabilities I2I, D2D, I2M, and D2M, a total of four lookup tables can be deleted. In various such cases, it is possible to have the static indel-to-indel and indel-to-match probabilities be entered via software or via an RTL parameter (and so that they can be programmed in dual-threaded mode in the FPGA). In certain cases, these values can be made dual-stream programmable, and in certain cases a training mode can be implemented that uses the training sequence to further improve the accuracy of the transition probability for a given sequencer run or genomic analysis.

[00373] In FIG. 14 shows what a diagram of the new state transition logic 17b might look like when these various simplifying assumptions are implemented. Namely, in FIG. 14 shows a simplified HMM state transition diagram depicting the relationships between GOPs, GCPs, and transition probabilities with the simplifications outlined above.

[00374] Similarly, in FIG. 15 shows a circuit diagram 17a,b for generating transition probabilities and Prior HMM values that supports the simplified state transition diagram depicted in FIG. 14. FIG 15 shows the implementation of the circuit diagram of this transition diagram. Thus, in various cases for the 8 HMM hardware accelerator, the cost of generating transition probabilities and Prior for each clock cycle is reduced by: two lookup tables of 64 words, one lookup table of 128 words.

[00375] As stated above, the engine control logic 15 is configured to generate a virtual matrix and/or pass the matrix so as to reach the edge of the swath, for example, by means of high-level state machines, where the resulting data can be finally summarized, for example, by means of logic 19 control the final summation, and are stored, for example through put/get logic.

[00376] Accordingly, as shown in FIG. 16, various embodiments provide a method for creating and/or traversing a matrix of 30 HMM cells. Namely, in FIG. 16 shows an example of how the HMM accelerator control logic 15 traverses back and forth across the virtual cells in the HMM matrix. For example, let's say for example that each multiply and add operation has a latency of 5 clock cycles, then the worst case latency of the M, I, D state update calculations would be 20 clock cycles and would occur in the M update calculation. In the I and D state update calculations half as many operations, meaning the latency for these operations is 10 clock cycles.

[00377] The effects of delay on the M, I, and D calculation operations can be understood using FIG. 16, which shows various examples of data dependencies between cells. In such cases, information about the I, M and D state of a given cell is fed into calculations of the D state of the cell in the HMM matrix that is immediately to the right (i.e. has the same read base as the given cell, but has the next haplotype base) . Likewise, information about the I, M, and D state of a given cell is fed into the computation of the I state of the cell in the HMM matrix that is immediately below (i.e., has the same haplotype base as the given cell, but has the next read base). Therefore, in specific cases, the M, I, and D states of the current cell are fed into the computation of the D and I states of cells on the next diagonal of the HMM cell matrix.

[00378] Likewise, the M, I, and D states of a given cell are fed into the calculation of the M state of the cell that is to the right and below (i.e., has the next haplotype base AND the next read base). This cell is actually two diagonals away from the cell it supplies (due to the fact that the calculations of states I and D rely on states from a cell that is one diagonal away). This quality of having I and D state computations rely on cells one diagonally distant, while M state computations rely on cells two diagonals distant, has a beneficial effect on hardware design.

[00379] In particular, with such configurations, I and D state calculations can be performed at twice the speed (eg, 10 cycles) compared to M state calculations (eg, 20 cycles). Therefore, if the M state computations begin 10 cycles before the I and D state computations for the same cell, then the M, I, and D state computations for a cell in the 30 HMM matrix will all complete simultaneously. In addition, if the matrix 30 is traversed diagonally such that the capture strip 35 contains approximately 10 cells (for example, covering ten read bases), then: The M and D states created by a given cell at coordinates (i, j), where i refers to the haplotype, and j refers to the read, can be used in the calculations of the state D of cell (i+1, j) as soon as they go through the entire computational pipeline of cell (i, j).

[00380] The M and I states produced by a given cell at coordinates (i, j), where i refers to the haplotype and j refers to the read, can be used in calculating the I state of the cell (i, j+1) after one clock cycle after they have gone through the entire computational pipeline of cell (i, j). Similarly, the states M, I and D produced by a given cell at coordinates (i, j), where i refers to the haplotype and j refers to the read, can be used in calculating the state M of the cell (i+1, j+1) via one clock cycle after they have gone through the entire computational pipeline of cell (i, j). Taken together, the above means that states M, I, and D along a capture stripe diagonal that extends the length of the capture stripe, say ten bases, require very little dedicated memory. In such a case, only registers are required to delay by one clock cycle the values M, I and D of cell (i, j) for use in the calculations of M in cell (i+1, j+1) M and in the calculations of I in cell ( i, j+1) in one clock cycle. Moreover, all this happens in some magical way, since the calculation of the M state for a given cell begins 10 clock cycles before the calculation of the I and D states for the same cell, which naturally leads to the simultaneous output of new states M, I and D for any of this cell.

[00381] In view of the above, and as shown in FIG. 16, the HMM accelerator control logic 15 may be configured to process data in each cell of the virtual traversal matrix 30. Specifically, in various implementations, operations begin at cell (0,0), with computations of state M beginning 10 clock cycles before computations of states I and D begin. The next cell should be cell (1,0). However, the results of the calculations of states I and D from cell (0,0) will be available with a delay of ten cycles after they begin. Therefore, the hardware inserts nine dead cycles into the computation pipeline. This is shown as cells with a haplotype index below zero in FIG. 16.

[00382] Upon completion of a dead cycle that has an effective cell position in the matrix (-9,-9), the M, I, and D state values for cell (0,0) are available. After this, they (for example, the outputs of the M and D states of cell (0,0)) can be immediately used to calculate the D state of cell (0,1). After one clock cycle, the M, I, and D state values from cell (0,0) can be used to begin calculations of the I state of cell (0,1) and calculations of the M state of cell (1,1).

[00383] The next cell to go through may be cell (2,0). However, the results of the calculations of states I and D from cell (1,0) will be available with a delay of ten cycles after they begin. Therefore, the hardware inserts eight dead cycles into the computation pipeline. This is shown as cells with a haplotype index below zero, as in FIG. 16 along the same diagonal where cells (1,0) and (0,1) are located. Upon completion of a dead cycle, which has an effective cell position in the matrix (-8,-9), the M, I and D state values for cell (1,0) are available. After this, they (for example, the outputs of the M and D states of cell (1,0)) are immediately used to calculate the D state of cell (2,0).

[00384] After one clock cycle, the M, I, and D state values from cell (1,0) can be used to begin cell (1,1) I state calculations and cell (2,1) M state calculations. In addition, the values of states M and D from cell (0,1) can be simultaneously used to calculate the state D of cell (1,1). One clock cycle later, the M, I, and D state values from cell (0,1) are used to begin the I state calculations of cell (0,2) and the M state calculations of cell (1,2).

[00385] Now the next cell to go through could be cell (3,0). However, the results of the calculations of states I and D from cell (2,0) will be available with a delay of ten cycles after they begin. Therefore, the hardware inserts seven dead cycles into the computation pipeline. This is again shown as cells with a haplotype index below zero in FIG. 16 along the same diagonal where cells (2,0), (1,1) and (0,2) are located. Upon completion of a dead cycle, which has an effective cell position in the matrix (-7,-9), the values of the M, I and D states for cell (2,0) are available. After this, they (for example, the outputs of the M and D states of cell (2,0)) are immediately used to calculate the D state of cell (3,0). And thus begins the calculation of the other ten cells on the diagonal.

[00386] Such processing may continue until the end of the last full diagonal in capture lane 35a, which, in this example (with a read length of 35 and a haplotype length of 14), will occur after the diagonal that begins at the cell with haplotype and read coordinates (13.0 ). After passing through cell (4,9) in FIG. 16, the next cell to go through should be cell (13,1). However, the results of the calculations of states I and D from cell (12,1) will be available with a delay of ten cycles after they begin.

[00387] Therefore, the hardware may be configured to begin operations associated with the first cell in the next capture strip 35, for example, at the cell at coordinates (0, 10). After processing cell (0, 10), cell (13, 1) can be traversed. Then the entire diagonal of cells is traversed, starting with cell (13, 1), until cell (5, 9) is reached. Similarly, after passing cell (5, 9), the next cell to pass must be cell (13, 2). However, as before, the results of the calculations of states I and D from cell (12, 2) may be available with a delay of ten cycles after they begin. Therefore, the hardware may be configured to begin operations associated with the first cell on the second diagonal of the next capture strip 35b, for example, at a cell at coordinates (1, 10) followed by cell (0, 11).

[00388] After cell (0, 11) has been processed, cell (13, 2) may be processed in accordance with the methods described above. The entire diagonal of 35 cells is then traversed, starting at cell (13, 2), until cell (6, 9) is reached. In addition, after passing cell (6, 9), the next cell to pass must be cell (13, 3). However, here again the results of calculations of states I and D from cell (12, 3) can be available with a delay period of ten cycles after their start. Therefore, the hardware may be configured to begin operations associated with the first cell on the third diagonal of the next capture strip 35c, for example, at the cell at coordinates (2, 10) followed by cells (1, 11) and (0, 12) and so on.

[00389] This continues as above until the last cell in the first capture stripe 35a (the cell with haplotype and read coordinates (13, 9)) has been traversed, at which point the logic can completely switch to traversing the second diagonally in the second capture strip 35b, starting from cell (9, 10). The pattern briefly described above is repeated as many times as 10-base capture bands are required until the bottom capture band 35c is reached (in this example, cells that are associated with read bases having an index of 30 or greater).

[00390] More dead cells may be inserted at the bottom of the capture strip 35, as shown in FIG. 16, as cells with read indices greater than 35 and haplotype indices greater than 13. In addition, an additional row of cells may actually be added in the final capture strip 35c. These cells are indicated on line 35 in FIG. 16 and refer to a special clock cycle in which the final summation operations occur in each diagonal of the final capture strip. In these loops, the M and I states of the cells immediately above are added, and this result is itself summed to the current final sum (which is initialized to zero at the left edge of the 30 HMM matrix).

[00391] In the context of the above and in view of FIG. 16, it can be noted that in this example with a read of length 35 and a haplotype of length 14, there are 102 dead cycles, 14 cycles associated with the final sum operations, and 20 pipeline delay cycles, which ultimately gives 102 + 14 + 20 = 146 overhead cycles resources. It can also be noted that any HMM job 20 with a read length greater than 10 has dead cycles in the upper left corner of FIG. 16 do not depend on the read length. In addition, it can be noted that the dead cycles in the lower and lower right parts of FIG. 16 depend on the read length, and the minimum number of dead cycles for reads is mod(read length, 10) = 9, and the minimum number of dead cycles is mod(read length, 10) = 0. It can also be noted that the percentage of cycles of non-productive resource costs of the total number of matrix evaluation cycles, 30 HMM decreases as haplotype lengths increase (the matrix is larger for a partially fixed number of resource overhead cycles) or as read lengths increase (note: this refers to the percentage of resource overhead associated with the final summation in the matrix decreasing as the difference between the read length and the number of lines increases). Using such histogram data from analyzes of representative complete human genomes, it was found that traversing the HMM matrix in the manner described above results in less than 10% of the overhead of processing the entire genome.

[00392] Additional methods, including those described below, can also be used to reduce the number of overhead resource cycles. Having dedicated logic for the final summation operations instead of using adders in conjunction with logic for calculating the M and D states. This eliminates one row of the 30 HMM matrix. Using dead loops to start HMM matrix operations for the next HMM job in the queue.

[00393] Each grouping of ten rows of the HMM matrix 30 constitutes a “capture strip 35” in the HMM accelerator function. It should be noted that the acquisition bandwidth can be increased or decreased to meet system efficiency and/or throughput requirements. Therefore, the length of the capture strip may range from about five lines to less than about fifty lines, say from about ten lines to about forty-five lines, for example from about fifteen or about twenty lines to about forty or about thirty-five lines, in including from about twenty-five lines to about thirty lines.

[00394] Subject to the exceptions noted in the section above, relating to data acquisition cycles that would otherwise be dead cycles at the right edge of the matrix in FIG. 16, the HMM matrix can be processed one capture strip at a time. As shown in FIG. 16, the states of the cells in the bottom row of each capture strip 35a are fed into logic for calculating the states in the top row of the next capture strip 35b. Therefore, it may be necessary to store (put) and retrieve (get) state information for these cells in the bottom row, or edge, of each capture strip.

[00395] The logic for doing this may include one or more of the following: upon completion of the M, I, and D state calculations for a cell in the HMM matrix 30 with mod(read index, 10) = 9, the result is stored in state storage memory M, I, D. When computations of states M and I begin (for example, when computations of state D do not require information from cells at the top in the matrix) for a cell in the 30 HMM matrix, the previously stored cell with mod(read index, 10) = 0 is retrieved information about the states M, I and D from the appropriate memory location for storing the states M, I, D. It should be noted that in these cases, when feeding the values of the states M, I and D to line 0 (top row) instead of computing the states M and I in the HMM matrix 30 simply uses a given constant value and does not need to be recalled from memory, which is also true for the values of the states M and D, which are fed into column 0 (left column) of the D state calculations.

[00396] As noted above, the HMM accelerator may or may not contain dedicated summation resources in the hardware HMM accelerator, which exists simply for the purpose of final summation operations. However, in specific cases, as described herein, an additional row may be added to the bottom of the HMM matrix 30, and the clock cycles associated with this additional row may be used for final summation operations. For example, the actual summation can be achieved by borrowing (for example, as shown in FIG. 13) an adder from the state calculation logic M to perform the operation M+I, and also borrowing an adder from the state calculation logic D to add the newly obtained sum M+I with the current one. the accumulated value of the final summation. In such a case, the control logic to activate the final summation operation may intervene whenever the read index that directs the HMM traversal operation is equal to the length of the read sequence input for the job. These operations can be observed on line 34 towards the bottom of the HMM matrix 30 shown in FIG. 16.

[00397] Therefore, as may be noted above, in one implementation, the variant detector may use matcher and/or aligner engines to determine the plausibility of the origins of various reads, for example, with respect to a given location, such as a location on a chromosome. In such cases, the variant detector may be configured to detect the underlying sequence at that location, for example, independently of other regions not immediately adjacent to it. This is particularly useful and works well when the region of interest is not similar to any other region of the genome throughout a single read (or pair of paired-end sequencing reads). However, a significant portion of the human genome does not meet this criterion, which can create problems in performing variant detection, for example, in the process of reconstructing a subject's genome from reads obtained using SNP.

[00398] In particular, although there have been tremendous improvements in DNA sequencing, identifying variants remains a difficult problem, due in large part to the redundant structure of the genome. However, as described herein, the challenges presented by genomic redundancy can be overcome, at least in part, using a short-read approach. More specifically, the devices, systems, and methods for using them described herein may be configured to focus on homologous or similar regions that might otherwise have low variant detection accuracy. In certain cases, this low variant detection accuracy may be due to difficulties observed in mapping and aligning reads to homologous regions, which can typically result in reads with very low read quality (MAPQ). Accordingly, this document proposes strategic implementations that pinpoint variants (SNPs, indels, etc.) in homologous regions, for example, by jointly considering information presented in these homologous regions.

[00399] For example, many regions of the genome are homologous, eg, having nearly identical copies found throughout the genome, eg, at multiple locations, and as a result, the true original location may be highly uncertain. Namely, if a group of reads is mapped with low confidence, for example due to obvious homology, a typical variant detector may ignore and not process these reads, even though they may contain useful information. In other cases, if the read is mismapped (for example, the primary alignment is not the true source of the read), this can lead to detection errors. More specifically, previously implemented short-read sequencing technologies have been susceptible to these problems, and traditional detection methods often remain completely unclear over large regions.

[00400] In some cases, long read sequencing can be used to mitigate these problems, but this typically costs much more and/or is more error prone, takes longer, and/or has other disadvantages. Therefore, in various cases, it may be useful to perform the multi-area joint detection operation that is described herein. For example, instead of considering each region individually and/or performing and analyzing sequencing of long reads, a multiple region joint discovery (MRJD) approach can be used, for example where the MRJD protocol considers many, such as all, locations from which each group of reads could have been obtained , and attempts to discover the underlying sequences together, eg jointly, using all available information, which can be done regardless of low or abnormal confidence and/or certainty scores.

[00401] For example, in the case of a diploid organism with statistically uniform coverage in a variant search analysis, a brute-force Bayesian computation can be performed. However, when computing MLRD by brute force, the computational complexity increases rapidly with the number of regions N and the number K of candidate haplotypes to consider. In particular, to consider all combinations of candidate haplotypes, the number of candidate haplotype solutions for which probabilities must be computed can often increase exponentially. For example, as described in more detail below, when implementing an enumeration, a number of candidate haplotypes include multiple active positions, and in the case of using a graph-based assembly method to generate a list of candidate haplotypes in a variant search operation, for example, by constructing a de Bruijn graph, as described herein, the number of active positions is the number of independent "bubbles" in the graph. Consequently, implementing such a brute-force computation may be prohibitively expensive, and the Bayesian brute-force computation itself may be prohibitively complex.

[00402] Accordingly, according to one aspect, as shown in FIG. 17A, a brute-force method for reducing the complexity of such calculations is proposed. For example, as described above, although the speed and accuracy of DNA/RNA sequencing have improved dramatically, particularly with respect to the methods described herein, identifying variants, such as the process of reconstructing a subject's genome from the reads that a sequencer produces, remains a challenging problem for the most part due to redundant genome structure. The devices, systems and methods described herein are designed to reduce the challenges presented by genomic redundancy using a short-read data-based approach, as opposed to long-read sequencing. In particular, this document provides methods for performing very long read detection based on homologous and/or similar regions of the genome that typically have low variant detection accuracy, without the need to perform long read sequencing.

[00403] For example, in one embodiment, a system and method for performing collaborative detection across multiple domains is provided. Namely, in the first case, a conventional variant determination operation can be performed, for example, using the methods described herein. In particular, a conventional variant detector may use a reference genomic sequence that represents all bases in a model genome. These references form the backbone of the analysis by which the subject's genome is compared with the reference genome. For example, as noted above, using a next-generation sequencer, a subject's genome can be broken down into subsequences, such as reads, typically about 100-1000 bases each, and these reads can be mapped and aligned to a reference, much like assembling a jigsaw puzzle of fragments .

[00404] Once a subject's genome has been mapped and/or aligned using a given reference genome for comparison with the subject's actual genome, it is possible to determine to what extent and how the subject's genome differs from the reference genome, for example, base-by-base sequentially. Specifically, when comparing a subject's genome to one or more reference genomes, for example, base by base, the analysis is performed by iteratively moving along the sequence and comparing one sequence to others to determine whether they are consistent or not. Accordingly, each base within the sequence represents a position to be determined, for example, as represented by position A in FIG. 18A.

[00405] Namely, for each reference position A to be determined relative to a subject's genome, a collection of sequences, or reads, will be mapped or aligned such that all reads from the larger set of images can cover each other at any given A position. In particular, this oversampling may contain a number of reads, for example from one to a hundred or more, where each read in the cluster has nucleotides overlapping the region being defined. Therefore, determining these reads from one base to another involves forming a processing window that slides along the sequence making determinations, with the length of the window, such as the number of bases being examined at any given time, forming the active determination region. Therefore, the window represents the active region of bases in a defined pattern, where the definition involves comparing each base at a given position, such as A, across all reads in a cluster within the active region, with the identity of the base at a given position across a number of clusters of reads providing evidence of true identity of the base in a given defined position.

[00406] For this purpose, based on the corresponding MAPQ confidence score obtained for each read segment, it can generally be determined, within a certain confidence score, that the mapping or alignment has been performed accurately. However, there is still a question, however minor, about whether the mapping or alignment of reads is accurate or not, and whether one or more reads actually belong somewhere else. Accordingly, in one aspect, devices and methods are provided herein for improving confidence in performing variant determinations.

[00407] In particular, in various cases, the variant determiner may be configured to perform one or more joint detection operations in multiple areas, as described herein, which can be used to impart greater confidence to the achievable results. For example, in such a case, the variant detector may be configured to analyze different regions in the genome to determine specific regions that appear similar. For example, as shown in FIG. 18A, there may be a reference region A and a reference region B, where the reference regions are very similar to each other, for example, except for a few regions with base pair dissimilarity, such as where example Ref A has "A" and example Ref B has "T", but beyond these dissimilarities anywhere else in the study area may be the same. Due to the degree of similarity, these two regions, for example, Ref A and Ref B, will generally be considered homologous, or paralogous, regions.

[00408] As shown in the figure, reference regions A and B are 99% similar. There may be other regions, such as Ref C and Ref D, that are relatively similar, for example about 93% similar, but compared to the 99% similarity between Reference Regions A and B, Reference Regions C and D will not be considered homologous. or at least will have less chance of actually being homologous. In such a case, variant detection procedures may be capable of adequately distinguishing between reference regions C and D, but may, in certain cases, have difficulty distinguishing between highly homologous reference regions A and B, for example due to their high homology. In particular, due to the degree of dissimilarity between reference sequences A and B and reference sequences C and D, mapping and alignment to reference sequence A or B should not be expected to be mistaken for mapping to reference sequence C or D. However, reads can be expected to , which map and align to reference sequence A may be incorrectly mapped to reference sequence B.

[00409] Given the degree of homology, mismapping between regions A and B may be quite likely. Accordingly, to improve accuracy, it would be advantageous to be able to distinguish and/or account for differences between homologous regions, for example, when performing a mapping, alignment and/or variant determination procedure. Namely, when generating a cluster of reads that map or align to a region within Ref A, and when generating a cluster of reads that map and align to a region within Ref B, either read may in fact be incorrectly mapped to the wrong location and, Therefore, to achieve higher accuracy, when performing the variant detection operations described herein, these homologous regions and the reads being mapped and aligned thereon must be considered together, for example, in a co-discovery protocol such as a multi-region co-discovery protocol , which is described in this document.

[00410] Accordingly, devices, systems, and methods of using them are provided herein that relate to multiple-region collaborative discovery (MRJD), such as when multiple, e.g., all, reads from different clusters of different identified homologous regions are considered together, e.g. where instead of performing one candidate determination for each location, joint discovery is performed for all locations that appear homologous. Making such joint determinations has the advantage that before attempting to make determinations for each reference individually, one would first need to determine to which region of which reference the various reads under study are actually mapped and aligned, and this is inherently not known with certainty, and precisely this problem is addressed by the proposed collaborative detection. Consequently, since the regions of two references are so similar, it is very difficult to determine which reads are mapped to which regions. However, if the definition of these regions is performed jointly, there is no need to make a prior decision about which homologous reads map to which reference region. Therefore, when performing joint definition, it can be assumed that any reads in a cluster of a region on one reference, for example, A, that are homologous another area in the second reference, for example B, could belong to either Ref. A, or Ref. B.

[00411] Therefore, if desired, the MRJD protocol may be implemented in addition to the variant determination algorithm implemented in the devices, systems, and methods set forth herein. For example, in one iteration, the variant detection algorithm takes into account the supporting data presented in mapped and/or aligned reads for a given region in the sample genome and the reference genome, based on comparison with the reference genome, analyzes the possibility of the actual presence in the sample genome of something that could be present in it, and, taking into account this evidence, makes a decision about how the sample really differs from the reference, for example, taking into account this evidence, the algorithm for determining the options determines the most likely answer, what is the difference between the read and the reference. However, MRJD is also an algorithm that can be implemented alongside a variant detector algorithm, wherein the MRJD is configured to assist the variant detector in more accurately determining whether an observed difference, for example in a subject's read, is actually a true deviation from the reference.

[00412] Accordingly, the first step in MRJD analysis involves identifying homologous regions based on the percentage of sequence correspondence between multiple regions of one or more references, for example, Ref. A and Ref. B, and sequences of clusters in one or more regions of the subject's reads. In particular, Ref. A and Ref. B may actually be diploid forms of the same genetic material, for example when there are two copies of a given region of the chromosome. Therefore, when analyzing diploid references, Ref A may have one specific nucleotide at different positions, but at the same position in Ref. B another nucleotide may be present. In this example Ref. A and Ref. B are homozygous at the A position for "A". However, as shown in FIG. 18A, the subject's DNA is heterozygous at this position A, for example, when relative to the cluster reads of the region Ref. A one allele of the subject's chromosome has "A" and the other allele has "C", and as for Ref. B, another copy of the subject's chromosome has an "A" in both alleles at position A. This too becomes more complex when the sample being analyzed contains a mutation, for example at one of those naturally occurring variable positions, such as a heterozygous SNP at position A ( not shown).

[00413] As shown in Ref. A shown in FIG. 18B, at position A, a subject's sample may contain reads that indicate the presence of heterozygosity at position A, for example, when some of the reads contain "C" at that position and some of the reads indicate "A" at that position (e.g., haplotype _a1 = "A", H _a2 = "C"); as for Ref. B, reads at position A indicate homozygosity, e.g. when all reads in a cluster have “A” at that position (e.g. H _b1 = “A”, H _b2 = “A”). However, MJRD overcomes these difficulties by simultaneously performing joint determination by analyzing all reads that map to both reference regions, while taking into account the possibility that reads may be in the wrong location. Once the various homologous regions have been identified, the next step is to determine the correspondence between the homologous reference regions and then, in accordance with the MRJD, determining, using the mapper and/or aligners, where the various suitable reads are “putatively mapped” between the two homologous regions can be discarded and , or rather, all reads in any of the clusters in these homologous regions can be considered together, taking into account the knowledge that any of these reads can belong to any of the homologous regions being compared. Therefore, the calculations to determine these joint definitions, as detailed below, take into account the possibility that any of these reads come from any of the homology reference regions and, where applicable, from any of the haplotypes of any of the reference regions.

[00414] It should be noted that although the above was concerned with multiple regions of homology within a reference, the same analysis can also be applied to the detection of a single region. For example, as shown in FIG. 18B, even if there is one region, in any given region there may be two distinct haplotypes, such as H ₁ and H ₂ , that a subject's genetic pattern may have for a particular region, and since they are haplotypes they are likely to be very similar to each other on a friend. Therefore, if these items were analyzed separately from each other, it would be difficult to determine whether there is true variation being accounted for. Thus, calculations performed on homologous regions are also useful for non-homologous regions, since any given region is likely to be a diploid, for example having both a first haplotype ( _H1 ) and a second haplotype ( _H2 ), and therefore joint analysis of these areas will improve the accuracy of the system. Similarly, in the case of a region with two references, for example, a homologous region, as described above, it is H _A1 and H _A2 for the first region and H _A1 and H _A2 for the second region that are determined (which is equivalent to two strands for each chromosome and two regions for each strands = 4 diploid types, typically).

[00415] Accordingly, MRJD can be used to determine an initial response about one or more, e.g., all, homologous regions, and then the detection of one region can be applied back to one or more, e.g., all, single, or non-homologous regions, e.g. using the same basic analysis, and thus greater accuracy can be achieved. Therefore, it is also possible to perform non-joint determination of one region. For example, regarding the definition of one region, for candidate H _A1 haplotypes in current iterations, the reference genome can be about 300-500 base pairs in length, and a graph, for example, a de Bruijn, is built on top of the reference, as shown in FIG. 18C, for example, from K-mers from reads, where any location that differs from the reference forms a deviant path or "bubble" in the graph from which haplotypes are extracted, where each isolated haplotype, for example, a divergent path, forms a potential hypothesis about , which could be on one of the two strands of the chromosome at a specific location in the active region being studied.

[00416] However, if there are many divergent paths, for example, many bubbles are formed throughout the graph, as shown in FIG. 18C, and a large number of haplotypes have been identified, then a maximum cutoff can be introduced to keep the calculations manageable. The cutoff may be at any statistically significant number, such as 35, 50, 100, 125-128, 150, 175, 200 or more, etc. However, in certain cases a larger number, e.g. all, haplotypes may be considered.

[00417] In such cases, instead of isolating complete haplotypes from source to sink from beginning to end, for example, from the beginning of a sequence to its end, only the sequences associated with individual bubbles need to be isolated, for example, only bubbles need to be aligned to a reference. Accordingly, bubbles are isolated from the DBG, the sequences are aligned to the reference, and from these alignments specific SNPs, insertions, deletions, etc. can be identified. in terms of why the sequences of different bubbles differ from the reference. Therefore, in this sense, all the different hypothetical haplotypes for analysis can be obtained by mixing and matching sequences belonging to all the different bubbles in different combinations. With this approach, it is not necessary to list all the haplotypes to be isolated. These methods for performing joint detection in multiple areas are described in more detail later in this document.

[00418] Additionally, hypothetically, even if all of these candidate haplotypes could be tested, the tree growing algorithm could be executed where the graph being created begins to look like a growing tree. For example, a branching tree graph of joint haplotypes/diplotypes can be constructed such that, as the tree grows, the underlying algorithm functions to both grow and prune the tree simultaneously as more and more calculations are performed and it becomes apparent that that all the different candidate hypotheses are simply too implausible. Therefore, as the tree is grown and pruned, not all hypothetical haplotypes need to be calculated.

[00419] Namely, with regard to the tree alignment function, when there is inconsistency between two references or between references and reads about which base is present at given positions being solved, it is necessary to determine which base actually belongs to which position, and in view of such inconsistencies, it is necessary determine which differences may be caused by SNPs, indels, etc., and which are machine errors. Accordingly, when growing a tree, for example by extracting bubbles from the de Bruijn graph, for example using Smith-Waterman or Needleman-Wunsch alignment, and positioning them within the emerging tree graph, each bubble to be extracted becomes an event in the tree graph. which represents possible SNPs, indels and/or other differences from the reference (see FIG. 18C).

[00420] In particular, on the DBG, bubbles represent mismatches to the reference, such as representing indels (when bases have been added or removed), SNPs (when bases are different), and the like. Therefore, as bubbles are aligned to the reference(s), the various differences between them are categorized as events and a list of different events, such as bubbles, is generated. Because of this, the definition then becomes: what combination of possible events, e.g., possible SNPs and indels, that led to the actual variation in a subject's genetic sequence, e.g., is true in each of the actual different haplotypes, e.g., 4, based on probability. More specifically, any one candidate, for example the joint candidate diplotype forming the _G0 root (representing events for a given segment) could have 4 haplotypes, and which of the four haplotypes would form the identified subset of events.

[00421] However, as shown in FIG. 18D, when performing the function of growing and/or pruning a tree, a complete list of the entire subset of event combinations may or may not be determined all at once. Instead, the definition starts at one position of G ₀ , for example with one event, and grows the tree from there one event at a time, which can result in the pruning function leaving various low-probability events unresolved. Thus, regarding the tree growing function, as shown in FIG. 18D, the calculation begins by defining haplotypes, eg, H _A1 , H _A2 , H _B1 , H _B2 (for a diploid organism), with the initial haplotypes considered all unresolved with respect to their respective references, eg, Ref. A and Ref. B, with essentially none of the events present.

[00422] Accordingly, the initial starting point is the root of the tree, which is G ₀ , and a joint diplotype in which all events are unresolved. A specific event, such as the initial bubble, is then selected as the start for determination, whereby the starting event must be resolved for all haplotypes, where this event may be the first point of divergence from the reference, for example, in relation to the potential presence of a SNP or indel at position one . As explained in FIG. 18E, at position one there is an event or bubble, such as an SNP, where "C" is replaced by "A", so that the reference has an "A" at position one and the read in question has a "C". In this case, since there are 4 haplotypes in the cluster for this position, and each can have either an “A” as a reference or a “C” event, there are potentially 2 ⁴ = 16 possibilities for resolving this position. Therefore, the computation goes straight from the root to the 16 branches representing potential resolutions for each event at position one.

[00423] Therefore, as shown in FIG. 18D, it is possible to determine all potential sequences for all four haplotypes, for example, H _A1 , H _A2 , H _B1 , H _B2 , where at position one there is either an “A”, which corresponds to the reference, or a “C” event, indicating the presence of an SNP , for this one event, where event "C" is determined by examining the different bubble paths through the graph. So, for each branch or child node, each branch can be distinguished on the basis of the plausibility of the base in position one, corresponding to or implied by the reference, while the remaining events remain unresolved. This process will then be repeated for each branch node and for each base within the variation bubbles to resolve all events for all haplotypes. Therefore, it is possible to recalculate the probabilities of observing any particular read given different potential haplotypes.

[00424] Specifically, there can be four haplotypes for each node, and each haplotype can be compared to each read in the cluster. For example, in one embodiment, a Smith-Waterman, Needleman-Wunsch, and/or HMM engine analyzes each node and considers each of the four haplotypes for each node. Therefore, generating each node activates the Smith-Waterman engine and/or HMM to analyze that node by considering all haplotypes, for example 4, for that node compared to each read, where the Smith-Waterman engine and/or HMM considers one haplotype per one read for each of the haplotypes and each of the reads for all viable nodes.

[00425] Thus, if in this example we assume that there is a heterozygous “C” SNP for one region of one haplotype, for example, one strand of the chromosome has a “C”, and all other strands have different bases at this position, for example, they all match the reference "A", then we would expect that all reads in the cluster support this data, for example, most of them have "A" in position one, and a minority, for example about ¼, of reads have "C" in position one in case of a true node. Therefore, if any previously observed events in another node show a set of "C" at position one, then that node is unlikely to be a true node, for example, will have a low probability since there will not be enough reads in the cluster with "C" at that position. to make its appearance probable. Namely, it is more likely that the existence of a “C” at this position in the reads under study indicates a sequencing error or other research error rather than the presence of a true candidate haplotype. Therefore, if certain nodes are pruned having low probabilities compared to the true node, the reason is that they are not supported by the majority of reads, such as in a cluster, and so these nodes can be pruned, thereby discarding the low probability nodes but still retaining the true ones nodes.

[00426] Accordingly, once event position one has been determined, the next event position can be determined, and the process described herein can then be repeated for that new position with respect to any of the surviving nodes that have not yet been pruned. In particular, event two can be selected from existing available nodes, and this event can serve as the G ₁ root for determining the probable identity of the base at position two, for example again by defining new haplotypes, for example 4, as well as their various branches, for example, 16, explaining possible variations regarding position 2. Thus, by repeating this same process, event 2 can now be resolved. Therefore, as shown in FIG. 18D, once position 1 is determined, a new node for position 2 can be selected and its 16 candidate haplotypes considered. In such a case, it is possible to determine candidates for each H _A1 , H _A2 , H _B1 , H _B2 , but now, since position 1 has already been solved for determining the nucleotide identity for each haplotype at position 1, it is position 2 that will now be resolved for each haplotype in position 2, as indicated in FIG. 18D showing position 2.

[00427] At the end of this process, after all events have been processed and resolved, for example, including child nodes and their children nodes that were not pruned, the unpruned nodes of the tree can be examined and, based on probability estimates, determined which tree represents the joint diplotype, for example, which sequence has the highest probability of being a tree. Therefore, in view of the pruning function, there is no need to build a complete tree, for example, most of the tree will break off by being pruned as the analysis continues, so the overall computational effort is greatly reduced compared to functions without pruning, although it is significantly larger than when performing a non-shared diplotype determination, such as a single region determination. Accordingly, the presented analytics modules can identify and resolve two or more regions of high homology with a high degree of accuracy, for example using joint diplotype analysis, where traditional methods are simply unable to resolve such regions at all, for example due to false positives and undecidability.

[00428] In particular, various implementations of the variant detector may be configured to simply not perform the analysis on regions of high homology. The presented iterations overcome these and other similar problems in the art. More specifically, the presented devices, systems and methods of using them can be configured to consider most, for example all, haplotypes, despite the presence of regions of high homology. Of course, the speed of these calculations can be further increased by not performing the calculations when the results of such calculations can be determined to have a low probability of being true, for example by implementing a cutoff function as described herein.

[00429] The advantage of these configurations, such as joint diplotype solution and trimming, is that the window size of the active region, such as the bases being analyzed, can be increased from about a few hundred bases being processed to several thousand or even tens of thousands of bases that can be processed together, for example in one continuous active region. This increase in the size of the active analysis window allows more supporting data to be considered when determining the identity of any particular nucleotide at any given position, thereby providing more context within which a more precise determination of nucleotide identity can be made. Likewise, broader context allows better tying together of supporting evidence when comparing one or more reads covering one or more regions that have one or more deviations from the reference. Therefore, in this way, one event can be connected to another, which itself can be connected to yet another event, etc., and based on these connections, a more precise determination can be made in relation to this particular event in question, thereby providing informative confirmation data from even further away, such as hundreds to thousands or more distant bases, while performing true variant detection (despite the fact that any given read is typically hundreds of bases in length), thereby further increasing accuracy processes described in this document.

[00430] In particular, further incorporation of thousands to tens of thousands, even hundreds of thousands or more bases into the active region can be achieved in this manner, and therefore the method of forming a de Bruijn graph by isolating all haplotypes can be avoided, since only a limited number of haplotypes need to be examined , those that have bubbles that may be viable, and even those that are viable can be trimmed after they become nonviable, and for those that remain viable, chaining can be used to improve the accuracy of the eventual variant determination . All of this is made possible by quantum and/or hardware computing. This can also be done, but more slowly, in software using a CPU or GPU.

[00431] Regarding the above examples, it should be noted that the probabilities of the input data, such as reads, are determined taking into account the hypothetical haplotypes created using the de Bruijn graph. However, it may also be useful to use Bayes' theorem, for example, to determine the probability of reads given a joint diplotype, up to the opposite probability of determining from the joint diplotype theory the best fit, given the reads being evaluated and the supporting data. Accordingly, as shown in FIG. 18C, based on the generated de Bruijn graph, after multi-region co-detection and/or pruning is performed, a set of potential haplotypes will be obtained, and these haplotypes will then be checked against the subject's actual reads. Specifically, each horizontal cross-section represents a haplotype, such as B1, which can then be subjected to a different read-matching HMM protocol to determine the probability of a particular read given the B1 haplotype.

[00432] However, in certain cases the haplotype, such as B1, may not yet be fully determined, but running an HMM may still be beneficial, in which case a modified HMM calculation, such as a partially determined HMM, (PD) operation, can be performed. -HMM, discussed below, where the haplotype is allowed to have undetermined variants, such as SNPs and/or indels that have yet to be determined, and the calculation is essentially like calculating the best possible probability of an achievable response given any combination of variants at unresolved positions. Thus, this further facilitates the iterative tree growing function, where the actual tree growing, such as performing PD-HMM operations, does not have to be limited to only those calculations where all possible options are known. Therefore, in this way, it is possible to perform a series of PD-HMM calculations in an iterative way to grow a tree of nodes despite the fact that there are still undefined regions of unknown possible events in specific candidate haplotypes, and so where it becomes possible to prune the tree, PD-resources HMMs can be seamlessly switched from computing pruned nodes to processing only those possibilities that have the highest probability of successfully characterizing the true genotype.

[00433] Accordingly, when determining the likelihood of a particular base actually being present at any one position, the identity of the base at that position can be determined based on the identity at that position in each region of the chromosome, for example, in each haplotype that represents a viable candidate. Therefore, any candidate whose determination is made is the identity of a given base at the position in question in each of the four haplotypes simultaneously. In particular, the probability of observing reads in each cluster is determined, subject to a certain likelihood. Namely, each candidate represents a joint diplotype, and by virtue of this, each candidate contains about four haplotypes, which can be represented in the following equation as G = genotype, where G = four haplotypes of one diploid chromosome region of the genome, such as a joint diplotype. In this case, it is necessary to calculate the probability of actually observing each of the identified candidate bases of the sequence read in clusters, assuming that they are indeed true. This initial determination can be made by calculating the HMM haplotype as described below in this document.

[00434] For example, "JointDiplotype" = 4 haplotypes: (region A: H _A1 H _A2 , and region B:⋅H _B1 H _B2 ) = G → P(R/G) as determined by HMM (error model) = ΠP(r/G) =

[00435] Therefore, if a particular H _a1 haplotype is assumed to be the true sequence in that region, and the read originates from there, then what are the chances that the given H _a1 of that read sequence was actually observed. Accordingly, the HMM evaluator works to determine, assuming that the H _a1 haplotype is true, what is the likelihood of actually observing a given read sequence of interest.

[00436] Namely, if the read actually matches the haplotype, the probability, of course, will be very high. However, if a particular read being examined does not match a haplotype, then any deviation from it must be attributed to research error, such as sequencing or sequencing equipment error, rather than actual variation. Therefore, the calculation of the HMM is a function of the error models. Namely, the question is what is the probability of the necessary combination of errors that would have to occur in order for these particular analyzed reads to be observed. Consequently, this model considers not only one region, but also multiple positions in multiple regions on multiple strands simultaneously (e.g., instead of considering the likelihood of two haplotypes in one region, it now simultaneously considers the likelihood of four haplotypes for any given position, simultaneously, using all read data from all study areas). Let us now take a closer look at these processes, such as tree pruning, multi-region collaborative detection, and PD-HMM.

[00437] Namely, as shown in FIG. 17 and 18, a high-level processing chain is proposed, for example, where the processing chain may include one or more of the following steps: identifying and inputting homologous regions, performing preprocessing of input homologous regions, performing very long read detection (VLRD) or co-detection in multiple regions (MJRD) and output a variant definition file. In particular, with respect to identifying homologous regions, a file of mapped, aligned and/or sorted SAM and/or BAM data, such as CRAM, may be input as primary input to the multi-region joint detection processing engine implementing the MRJD algorithm. The MRJD processing engine may be part of an integrated circuit, such as a CPU and/or GPU, and/or a quantum computing platform executing software, such as a quantum algorithm, or may be implemented in an FPGA, ASIC, or the like. For example, the mapper and/or aligner described above can be used to generate a CRAM file and configured to output N secondary alignments for each read along with the primary alignments. These primary and secondary alignments can then be used to determine a list of homologous regions, where homologous regions can be calculated based on a user-defined threshold of similarity between N regions of the reference genome. This list of identified homologous regions can then be fed to the preprocessing stage of a suitably configured MRJD module.

[00438] Accordingly, in the preprocessing step, for each set of homologous regions, a co-cluster may first be formed, for example, using primary alignments from one or more, for example, each of the regions in the set. See, for example, FIG. 19. Using this co-aggregation, a list of active/candidate variant positions (SVPs/indels) can then be generated, with which each of these candidate variants can be processed and scored through the MRJD pre-processing engine(s). To reduce computational complexity, a connectivity matrix can be computed, which can be used to determine the order in which candidate variants are processed.

[00439] In such embodiments, the multi-domain joint detection algorithm evaluates each identified candidate variant based on the processing order determined in the generated connectivity matrix. First, one or more joint candidate diplotypes (G _i ) for a candidate variant can be generated and returned. The posterior probabilities of each joint diplotype (P(G _i |R)) can then be calculated. From these posterior probabilities, a genotype matrix can be calculated. The N diplotypes with the lowest posterior probabilities can then be pruned to reduce the computational complexity of the calculations. You can then include the next candidate that provides supporting evidence for the current candidate being evaluated and repeat the above process. By including information from, for example, one or more, for example, all, candidate variants from one or more, for example, all, regions in the set of homology regions for the current variant, variant detection can be performed based on the final genotyping matrix. Thus, each of the active positions can be evaluated as described above, thereby obtaining the final VCF file.

[00440] Specifically, as shown in FIG. 17B, the MRJD processing step may be implemented, for example, by including one or more of the following steps or blocks. The identified and collected co-cluster is loaded, then a list of candidate variants is created from the collected co-cluster and a connectivity matrix is calculated. In particular, in various embodiments, a preprocessing technique may be implemented, for example, to perform one or more variant determination operations, such as a multiple read co-discovery operation. Such operations may include one or more preprocessing blocks, including steps related to loading co-clusters, generating a candidate list from the co-cluster, and computing a connectivity matrix. Now let's take a closer look at each of the blocks and the potential stages associated with them.

[00441] Namely, a first co-cluster pre-processing block may be included in the analysis procedure. For example, for the identified gap, various reference regions can be extracted, for example, from mapped and/or aligned reads. In particular, using a list of homologous regions, a joint cluster can be formed for each set of homologous regions. After this, using a user-specified gap, N reference regions can be identified corresponding to N homologous regions in the set. One or more, for example, all reference regions can then be aligned, for example using a Smith-Waterman alignment, which can be used to form a universal coordinate system of all bases in the N reference regions. Next, all primary reads corresponding to each region can then be extracted from the input SAM or BAM file and mapped to universal coordinates. This mapping can be performed as described herein, for example, using the alignment information (CIGAR) provided in the CRAM file for each mapping. In settings where some reads have not been previously mapped, these reads can be mapped and/or aligned, for example using the Smith-Waterman algorithm, to their corresponding reference region.

[00442] More specifically, after a joint cluster has been generated and loaded (see, for example, FIG. 19), a list of candidate candidates can be created, for example, from the joint cluster. For example, to highlight the different candidate variants (SNPs/indels) that can be identified from a co-assembly, a de Bruijn graph (DBG) or other assembly graph can be created. Once the DBG is created, various bubbles can be searched on it to obtain a list of candidate variants.

[00443] In particular, given all the reads, a graph can be constructed using each reference region as a skeleton. All positions of the identified candidate variant can then be aligned to universal coordinates. A connectivity matrix can then be computed, which determines the order in which active positions are processed, which can be a function of read lengths and/or insertion size. In FIG. 19 shows an example referenced herein to a joint cluster of two homologous regions on chromosome 1. Although this cluster is constructed from two homologous regions of chromosome 1, this is for illustrative purposes only, as this clustering process can be used for any and all homologous regions outside depending on the chromosome.

[00444] As shown in FIG. 20, a candidate list can be generated as follows. You can first form a joint cluster and construct a de Bruijn graph (DBG) or other assembly graph in accordance with the methods described herein. DBG can then be used to isolate candidate variants from co-clusters. DBGs are constructed to form bubbles that indicate variations representing alternative paths through the graph, with each alternative path being a candidate haplotype. See, for example, FIG. 20 and 21.

[00445] Accordingly, the various bubbles in the graph represent a list of haplotype positions of a candidate variant. Therefore, if all the reads are available, a DBG can be constructed using each reference region as a scaffold. All candidate variant positions can then be aligned to universal coordinates. In particular, in FIG. 20 is a flowchart showing the process of generating a DBG and using it to generate candidate haplotypes. More precisely, the de Bruijn graph can be used to generate a list of candidate SNP variants and indels. Given that there are N regions that are jointly processed by MRJD, N de Bruijn graphs can be constructed. In this case, each graph can use one reference region as a backbone and all reads corresponding to N regions.

[00446] For example, in one implementation method, after constructing a DBG graph, candidate haplotypes can be extracted from it based on candidate events. However, when using the MRJD preprocessing protocol as described herein, N regions can be processed together, for example, where the length of the regions may be several thousand bases or more and the number of haplotypes to be isolated may grow exponentially very quickly. Accordingly, to reduce computational complexity, instead of extracting all haplotypes from the graphs, we need to extract only the bubbles that represent candidate variants.

[00447] An example of bubble structures formed on a de Bruijn graph is shown in FIG. 21. A number of areas that are subject to joint processing have been identified. This determines one of two processing paths that can be followed. If all matching regions are identified, all reads can be used to form a DBG. Bubbles showing possible variants can be highlighted to identify different candidate haplotypes. Namely, for each bubble, a Smith-Waterman alignment can be performed on alternative paths to the reference skeleton. From here, candidate options can be selected and events from each graph can be stored.

[00448] However, in other cases, after the first process to form one or more DBGs has been performed and/or i is now 0, a union of all candidate events from all DBGs can be formed and any duplicates removed from there. In such a case, all candidate variants can be mapped, for example, onto a universal coordinate system to create a list of candidate variants, and this list of candidate variants can be sent as input to a pruning module, such as an MRJD module. An example of performing bubble extraction instead of all haplotypes is shown in FIG. 22. In this case, only the area of the bubble showing possible options is isolated and processed, as described herein.

[00449] Namely, once representative bubbles are isolated, a global alignment, such as a Smith-Waterman alignment, of the bubble paths and the corresponding reference scaffold can be performed to obtain candidate variants and their positions in the reference. This can be done for all selected bubbles in all de Bruijn graphs. Next, from N graphs one can obtain a union of all selected candidate options, remove duplicates of potential options, if any, and the positions of unique candidate options can be mapped to a universal coordinate system obtained from the joint cluster. This produces a final list of candidate variant positions for N regions that can act as input to the “pruned” MRJD algorithm.

[00450] In particular, using the processing blocks that are described above herein, a connectivity matrix can be constructed. For example, the connectivity matrix can be used to determine the order in which active, e.g. candidate, positions are processed, for example, as a function of the read length and insertion size. For example, to further reduce computational complexity, a connectivity matrix can be computed to determine the processing order of identified candidate variants that are derived from the de Bruijn graph. This matrix can be constructed and used as a sorting function or in conjunction with it to determine which candidate variants should be processed first. Therefore, this connectivity matrix can be a function of the average length of reads and the insertion size of paired-end reads. Accordingly, for a given candidate variant, other candidate candidate positions that are integer multiples of the insertion size or are within the read length are weighted higher than candidate variants at other positions. The reason is that these candidate options are more likely to provide supporting evidence for the current option being evaluated. In FIG. Figure 23 shows an example of the sorting function that is implemented herein, for an average read length of 101 and an insertion size of 300.

[00451] Regarding the MRJD trimming function, examples of steps of the MRJD trimming algorithm referred to herein are shown in FIG. 24. For example, the input to the MRJD platform and algorithm is a joint cluster of N regions, such as all candidate variants (SNPs/indels), prior probabilities based on the mutation model, and a connectivity matrix. Accordingly, the input data of the trimmed MRJD processing platform may be the co-cluster, the identified active positions, the generated connectivity matrix and the posterior probability model and/or its results.

[00452] Next, each candidate in the list can be processed, and the remaining candidates can be sequentially added as supporting data for the currently processed candidate using a connectivity matrix. Accordingly, given the current candidate variant and any supporting candidate variants, joint candidate diplotypes can be generated. For example, a co-diplotype is a set of 2N haplotypes, where N is the number of co-processed regions. The number of co-candidate diplotypes M is a function of the number of co-processed regions, the number of active/candidate variants considered, and the number of phases. An example of the formation of joint diplotypes is shown below.

For: P = 1, number of positions of active/candidate options under consideration;

N = 2, number of jointly processed areas;

M = 2 ^2⋅N⋅P = 2 ⁴ = 16 joint candidate diplotypes

[00453] Therefore, let there be one active candidate position and all the reads and both reference regions are given, and let the two haplotypes be “A” and “G”.

Unique haplotypes = "A" and "G".

Candidate haplotypes = 'AA', 'AG', 'GA' and 'GG', (4 potential haplotypes per region).

Joint candidate diplotypes =

[00454] Accordingly, using the joint diplotype candidates, haplotype-aware read likelihoods can be calculated for each haplotype in each set of joint diplotype candidates. This can be done using the HMM algorithm as described herein. However, the HMM algorithm can be modified from its standard application to take into account candidate variants (SNPs/indels) in the haplotype that have not yet been processed. Accordingly, the likelihoods of reads can be calculated in the presence of a joint diplotype (P(r _i |G _m )) using the results from the modified HMM. This can be done using the following formula.

[00455] In case of joint definition in 2 areas:

G _m =

[ϑ _11,m, ϑ _12,m, ϑ _21,m, ϑ _22,m ], and in ϑ _ij,m , i is the region, and j is the phase, P(r _i |G _m ) =

P(R| _Gm ) = . Given P(r _i |G _m ), one can easily calculate P(R|G _m ) for all reads. Next, using Bayes' formula, you can calculate the prior probability (P(G _i |R)) from P(R|G _i ) and the prior probabilities (P(G _i )).

P(G _i |R) = P(R|G _i ) P(G _i )/ .

[00456] In addition, an intermediate genotype matrix can be calculated for each region, taking into account the posterior probabilities for all joint candidate diplotypes. For each combination of events in the genotype matrix, all posterior probabilities of all joint diplotypes supporting that event can be summed. At this stage, the genotype matrix can be considered as "intermediate" since not all candidate variants supporting the current candidate are included. However, as noted earlier, the number of joint candidate diplotypes grows exponentially depending on the number of candidate variant positions and the number of regions. This in turn exponentially increases the computation required to calculate the posterior probabilities. Therefore, to reduce the computational complexity at this stage, the number of joint diplotypes can be pruned based on posterior probabilities to maintain a number of joint diplotypes that can be defined by the user and programmed. Finally, the final genotype matrix can be updated based on the user-defined variant metrics that are computed using the intermediate genotype matrix. The various steps of these processes are shown in the process flow diagram depicted in FIG. 24.

[00457] The above process can be repeated until all candidate variants are included as supporting data for the current candidate variants processed by the connectivity matrix. Once all candidate variants are included, the current candidate variant is processed. Other stopping criteria are also possible for candidate processing. For example, the process can be stopped when the confidence stops increasing with the addition of further candidate options. This analysis, as exemplified in FIG. 24 can be restarted and repeated in a similar manner for all other candidate options in the list, resulting in a final MRJD output option definition file. Accordingly, rather than considering each region individually, the multi-region co-discovery protocol described herein can be used to consider all locations from which a group of reads may have originated, as it attempts to discover the underlying sequences together using all available information.

[00458] Accordingly, in the case of joint detection in multiple areas, in an example MRJD protocol, one or more of the following equations may be used in accordance with the methods described herein. Namely, rather than looking at each region individually, MRJD considers multiple locations from which a group of reads may have originated and attempts to jointly discover their underlying sequences, e.g., using as much of the available information, e.g., as all of it, as is useful. For example, in one example embodiment:

[00459] Let N be the number of areas to be jointly processed. And let H _k be a candidate haplotype, k = 1...K, each of which contains different SNPs, insertions and/or deletions compared to the reference sequence. Each H _k haplotype represents one region along one strand (or "phase", e.g. maternal or paternal), and they are not necessarily continuous (e.g., may contain gaps or "don't care" sequences).

[00460] Let G _m be the candidate solution for both phases Φ = 1.2 (for a diploid organism) and all regions n = 1...N:

G _m =

where each element G _m,Φ,n is a haplotype selected from the set of candidate haplotypes {H ₁ ...H _k }.

[00461] First, one can calculate the probability of each read for each haplotype P(r _i |H _k ), for example, using a hidden Markov model (HMM). In the case of data sets with paired reads, r _i specifies the pair {r _i,1 , r _i,2 }, and P(r _i |H _k ) = P(r _i,1 |H _k ) P(r _i,2 | _Hk ). In the case of datasets with related reads (e.g. reads in barcodes), r _i indicates a group of reads {r _i,1 ...r _i,NL } that are formed from the same molecule, and P(r _i |H _k ) = .

[00462] Next, for each candidate solution G _m , m=1...M, calculate the conditional probability of each read P(r _i |G _m ) = and the conditional probability of each complete cluster R ={r ₁ …r _NR }: P(R|G _m ) = .

[00463] Next, calculate the posterior probability of each candidate solution given the observed cluster: P(G _m |R) = , where P(G _m ) indicates the posterior probability of a candidate solution, which is described in detail later in this document.

[00464] Finally, the relative probability of each candidate variant V _j is calculated = / , for example, where G _m → V _j indicates that G _m supports the variant V _j , and G _m → ref indicates that G _m supports the reference. In the VCF file this can be reported as a quality rating on the phred scale: QUAL(V _j ) = -10log ₁₀ .

[00465] An example of a process for performing various variant determination operations is described herein with reference to FIG. 25, which compares the traditional detection process and MRJD. Namely, in FIG. Figure 25 shows a co-aggregation of paired-end reads for two regions whose reference sequences differ by only 3 bases across the entire range of interest. All reads are known to originate from either region #1 or region #2, but it is not known with certainty which region any individual read originates from. Note, as described above, bases are shown only for positions where the two references differ, such as bubble regions, or where reads differ from the reference. These areas are called active positions. All other positions can be ignored as they do not affect the calculation.

[00466] Accordingly, as shown in FIG. 25, in a traditional detector, pairs of reads 1-16 will be mapped to region #2, and only these will be used to identify variants in region #2. All these reads match the reference in area No. 2, so no variants will be found. Similarly, read pairs 17-23 will be mapped to region #1, and only these will be used to identify variants in region #1. As you can see, all these reads coincide with the reference in area No. 1, so no variants will be found. However, read pairs 24-32 are equally suitable for region #1 and region #2 (each differing by one base from reference #1 and reference #2), so the mapping is uncertain and a typical variant resolver will simply ignore these reads. Therefore, a traditional variant determiner will not perform variant determination for either of the two regions, as shown in FIG. 25.

[00467] However, in the case of MRJD as shown in FIG. 25, the result is completely different from the result obtained by applying traditional methods. The corresponding calculations are given below. In this case, N = 2 areas. In addition, there are three positions, each with 2 candidate bases (bases whose number is low enough can be safely ignored, and in this example the number is zero for all but 2 bases at each position). If all combinations are considered, this will give K = 2 ³ = 8 candidate haplotypes: H ₁ = CAT, H ₂ = CAA, H ₃ = CCT, H ₄ = CCA, H ₅ = GAT, H ₆ = GAA, H ₇ = GCT, H ₈ = GCA.

[00468] In a brute force calculation where all combinations from all candidate haplotypes are taken into account, the number of potential solutions will be M = K ^2N = 8 ^{2 2} = 4096, and for each candidate solution G _m one can calculate P(G _m /R ). This calculation for two candidate solutions is illustrated below.

G _m1 = , G _m2 =

Where G _m1 has no variants (this is the solution found by a traditional detector), and G _m2 has a single heterozygous A→C SNP at position No. 2 of region No. 1.

[00469] The probability P(r _i |H _k ) depends on various factors, including the quality of the base and other parameters of the HMM. It can be assumed that only base determination errors are present and all base determination errors are equally probable, therefore P(r _i |H _k ) = (1 - p _e ) ^Np(i)-Ne(i) (p _e /3) ^{Ne( i)} , where p _e is the probability of error in determining the base, N _p (i) is the number of active positions of the base(s) covered by read i, and N _e (i) is the number of errors for read i assuming the haplotype H _k . Accordingly, we can assume that p _e = 0.01, which corresponds to a base quality of 20 on the phred scale. In the table shown in FIG. 26 shows P(r _i |H _k ) for all pairs of reads and all candidate haplotypes. The two rightmost columns show P(r _i |G _m1 ) and P(r _i |G _m2 ) with the product at the bottom. In FIG. Figure 26 shows that P(R|G _m1 ) = 3.5 ^-30 and P(R|G _m2 ) = 2.2 ^-15 , that is, a difference of 15 orders of magnitude in favor of G _m2 .

[00470] The posterior probabilities P(G _m |R) depend on the prior probabilities P(G _m ). To complete this example, we can assume a simple model of independently identically distributed (IID) random variables such that the prior probability of a candidate solution with Nv options is (1 - p _v ) ^N·Np-Nv (p _v /9) ^Nv , where N _p is the number of active positions (3 in this case), and Pv is the probability of the option, in this example assumed to be 0.01. This gives P(G _m ) = 7.22e-13, and P(G _m2 ) = 0.500. It should be noted that G _m2 is heterozygous for all region No. 1, and all heterozygous pairs of haplotypes have a mirror image with the same probability (obtained by simply swapping phases). In this case, the sum of the probabilities for G _m2 and its mirror image is 1.000. Having calculated the probabilities of individual variants, one can see a heterozygous SNP A→C in position 2 of region No. 1 with a quality score of 50.4 on the phred scale.

[00471] Accordingly, it can be seen that the computational complexity of the brute-force detection operation is enormous, and this complexity can be reduced by performing joint detection across multiple domains, as described herein. For example, the computational complexity of the above calculations increases rapidly with the number of regions N and the number K of candidate haplotypes. To consider all candidate haplotype combinations, the number of candidate solutions for which probabilities need to be calculated is M = K ^2N . In the enumeration implementation, the number of candidate haplotypes is K = 2 ^Np , where _Np is the number of active positions (for example, as explained above, if graph assembly methods are used to generate a list of candidate haplotypes, then Np is the number of independent bubbles in the graph). Therefore, brute-force computation may be prohibitively expensive to implement. For example, if N = 3 and N _p =10, the number of candidate solutions is M = 2 ^3⋅2⋅10 = 2 ⁶⁰ = 10 ¹⁸ . Therefore, in practice, values of N _p much higher than this are not accepted.

[00472] Therefore, since Bayesian brute-force computation can be prohibitively complex, further methods for reducing the complexity of such computations are described below. For example, in the first stage of another implementation starting with a small number of positions (or even one position = 1), Bayesian computation can be performed on these positions. At the end of the computation, candidate variants whose probability falls below a given threshold can be removed, for example, using the tree pruning function as described above. In this case, the threshold may be adaptive.

[00473] Next, at the second stage, the quantity position can be increased by a small number ΔN _p (for example, like this: = + ΔN _p ), and the surviving candidates can be combined with one or more, for example all, possible candidates for new positions, for example in the tree growing function. The steps of (1) performing Bayesian computation, (2) pruning the tree, and (3) growing the tree can then be repeated, for example, sequentially, until the stopping criteria are satisfied. The history of the threshold can then be used to determine the confidence of the result (for example, the probability that a true solution was found or not found). This process is illustrated by the flow chart in FIG. 27.

[00474] It should be understood that there are many possible variations on this approach. For example, as noted, the cutoff threshold may be adaptive, eg based on the number of surviving candidates. For example, in a simple implementation, the threshold may be set to keep the number of candidates below a fixed number, while in a more complex implementation, the threshold may be set based on a cost-benefit analysis of including additional candidates. In addition, stopping criteria may be that the result is found with a sufficient level of confidence, or that the confidence in the starting position has stopped increasing when new positions are added. Moreover, a more complex implementation may perform a cost-benefit analysis of continuing to add more items. In addition, as shown in FIG. 27, the order in which new positions are added may depend on several criteria, such as the distance to the starting positions, or how highly connected these positions are to already included positions (eg, the amount of overlap with paired reads).

[00475] A useful feature of this algorithm is that the probability that a true solution was not found can be quantified. For example, a useful estimate is obtained by simply summing the probabilities of all pruned branches at each stage: P _pruned = P _pruned + . This assessment is useful for calculating the confidence of the resulting variant definitions: = / . Good confidence estimates are essential to creating good receiver operating characteristic (ROC) curves. This is the main advantage of this pruning method over other complexity reduction methods suitable for this case.

[00476] Returning to the cluster example in FIG. 25 and, starting from the leftmost position (position #1), we will move to the right one base position at a time, using a phred score of 60 as the cutoff threshold at each iteration. Let be { , m=1…M _j } represents candidate solutions at the jth iteration. In FIG. Figure 28 shows the candidate solutions in the first iteration, representing all combinations of bases C and G, listed in descending order of probability. For any solution with equivalent mirror-image representations (obtained by exchanging phase positions), only one representation is shown here. For all candidate solutions, probabilities can be calculated, and those probabilities that are below a cutoff threshold (indicated by the solid line in FIG. 28) can be discarded. As shown in FIG. 28, six candidates will survive as a result of the circumcision methods described herein.

[00477] Next, as shown in FIG. 29, the tree can be grown by finding all combinations of candidates surviving at iteration #1 and candidate bases (C and A) at position #2. A partial list of new candidates is shown in FIG. 29 again in descending order of probability. Again the probabilities can be calculated and compared to the cutoff threshold, and in this case 5 candidates will survive.

[00478] Finally, all combinations of candidates surviving at iteration #2 and candidate bases at position #3 (A and T) can be determined. The final candidates and their associated probabilities are shown in FIG. 30. Accordingly, when calculating the probabilities of individual variants, the result will be a heterozygous SNP A→C in position No. 2 of region No. 1 with a quality score of 50.4 on the phred scale, which coincides with the result obtained using brute force calculation. In this example, trimming did not have a significant effect on the final result, but in general, trimming can affect the calculation, often leading to a more reliable estimate.

[00479] There are many possible implementations of this approach that can affect system performance and complexity, and different options may be suitable for different scenarios. For example, there may be differences in deciding which areas to include. For example, the variant determiner may be configured to determine, before performing joint detection across multiple regions, whether a given hot region should be processed separately or in conjunction with other regions, and if jointly, may then determine which regions to include. In other cases, some implementations may rely on the list of secondary alignments provided by the matcher for information or otherwise making a decision. Other implementations may use a database of homologous regions computed offline, for example based on a reference genome search.

[00480] Accordingly, a useful step in such operations is to decide which items to include. For example, it should be noted that the various regions of interest may not be self-contained and/or isolated from adjacent regions. Therefore, the information in the cluster can influence the probability of separated bases much more than the overall read length (e.g., pairwise read length or long molecule length). Therefore, decisions must be made as to which positions to include in the MRJD calculation, and the number of positions is not limited (even if trimmed). For example, some implementations may process overlapping blocks of positions and update results for a subset of positions based on the level of confidence in those positions or the completeness of the supporting data in those positions (for example, positions near the middle of a block tend to have more complete supporting data than those that are located near the edge).

[00481] Another determining factor may be the order in which new items may be added. For example, in the case of a trimmed MRJD, the order in which new positions are added may affect performance. For example, in some implementations, new positions may be added based on the distance to already included positions or the degree of connectivity to those positions (eg, the number of reads that overlap both positions). Additionally, there are also many variations in how circumcision can be performed. In the example above, the cutoff was based on a fixed probability threshold, but typically the cutoff threshold can be adaptive or based on the number of surviving candidates. For example, in a simple implementation, the threshold may be set to keep the number of candidates below a fixed number, while in a more complex implementation, the threshold may be set based on a cost-benefit analysis of including additional candidates.

[00482] In various implementations, pruning may be performed based on probabilities P(R|G _m ) instead of prior probabilities P(G _m |R). The advantage here is the ability to eliminate equivalent mirrored representations across all regions (in addition to phases). This advantage is at least partially offset by the disadvantage that candidates with very low priors are not pruned, which can be useful in various cases. Therefore, the useful solution may depend on the scenario. If the pruning is performed based on P(R|G _m ), for example, then a Bayesian computation will be performed after the final iteration.

[00483] Additionally, in the example above, the process stopped after processing all base positions in the cluster shown, but other stopping criteria are possible. For example, if the search for a solution is carried out only for a subset of the base positions (for example, when processing overlapping blocks), the process can be stopped when a result for this subset is found with a sufficient level of confidence, or when the confidence stops increasing as more positions are added. However, more sophisticated implementations may perform some sort of cost-benefit analysis, assigning a weight to the cost of the computation depending on the potential value of adding additional items.

[00484] Perhaps prior probabilities will also be useful. For example, the examples above used a simple IID model, but other models can be used. For example, it should be noted that variant clusters are more common than would be predicted by the IID model. It should also be noted that variants are more likely to occur in positions where the references differ from each other. Therefore, incorporating such knowledge into the prior probabilities P(G _m ) can improve detection performance and produce better ROC curves. In particular, it should be noted that there is no clear understanding among genomics experts regarding prior probabilities for homology regions. Therefore, in some implementations it is possible to update prior models as better information becomes available. This may happen automatically as more results are obtained. Such updates may be based on other biological samples or other genomic regions of the same sample, the study of which can be applied to the methods set forth herein to further advance faster and more accurate analysis.

[00485] Accordingly, in some cases an iterative MRJD process may be implemented. Namely, the technique described herein can be extended to allow messages to be passed between related domains to further reduce complexity and/or improve the detection performance of the system. For example, the results of a computation at one location can be used as an input prior probability for a computation at a nearby location. Additionally, some implementations may use a combination of pruning and iteration to achieve the desired trade-off between performance and complexity.

[00486] It is also possible to implement sample preparation to optimize the MRJD process. For example, in the case of paired-end sequencing, it may be useful to have a rigid insertion size distribution when using traditional detection. However, in various cases, introducing variation in the insertion size could significantly improve the performance of MRJDs. For example, a sample may be prepared to intentionally introduce a bimodal distribution, a multimodal distribution, or a bell curve distribution with higher variation than is typically implemented for traditional detection.

[00487] In FIG. Figure 31 shows ROC curves for MRJD and the conventional detector for the human sample NA12878 in selected regions of the genome with one homologous copy, so N = 2, with varying degrees of similarity to the reference sequence. For this data set, paired-end sequencing was used with a read length of 101 and an average insertion size of approximately 400. As shown in FIG. 31, MRJD provides dramatically improved sensitivity and specificity in these areas compared to traditional detection methods. In FIG. Figure 32 shows the same results as a function of sequence similarity measured over a 1000-base window (e.g., if the references differ by 10 bases out of 1000, then the similarity is 99.0 percent). It can be observed that for this dataset, traditional detection starts to perform poorly at sequence similarity of ~0.98, while MRJD performs quite well up to 0.995 and even higher.

[00488] Additionally, in various cases, this technique can be extended to allow messages to be passed between related areas to further reduce complexity and/or improve detection performance. For example, the results of a computation at one location may be used as an input prior to a computation at an adjacent location, and some implementations may use a combination of pruning and iteration to achieve the desired tradeoff between performance and complexity. In specific cases, as noted above, before performing joint detection across multiple regions, the variant determiner may determine whether a given active region should be processed separately or in conjunction with other regions. Additionally, as noted above, some implementations may rely on the list of secondary alignments provided by the matcher to make this decision. Other implementations may use a database of homologous regions computed offline based on a reference genome search.

[00489] In view of the above, it is possible to implement a partially defined hidden Markov model (PD-HMM) in such a way as to take advantage of the MRJD. For example, MRJD can separately estimate the probability of observing part or all of the reads for a given possible joint diplotype that contains one haplotype per ploidy in each homologous reference region, e.g., in the case of two homologous regions in diploid chromosomes, each joint diplotype will contain four haplotypes. In such cases, all or part of the possible haplotypes can be considered, for example by construction, for example by changing each reference region using each possible subset from all variants for which there is non-trivial supporting evidence. However, in the case of long homologous reference regions, the number of possible variants is large, so the number of haplotypes (combinations of variants) grows exponentially, and the number of joint diplotypes (combinations of haplotypes) can become astronomical.

[00490] Therefore, to keep MRJD calculations manageable, it may not be practical to test all possible joint diplotypes. Rather, in some cases the system can be designed to test only small subsets of the “most likely” joint diplotypes. These "most likely" joint diplotypes can be determined by incrementally constructing a tree of partially identified joint diplotypes. In such cases, each node of the tree may be a partially defined diplotype, which contains a partially defined haplotype for each ploidy of each homologous reference region. In this case, a partially defined haplotype may contain a reference region modified by a partially defined subset of possible variants. Accordingly, the partially defined subset of possible options may, for each possible option, comprise an indication of one of three states: the option is defined and present, or the option is defined and absent, or the option is not yet defined, for example, it may be present or absent. At the root of the tree, all variants are not defined in all haplotypes; tree nodes that branch successively further away from the root have successively more variants identified as being present or absent in each haplotype of each joint diplotype of the node.

[00491] Additionally, in the context of a given co-diplotype tree as described above, the amount of MRJD computation is kept limited and manageable by pruning branches of the tree in which all co-diplotype nodes are unlikely, e.g., moderate to extreme, compared to other more likely branches or nodes. Accordingly, such pruning can be performed on branches or nodes that are still only partially determined, for example, several or many variants have not yet been determined to be present or absent in the haplotypes of the joint diplotype of the pruned node. Therefore, in such a case, it is useful to be able to estimate or associate the likelihood of each observed read under the assumption of the truth of a partially defined haplotype. Computing a modified pairwise hidden Markov model (pHMM), denoted "PD-HMM" in the case of a "partially defined pairwise hidden Markov model", is useful for estimating the probability P(R|H) of observing read R, assuming that the true haplotype H* is consistent with partially defined haplotype H. In this context, "consistent" means that some particular true haplotype H* is consistent with the partially defined haplotype H for all variants whose presence or absence is defined in H, but for variants not defined in H, H * may be consistent with the reference sequence, either modified or unmodified by each unspecified variant.

[00492] Note that simply performing a conventional pHMM calculation is generally not sufficient to ensure that some selected H subhaplotype covers only certain variant positions. In general, it is important to construct a tree of co-diplotypes with uncertain variants resolved in an efficient order, which is usually different from their geometric order, so that a partially defined haplotype H will tend to have many uncertain variant positions interspersed with certain ones. To properly address indel-related PCR errors, it is useful to use a pHMM-like calculation that covers all defined variants and a significant radius around them, which may be inconsistent with attempts to avoid uncertain variant positions.

[00493] Accordingly, PD-HMM input data may include the variant-determined nucleotide sequence of the R read, base quality scores (eg, phred scale) of specific nucleotides of the R read, the initial H0 haplotype, and a list of undefined variants (revisions) from H0. Undefined variants may include single base substitutions (SBS), multiple base substitutions (MBS), insertions, and deletions. The advantage is that this may be sufficient to support undefined SNPs and deletions. Undefined SNPs may be incompletely but sufficiently represented as multiple independent SNPs. An undefined insertion may be represented by the first edition of the insertion in the original haplotype followed by an undefined deletion that abolishes the insertion.

[00494] Restrictions can be placed on undefined deletions to facilitate the implementation of a hard-wired engine with limited memory and logic, such as preventing two undefined deletions from overlapping (removing the same bases of the original haplotype). If there is a need to test a partially defined haplotype with undefined variants that violate such constraints, this can be addressed by converting one or more undefined variants into defined variants through a larger number of PD-HMM operations covering cases with and without those variants. For example, if two undefined deletions A and B violate the constraint due to overlap with each other in the original H0 haplotype, then deletion B can be edited into H0 to produce H0B, and two PD-HMM operations using only the undefined deletion A, one for the original haplotype H0, and the other for the original haplotype H0B, and the resulting maximum probability from these two PD-HMM operations can be retained.

[00495] The result of the PD-HMM operation may be an estimate of the maximum P(R|H*) among all H* haplotypes that can be formed by editing H0 using only any subset of undefined variants. The maximization can be performed locally and contribute to pHMM-like dynamic programming in a given cell as if an adjacent uncertain variant were present or absent in the haplotype depending on which score is higher, for example contributing to a higher partial probability. Such maximization during dynamic programming may result in higher estimates of the P(R|H*) maximum than true maximization of individual pure H* haplotypes, but the difference is usually negligible.

[00496] Undefined SNPs can be included in the PD-HMM by allowing one or more matching nucleotide values to be specified for each haplotype position. For example, if base 30 of haplotype H0 is "C" and an undefined SNP replaces that base "C" with "T", then the haplotype in the PD-HMM operation may specify position 30 as both bases "C" and "T". In normal dynamic programming of pHMM, any transition to the “M” state results in the path probability being multiplied by the probability of a correct base determination (if the haplotype position matches the read position) or by the probability of a certain base determination error (if the haplotype position does not coincide with the read position); in the case of PD-HMM, this is modified by using the probability of a correct assignment if the read position matches any of the possible haplotype positions (e.g., “C” or “T”), and the probability of a base assignment error otherwise.

[00497] Uncertain haplotype deletions can be included in the PD-HMM by flagging optionally deleted haplotype positions and changing the dynamic programming of the pHMM to allow alignment paths to bypass laterally the haplotype segments with an undefined deletion without losing probability. This can be done in a variety of ways, but the general rule is that the probabilities of states M, I and/or D can move horizontally (along the haplotype axis) across the entire span of an unspecified deletion without reducing the normal probabilities of gap opening and gap extension .

[00498] In one particular embodiment, positions where undefined deletions begin are marked with a flag "F1", and positions where undefined deletions end are marked with a flag "F2". In addition to the M, I, and D “states” (partial probability representations) for each cell of the HMM matrix (horizontal haplotype/vertical read), each PD-HMM cell may additionally contain the “bypass” states BM, BI, and BD. In the haplotype columns flagged with the F1 flag, the states BM, BI and BD take values copied from the states M, I and D of the cell on the left, respectively. In non-F2-flagged haplotype columns, particularly in columns starting from the F1-flagged column and continuing into the undefined deletion, the BM, BI, and BD states transfer their values to the BM, BI, and BD states of the cell on the right, respectively. In haplotype columns flagged with the F2 flag, the maximum M and BM, maximum I and BI, and maximum D and BD are used instead of the states M, I, and D used to calculate the states of adjacent cells, respectively. This is shown as an example in column F2 as a multiplexed selection of signals from registers M and BM, I and BI, D and BD.

[00499] Note that although the status registers BM, BI and DB may be shown in columns F1 through F2, and the maximizing multiplexers M/BM, I/BI and D/BD may be shown in column F2, these components may be present for calculations of all cells, allowing the processing of undefined deletions at any position and allowing multiple undefined deletions with corresponding F1 and F2 flags throughout the haplotype. Note also that flags F1 and F2 can be in the same column when the undefined deletion consists of a single base. It should also be noted that the matrix of PD-HMM cells can be depicted as a schematic representation of logical calculations of the states M, I, D, BM, BI and BD, but in the hardware implementation there may be a smaller number of cells that calculate logical elements, and organized in a pipeline appropriately to compute the values of the M, D, I, BM, BI and BD states at high clock rates, and the matrix cells can be computed with varying degrees of hardware parallelization in different orders according to the inherent logical dependencies of the PD-HMM computation.

[00500] Thus, in this embodiment, the pHMM state values in one column can be immediately to the left of the deletion, and they can be captured and passed right, without modification, to the rightmost column of this intermediate deletion, where they will be substituted into the pHMM calculations whenever they exceed the normal path estimates. When these maximum values are selected, the values of the bypass states BM, BI, and BD represent the results of local dynamic programming assuming the presence of a deletion, while the values of the normal states M, and D represent the results of local dynamic programming assuming the absence of an indeterminate deletion.

[00501] In another implementation, a single bypass state may be used, for example, the BM state taking the value from the M state of the column flagged with the F1 flag, or taking the sum of the M, D and/or I states. In another implementation, instead of using the states " bypass" remove penalties for opening a gap/extending a gap in columns of undefined deletions. In another implementation, bypass states contribute additively to dynamic programming to the right of undefined deletions, rather than being used for local maximization. In yet another embodiment, more or less, or differently defined, or differently positioned haplotype position flags are used to initiate bypass or similar actions, for example, one flag indicating membership in an unspecified deletion. In an additional embodiment, two or more overlapping undefined deletions may be involved, for example, using additional flags and/or bypass states. In addition, undefined insertions in the haplotype are supported instead of or in addition to undefined deletions. Similarly, undefined insertions and/or deletions on the read axis are supported instead of or in addition to undefined deletions and/or insertions on the haplotype axis. In another embodiment, indefinite substitutions of multiple bases are supported as indivisible variants (all present or all absent). In yet another embodiment, variable length unspecified substitutions are supported as indivisible variants. In another implementation, uncertain options are penalized using fixed or configurable probability or scoring adjustments.

[00502] This PD-HMM calculation may be implemented as a hardware engine, such as in FPGA or ASIC technology, by extending the hardware engine architecture for a “conventional” pHMM calculation, or may be implemented using one or more quantum circuits on a quantum computing machine. platform. In addition to the engine pipeline logic for computing, transmitting, and storing M, I, and D state values for different or sequential cells, parallel logic can be created to calculate, transmit, and storing BM, BI, and BD state values, as described above herein. The memory resources and ports for storing and retrieving state values M, I and D may be complemented by similar or wider or deeper memory resources and ports for storing and retrieving state values BM, BI and BD. Flags such as F1 and F2 can be stored in memories along with associated haplotype bases.

[00503] Multiple matching nucleotides for, for example, positions of an undefined haplotype SNP may be encoded such that a vector containing one bit for each possible nucleotide value is used. The computation dependencies of the cells in the pHMM matrix remain unchanged in the PD-HMM, so the order and organization of the computation pipeline of multiple cells can remain the same for the PD-HMM. However, the delay in time and/or clock cycles for a complete cell computation increases slightly in the case of PD-HMM, since it is necessary to compare the values of the “normal” and “bypass” states and select those that are larger. Accordingly, it is possible that it would be advantageous to include one or more additional pipeline stages for computing PD-HMM cells, which would introduce additional latency in clock cycles. Additionally, another benefit could be to extend each "swap" of cells calculated with one or more rows to keep the pipeline full longer without dependency issues.

[00504] This PD-HMM calculation has twice the number of state values (BM, BI, and BD in addition to M, I, and D) than a conventional pHMMB calculation, and may require approximately twice the hardware resources to implement the engine with equivalent throughput. However, the PD-HMM engine has the advantages of exponential speed and efficiency in terms of increasing uncertain variants compared to the conventional pHMM engine, making one run for each haplotype representing a different combination of uncertain variants present or absent. For example, if a partially defined haplotype has 30 undefined variants, each of which may be independently present or absent, then there are 2^30, or over 1 billion, different specific haplotypes that would otherwise have to be handled by pHMM.

[00505] Accordingly, these and other similar operations described herein can be performed in such a way as to better understand and more accurately predict what has happened to a subject's genome that the reads have changed relative to the reference. For example, even though mutations may occur randomly, there are cases in which the likelihood of their occurrence seems potentially predictable to some extent. In particular, in some cases where mutations occur, they may occur in certain known locations and in certain forms. More specifically, mutations, if they occur, will occur on one or the other allele, or both, and usually occur more often in some places than others, such as at the ends of chromosomes. Therefore, this and other related information can be used to develop mutation models that can be generated and used to better estimate the likely presence of a mutation in one or more regions of the genome. For example, by taking into account various a priori knowledge, such as one or more mutation patterns, when performing analyzes of genomic variations, better and more accurate genomic analysis results can be achieved, for example, with more accurate delineations of genetic mutations.

[00506] Such mutation models may allow consideration of the frequency and/or location of various known mutations and/or mutations that appear to occur in combination with each other or in an otherwise non-random manner. For example, it has been established that variations occur predominantly closer to the ends of a given chromosome. Thus, known mutation patterns can be generated, stored in the database described herein, and used by the system to better predict the presence of one or more variations in the genomic data being analyzed. In addition, a machine learning process can also be implemented, as described in more detail later in this document, so that the various outcome data generated by the analyzes performed here can be analyzed and used to better inform the system when to make a particular determination. options, for example, in accordance with the machine learning principles described in this document. Namely, machine learning can be implemented on aggregate data sets, especially for specific variations, and this learning can be used to better form more comprehensive mutation models, which in turn can be used to make more accurate variation determinations.

[00507] Therefore, the system can be configured to look at various variation data, check the data for various correlations and, if a correlation is found, such information can be used to better weigh and therefore more accurately determine the presence of other variations in other samples genome, for example on a regular basis. Accordingly, in a similar manner, the system, especially the variant identification engine, can be continuously updated with the learned variant correlation data to make progress in better variant identification to obtain better and more accurate outcome data.

[00508] Namely, telemetry can be used to update a growing mutation model to achieve improved analysis in the system. This can be particularly useful when analyzing samples that are related to each other in some way, such as from the same geographic population, and/or can be used to determine which reference genome from a set of reference genomes may be the best reference genome for analysis of a specific sample using it. Additionally, in various cases, mutation model and/or telemetry can be used to better select a reference genome for use in system processes and thereby improve the accuracy and efficiency of system outputs. In particular, when multiple reference genomes may be used in one or more of the assays described herein, the selection for use may be given preference to a particular reference genome, for example, through the use of a mutation model in selecting the most appropriate reference genome for use.

[00509] It should be noted that when performing secondary analysis, the underlying structure for each genomic region being mapped or aligned may contain one or more underlying genes. Accordingly, in various cases, this understanding of the underlying genes/or the functions of the proteins they encode can provide useful information when performing secondary analyses. In particular, tertiary metrics and/or results can be useful in secondary analysis protocols performed by the presented system, for example, in the process of biological context-sensitive mutation model. More specifically, since DNA encodes genes and genes encode proteins, information about such proteins that lead to mutations and/or odious functions can be used to inform mutation models used when performing secondary and/or tertiary analyzes on a subject's genome.

[00510] For example, a tertiary analysis, such as a set of samples of genes encoding mutated proteins, can be informative when performing a secondary analysis of genomic regions known to encode such mutations. Therefore, as stated above, the various tertiary processing results can be used to inform and/or update the mutation models described herein to achieve higher accuracy and efficiency when performing the various secondary analysis operations described herein. Namely, information about mutated proteins, such as contextual tertiary analysis, can be used to update the mutation model by performing secondary analysis on regions known to encode proteins and/or potentially harbor such mutations.

[00511] Accordingly, in view of the foregoing, for embodiments including FPGA accelerated mapping, alignment, sorting, and/or variant determination applications, one or more of these functions may be implemented in one or both of software or hardware ( AO) processing components, for example, in software running on traditional CPUs, GPUs, CPUs, and/or in firmware that can be embedded in FPGAs, ASICs, sASICs, etc. In such cases, the CPU and FPGA must be configured to communicate to transfer results from one stage in one device, such as a CPU or FPGA, for processing in the next stage in another device. For example, when performing a mapping function, constructing large data structures such as a reference index can be implemented using the CPU, and executing a hash function on them can be implemented using an FPGA. In such a case, the CPU can build data structures, store them in associated memory such as DRAM, which memory can then be accessed by processing engines running on the FPGA.

[00512] For example, in some embodiments, communications between the CPU and the FPGA may be implemented using any suitable interconnect, such as a peripheral bus such as a PCIe bus, USB, or a network interface such as Ethernet. However, the PCIe bus can provide relatively poor integration between the CPU and FPGA, which means that transfer latencies between them can be quite high. Accordingly, although one device (e.g., CPU or FPGA) can access memory attached to another device (e.g., through a DMA transfer), the recalled memory regions are non-cacheable because they have no means to maintain cache coherence between the two devices. As a result, transfers between the CPU and FPGA are limited to execution between large stages of high-level processing, and large numbers of input and output jobs must be queued between devices to avoid slowing each other down while waiting for high-latency operations. This slows down various processing operations described in this document. Additionally, when an FPGA accesses uncacheable CPU memory, the entire load of that memory access falls on the CPU's external memory interfaces, which are limited in bandwidth compared to their internal cache interfaces.

[00513] Accordingly, due to such loose CPU/FPGA integrations, it is typically necessary to have “centralized” software control of the FPGA interface. In such cases, different software threads may process different units of data, but when those threads generate work for execution by the FPGA engine, that work must be aggregated into "central" buffers, such as using a single aggregator software thread or using multiple aggregation access threads via semaphores, with control of the transfer of aggregated work via DMA packets being handled by a central software module such as a kernel space driver. Therefore, as the hardware engines produce results, the reverse process occurs, with the software driver receiving DMA packets from the hardware and the thread deaggregator distributing the results to various waiting software worker threads. However, this centralized software control of communication with the FPGA hardware logic is cumbersome and resource-intensive, reduces the efficiency of software threading and communication between hardware and software, limits the practical bandwidth of communication between hardware and software, and dramatically increases its latency.

[00514] Additionally, as shown in FIG. 33A, loose integration between CPU 1000 and FPGA 7 may require each device to have its own dedicated external memory, such as DRAM 1014, 14. As shown in FIG. 33A, CPU 1000 has its own DRAM 1014 on the system motherboard, such as DDR3 or DDR4 DIMMs, while FPGA 7 has its own DRAM 14, such as 8 GB SODIMM, which can be directly coupled to FPGA 7 via one or more 6 buses for DDR3, such as the high latency PCIe bus. Likewise, the CPU 1000 may be communicatively coupled to its own DRAM 1014, for example via a suitably configured bus 1006. As discussed above, the FPGA 7 may be configured to contain one or more processing engines 13, which processing engines may be configured to implement one or more functions in a bioinformatics pipeline as described herein, for example, when the FPGA 7 includes a mapping engine 13a, an alignment engine 13b, and a variant detection engine 13c. Other engines described herein may also be included. In various embodiments, one or both of the CPU and the FPGA may be configured to contain a cache memory 1014a, 14a, respectively, that is configured to store data, such as resultant data, that is passed thereto by one or more different system components, such as one or more memories and/or one or more processing engines.

[00515] Many of the operations described herein to be performed by the FPGA 7 for genomic processing require access to large memory to perform the underlying operations. Namely, due to the use of large data units, such as reference genomes of over 3 billion nucleotides, over 100 billion nucleotides of raw sequencer data, etc., the FPGA 7 may need to access the main memory 1014 multiple times, for example, to access an index, such as a 30 GB hash table, or other reference genome index, e.g. for the purpose of mapping seeds from the sequenced DNA/RNA of interest to a 3 billion base pair reference genome and/or sampling candidate segments, e.g. from the reference genome, for alignment on them.

[00516] Accordingly, in various implementations of the system described herein, multiple fast accesses to main memory by one or more hard-wired engines 13 may be necessary, for example, when performing mapping, alignment, and/or variant determination operations. However, for the FPGA 7 to make such a large number of small random memory accesses via the peripheral bus 3 or other network link to the memory 1014 attached to the main CPU 1000 may be prohibitively impractical. For example, in such cases, return data latencies may be very large, bus efficiency may be very low, such as for such small random accesses, and the load on the CPU's external memory interface 1006 may be prohibitively large.

[00517] Additionally, the result is that each device needs its own external memory, forcing the overall form factor of a full 1000 CPU + FPGA 7 platform to be larger than would be desirable, for example, for some applications. In such cases, in addition to a standard system motherboard for one or more 1000 CPUs and 7 supporting chips and 1014 and/or 14 memories, board space is required for a large FPGA package (which may be even larger due to the need for sufficient pins for several external memory buses) and several memory modules 1014, 14. However, standard motherboards do not contain these components, and it is difficult to find free space on them, so a practical implementation could be made with the possibility of using an expansion board 2 containing an FPGA 7, its memory 14 and other supporting components, such as a power supply, which, for example, is connected to the PCIe expansion slot on the CPU motherboard. To have space for expansion card 2, the system can be implemented in a fairly large enclosure, such as a 1U or 2U server or a larger rack-mounted server.

[00518] In view of the above, in various cases, as shown in FIG. 33B, to overcome these factors, it may be necessary to configure the CPU 1000 in a hard-coupled arrangement with the FPGA 7. Specifically, in various cases, the FPGA 7 may be hard-coupled to the CPU 1000, for example, using a low-latency interconnect 3 such as fast access interconnect (QPI). Namely, to enable tighter CPU+FPGA integration, the two devices can be connected using any suitable low latency interface such as "inter-processor interconnect" etc., such as INTELS® Quick Path Interconnect (QPI) or HyperTransport ( HT).

[00519] Accordingly, as shown in FIG. 33B, system 1 proposed herein includes a CPU 1000 and a processor such as an FPGA 7, both devices coupled to one or more memory modules. For example, CPU 1000 may be coupled, for example, via a suitably configured bus 1006, to DRAM 1014, and likewise, FPGA 7 is communicatively coupled to associated memory 14 via DDR3 bus 6. However, in this case, instead of being connected to each other, for example, through a typical low latency interconnect, such as a PCIe interface, the CPU 1000 is connected to the FPGA 7 using a HyperTransport interconnect 3, such as QPI. In such a case, due to the low latency inherent in such interconnects, the associated memories 1014, 14 of the CPU 1000 and FPGA 7 are easily accessible to each other. Moreover, in various cases, due to this tightly coupled configuration, one or more caches 1114a/14a associated with the devices may be configured to maintain coherence with each other.

[00520] Some of the key properties of such a tightly coupled CPU/FPGA interconnect include high bandwidth, such as 12.8 GB/s; low latency, for example, 100-300 ns; an adapted protocol configured to provide efficient remote memory accesses and efficient small memory transfers, for example, on the order of 64 bytes or less; and integration of supported protocol and CPU for cache access and cache coherence. In such cases, a natural interconnect for exploiting such tight integration with a given CPU 1000 may be its own inter-processor interconnect 1003, which can be used here to enable multiple cores and multiple CPUs to operate in parallel in shared memory space 1014, thereby allowing access to each other's cache stacks and external memory while maintaining cache coherence.

[00521] Accordingly, as shown in FIG. 34A and 34B, a card 2 may be provided, for example where the card may be configured to receive one or more CPUs 1000, for example, via a plurality of interconnects 1003, such as native inter-processor connections 1003a and 1003b. However, in this case, as shown in FIG. 34A, the CPU 1000 is connected to the interconnect 1003a, but instead of connecting another CPU to it via the interconnect 1003b, the FPGA 7 of the present invention is connected to it. In addition, the system 1 is configured such that the CPU 1000 can be coupled to a coupled FPGA 7, for example, via a low latency hard coupled interconnect 3. In such cases, each memory 1014, 14 associated with a corresponding device 1000, 7 may be configured to access each other, for example, with wide bandwidth and maintaining cache coherence.

[00522] Likewise, as shown in FIG. 34B, the system may also be configured to receive packages 1002a and/or 1002b, for example where the packages contain one or more CPUs 1000a, 1000b that are tightly coupled, for example via low latency interconnects 3a and 3b, to one or more FPGAs 7a, 7b, for example, when, given the system architecture, each housing 2a and 2b can be connected to one another, for example by means of a hardwired interconnect 3. In addition, as shown in FIG. 35, in various cases, a package 1002a may be provided, wherein the package 1002a contains a CPU 1000 configured to be tightly coupled to an integrated circuit such as an FPGA 7. In such a case, by tightly coupling the CPU 1000 to the FPGA 7, a system can be built to directly sharing the cache 1014a to ensure consistency, coherence and easy accessibility for each of the two devices, for example, with respect to data stored there.

[00523] Therefore, in such cases, FPGA 7 and one of the packages 2a/2b can essentially impersonate another CPU and thereby operate in a shared memory cache coherence environment with one or more CPUs just like this would make multiple CPUs on a multi-slot motherboard 1002, or multiple CPU cores on a multi-core CPU. With this FPGA/CPU interconnect, FPGA 7 can effectively share CPU memory 1014 rather than having its own dedicated external memory 14 that may or may not be enabled or accessed. Therefore, in such a configuration, fast, short, random accesses are effectively supported by interconnect 3, for example, with low latency. This makes it practical and efficient for various processing engines 13 in the FPGA 7 to access large data structures in the memory of the CPU 1000.

[00524] For example, as shown in FIG. 37, a system is provided for performing one or more of the methods described herein, for example where the method includes one or more steps for performing functions of the present invention, such as one or more of mapping and/or alignment functions and/ or defining options that are described herein in a standard manner. In particular, in step (1), a data structure may be generated or otherwise provided, for example, by the NAS and/or CPU 1000, after which the data structure may be stored (2) in associated memory, such as DRAM 1014. Structure data may be any data structure, such as those described herein, but in this case may be a set of reads of sequenced data, and/or a reference genome, and/or an index of a reference genome, for example, to perform mapping and/or alignment and/or option identification functions.

[00525] In a second step (2), for example with regard to mapping and/or alignment functions and the like, the FPGA 7 coupled to the CPU 1000, for example via a tightly coupled interface 3, can access the associated memory 1014 CPUs for performing one or more actions on stored sequenced reads, reference genomes, and/or indexes thereof. In particular, in step (3), for example, in a mapping operation, the FPGA 7 may access a data structure, for example, sequenced reads and/or reference sequences, to create one or more seeds from them, for example, when the data structure contains one or more genome reads and/or sequences. In such a case, primers, eg reference and/or read sequences, can be used to perform a hash function on them, eg to create one or more reads that map to one or more positions relative to the reference genome.

[00526] In a further step (3), the resulting mapped result data may be stored, for example, in either the main memory 1014 or the associated DRAM 14. In addition, after the data has been mapped, the FPGA 7, or its processing engine 13, can be reconfigured, for example, partially reconfigured, as an alignment engine, which can then access the stored mapped data structure to perform an alignment function on it to generate one or more reads aligned to the reference genome. In an additional step (4), the main CPU can then access the mapped and/or aligned data to perform one or more functions on it, for example to create a de Bruijn graph (“DBG”), which can then be stored in its associated memory . Likewise, in one or more additional steps, FPGA 7 may again access host CPU memory 1014 to access the DBG and perform HMM analysis thereon to create one or more case definition files.

[00527] In specific cases, the CPU 1000 and FPGA 7 may have one or more cache memories that, by tightly coupling the interface between the two devices, will ensure coherence of the individual caches, for example, with respect to intermediate data, such as result data, stored therein. for example, as a result of the performance of one or more functions in them. Likewise, data can be shared essentially seamlessly between tightly coupled devices, thereby allowing pipeline functions to be intertwined, for example in a bioinformatics pipeline. Therefore, in such a case, the FPGA 7 no longer needs to have its own attached dedicated external memory 14, and therefore, thanks to such a tightly coupled configuration, the stored reads, reference genome and/or reference genome index, which are described herein, can be intensively shared. for example, maintaining cache coherence, such as for read mapping and alignment and other genomic data processing operations.

[00528] Additionally, as shown in FIG. 38, tightly coupled and cache coherent configurations, like other component configurations discussed herein, allow smaller, low-level operations to be performed on one device (e.g., CPU or FPGA) before returning the data structure or processing thread 20 to another device, for example, for further processing. For example, in one case, CPU thread 20a may be configured to queue large quantities of work into FPGA hardware logic 13 for execution, and the same or another thread 20b may be configured to subsequently process a large queue of results so generated, e.g. much later. However, in various cases, it may be more efficient if CPU thread 20, as described herein, blocks a "function call" to the associated FPGA hardware engine 13, where the CPU can be set to resume software execution immediately upon completion. FPGA hardware function. Therefore, instead of packing data structures for streaming using DMA 14 into the FPGA 7 and unpacking the results when they return, the software thread 20 could simply provide a memory pointer to the FPGA engine 13, which could access the shared memory 1014 /14 and make changes in place while maintaining cache coherence.

[00529] In particular, with such interconnections between structures proposed herein, the granularity of software/hardware interactions can be finer-grained to assign smaller low-level operations to be performed by different hardware engines 13, for example by calling functions from various dedicated software threads 20. For example, on a loosely coupled CPU/FPGA platform to efficiently accelerate the mapping, alignment and/or variant determination of a DNA/RNA read, a complete mapping/alignment/variant determination pipeline can be created in the form of one or more software and/or or FPGA-implemented engines, where unmapped and unaligned reads are streamed from software to hardware, where the process can be repeated, for example, to identify variants. As for the configurations described in this document, they can be very fast. However, in various cases, such a system may suffer from limitations in flexibility, complexity and/or programmability, for example due to the fact that the entire mapping/alignment and/or variant detection pipeline is implemented in hardware circuitry, which, although reconfigurable in the FPGA, typically much less flexible and programmable than software, and may therefore be limited by lower algorithmic complexity.

[00530] In contrast, by using a hard CPU/FPGA interconnect, such as a QPI or other interconnect, in the configurations described herein, several resource-intensive discrete operations such as seeding and/or mapping, recovery scanning, alignment without gaps, gap alignment, e.g., Smith-Waterman alignment, etc., may be implemented as various separately available hardware engines 13 (see, e.g., FIG. 38), and general mapping/alignment algorithms and/or case definitions can be implemented in software, with low-level acceleration achieved by calling on the FPGA for special, expensive processing steps. Such an infrastructure provides full software programmability beyond specific acceleration calls, and enables greater algorithmic complexity and flexibility than with standard hard-wired operations.

[00531] Moreover, in such a software execution infrastructure accelerated by low-level FPGA hardware acceleration calls, hardware acceleration functions can be more easily shared for multiple purposes. For example, when hardware engines 13 form large monolithic pipelines, the individual pipeline subcomponents may typically be specifically designed for their environment and interconnected only within a single pipeline, which, if not tightly coupled, may not generally be available for any purpose. But many genomic data processing operations, such as Smith-Waterman alignment, de Bruijn graph or assembly graph construction, and other similar operations, can be used in various top-level parent algorithms. For example, as described herein, a Smith-Waterman alignment may be used in mapping and aligning a DNA/RNA read, for example, to a reference genome, but may also be capable of being used by haplotype-based variant definers to align candidate haplotypes to reference genome either to each other or to sequenced sequences, for example in HMM analysis and/or variant detection functions. Therefore, invoking various discrete low-level hardware acceleration functions through common software function calls may allow the same acceleration logic, e.g., 13, to be leveraged across an entire genomic data processing application, e.g., when performing both alignment and variant detection, e.g. HMM operations.

[00532] In the case of a tight CPU/FPGA interconnect, it is also practical from a practical standpoint to have distributed, rather than centralized, control by the CPU 1000 software over communications links with the various FPGA hardware engines 13 described herein. In widely used multi-threaded, multi-core, and multi-processor software architectures, multiple software threads and processes communicate and interact seamlessly without any central software modules, drivers, or threads to manage internal communications. In this format, it is practical due to the shared cache-coherent memory that is visible to threads on all cores on all CPUs, although physically coherent memory sharing between the cores and the CPU occurs through internal communication via a processor interconnect such as QPI or HT.

[00533] Likewise, as shown in FIG. 36-38, systems proposed herein may have a number of CPUs and/or FPGAs, which may be in a tightly coupled CPU/FPGA interconnect configuration including multiple threads, e.g., 20a, b, c, and multiple processes. running on one or more cores and/or CPUs, such as 1000a, 100b and 1000c. Therefore, the system components are configured to exchange data and interact with distribution among each other, for example, between various different CPUs and/or FPGA hardware acceleration engines, for example, by sharing memory with maintaining cache coherence among different CPUs and FPGAs. For example, as shown in FIG. 36, multiple CPU cores 1000a, 1000b, and 1000c may be linked together to share one or more memories, such as DRAM 1014, and/or one or more caches having one or more layers, such as L1, L2, L3. etc., or levels associated with them. Likewise, as shown in FIG. 38, in another embodiment, a single CPU 1000 may be configured to contain multiple cores 1000a, 1000b, and 1000c that may be linked together so as to share one or more memories, such as DRAM. 1014, and/or one or more caches 1014a having one or more layers or levels associated therewith.

[00534] Therefore, in another embodiment, data to be processed from one or more software threads 20 from one or more CPU cores 1000 to a hardware engine 13, such as from an FPGA, or vice versa, can be continuously and/or seamlessly updated in shared memory 1014 or cache and/or its layer, which is visible to every device. Additionally, requests to process data in shared memory 1014, or notifications of results updated there, may be communicated via signaling between software and/or hardware, e.g., over a suitably configured bus, e.g., a DDR4 bus, e.g., in queues, which can be implemented within the shared memory itself. Standard software control, transmission, and data protection engines, such as semaphores, mutex locks, and non-decomposable integers, can also be implemented for software/hardware coordination.

[00535] Therefore, in some embodiments exemplified in FIG. 36 and which do not require the FPGA 7 to have its own dedicated memory 14 or other external resources due to coherent memory sharing via a tight CPU/FPGA interconnect, it becomes much more practical to package the FPGA 7 in a more compact and native system manner within the confines of traditional CPU motherboards 1000 without the use of expansion cards. See, for example, FIG. 34A and 34B and FIG. 35. There are several packaging options. Namely, FPGA 7 can be installed on a multi-processor motherboard in a CPU socket as shown in FIG. 34A and 34B, for example, using a suitable riser board, such as a small PC board 2 or an alternative wired FPGA die package within the CPU chip package 2a, where the CPU socket pins are suitably connected to the FPGA pins, including power and ground connections, the processor Interconnect 3 (QPI, HT, etc.) and other system connections. Accordingly, the FPGA die and the CPU die can be included in the same multi-chip package (MCP) with the necessary connections, including power, ground, and CPU/FPGA interconnect, created within the package 2a. Inter-chip connections can be made by inter-chip wire bonding, or by bonding to a common substrate or riser board, or by bonded pads or vias across the silicon between stacked dies.

[00536] Additionally, in various implementations, the FPGA and CPU cores may be fabricated on the same chip (see FIG. 35) in a system-on-chip (SOC) manner. In any of these cases, custom logic, for example 17, can be implemented within the FPGA 7, both to communicate across the CPU/FPGA interconnect 3, for example using properly crafted protocols, and to serve, transform and/or route requests to accessing memory from internal FPGA engines 13 via CPU/FPGA interconnect 3 via proper protocols to shared memory 1014a. Additionally, all or some of this logic may be implemented on a custom silicon die to avoid using FPGA logic space for these purposes, such as where the committed logic may reside on a CPU die and/or FPGA die, or on a separate die. Moreover, in any of these cases, the power supply and heat dissipation requirements can be achieved accordingly, for example in a single package (MCP or SOC). Additionally, the FPGA size and number of CPU cores can be selected to remain within safe power envelope limits, and/or dynamic techniques (clock control, clock gating, core shutdown, power islands, etc.) can be used to regulate power consumption energy in accordance with changes in CPU and/or FPGA calculation demand.

[00537] All of these packaging options have a number of common advantages. The tightly integrated CPU/FPGA platform becomes compatible with standard motherboards and/or system chassis sizes. In the case of installing an FPGA via a riser card into a CPU socket (see FIGS. 34A and 34B), at least a two-slot motherboard 1002 can be used. In other cases, a four-slot motherboard can be used to enable 3 CPU + 1 FPGA, 2 CPU + 2 FPGA or 1 CPU + 3 FPGA, etc. If the FPGA is in the same package as the CPU CPU (either MCP or SOC), then a single-socket motherboard can be used, potentially in a very small system case (although a dual-socket motherboard is shown); it is also very well suited for scaling up, such as 4 FPGAs and 4 multi-core CPUs on a 4-socket server motherboard, which can still run in a compact package, such as a 1U rackmount server.

[00538] Accordingly, in various cases there may therefore be a need to install an expansion card to integrate the CPU and FPGA acceleration, since the FPGA 7 can be integrated into the CPU socket 1003. This implementation eliminates the additional space and power requirements for the expansion card, and eliminates the possibility of various additional failures that sometimes occur with expansion cards due to relatively low reliability components. In addition, standard CPU cooling solutions (heat sinks, heat pipes and/or fans) can be applied to FPGAs or CPU/FPGAs in packages or CPU sockets, which are effective even at low cost because they are manufactured in high volumes, while cooling Expansion cards can be expensive and ineffective.

[00539] Similarly, an FPGA/riser card and/or CPU/FPGA in the same package may utilize full power from the CPU socket, e.g., 150 W, while an expansion card may be power limited, e.g., 25 W or 75 W from the bus PCIe. In various use cases for genomic data processing applications, all of these packaging options can facilitate the installation of a tightly coupled CPU+FPGA compute platform, such as inside a DNA sequencer. For example, typical modern "next generation" DNA sequencers contain sequencing hardware (sample and reagent storage, fluid tubing and controls, sensor arrays, image and/or signal processing) in a housing that also contains standard or custom a server motherboard wired to the sequencing device to control sequencing and receive sequencing data. The tightly integrated CPU+FPGA platform described herein can be achieved in such a sequencer, for example, by simply installing one or more FPGA/motherboard and/or FPGA/CPU complexes in a single package into the CPU sockets of its existing motherboard, or, in an alternative implementation, by installing a new motherboard with a CPU and FPGA, for example, tightly coupled, as described herein. Additionally, all of these packaging options can be configured to help facilitate the development of a tightly integrated CPU+FPGA platform, for example, in a cloud-accessible and/or data center-based server rack containing compact/dense servers with very high reliability/ accessibility.

[00540] Therefore, in accordance with the ideas set forth herein, there are multiple processing steps for mapping and alignment, sorting and/or deduplication, to identify variants of DNA (or RNA) sequencing data, which may vary depending on the technologies used primary and/or secondary and/or tertiary processing and their applications. Such processing steps may include one or more of the following: signal processing on electrical measurements from the sequencer, image processing on optical measurements from the sequencer, base determination using processed signal or image data to determine the most likely nucleotide sequence and confidence estimates, filtering sequenced low quality reads or polyclonal clusters, detection and trimming of adapters, key sequences, barcodes and ends of low quality reads, as well as de novo sequence assembly, generation and/or use of de Bruijn graphs and/or sequence graphs, e.g. and sequence graph, editing, trimming, cleaning, coloring, annotation, comparison, transformation, splitting, analysis, subgraph selection, traversal, reverse traversal, search, filtering, import, export, including mapping of reads to the reference genome, alignment of reads to possible mapping locations in the reference genome, local assembly of reads mapped to the reference region, sorting of reads into aligned positions, tagging and/or removal of overlapping reads to match indels, recalibration of base quality assessment, identification of variants (same sample or jointly), analysis of structural variants , analysis of copy number of variants, determination of somatic variants (e.g. tumor sample only, tumor/normal matches or tumor/mismatched normal, etc.), RNA splicing boundary detection, alternative RNA splicing analysis, RNA transcript assembly, RNA transcript expression analysis , RNA differential expression analysis, RNA variant identification, DNA/RNA difference analysis, DNA methylation analysis and determination, variant quality assessment recalibration, variant filtering, variant annotation using known variant databases, sample contamination detection and assessment, phenotype prediction, testing for disease, prediction of response to therapy, development of individualized therapy, pedigree and mutation history analysis, population DNA analysis, identification of genetic markers, encoding genomic data into standard formats and/or compression files (e.g. FASTA, FASTQ, SAM, BAM, VCF, BCF ), decoding genomic data from standard formats, querying, selecting or filtering subsets of genomic data, general compression or decompression of genomic files (gzip, BAM compression), specialized compression and decompression of genomic data (CRAM), encryption and decryption of genomic data, calculation of statistics, comparison and presentation of genomic data, comparison of resulting genomic data, accuracy analysis and reporting, storage, archiving, retrieval, backup, recovery and transfer of genomic file, as well as genomic database construction, query execution, access control, data extraction, etc. P.

[00541] All of these operations can be quite slow and expensive when implemented on traditional computing platforms. The slowness of such software-only operations may be due in part to the complexity of the algorithms, but is generally due to the input and output of very large data sets, resulting in a large latency compared to the movement of the data. The devices or systems described herein overcome these problems, in part, through the configuration of different hardware processing engines, acceleration through different hardware implementations, and/or, in part, through tight CPU/FPGA coupling. Accordingly, as shown in FIG. 39, one or more, for example, all, of these operations may be accelerated through interaction between the CPU 1000 and the FPGA 7, for example, in a distributed processing model as described herein. For example, in some cases (encryption, general compression, mapping, and/or read alignment), the entire operational function may be implemented substantially or entirely in custom FPGA logic (e.g., using a hardware design technique such as RTL), e.g. The CPU software primarily performs the function of compiling batches of big data for processing by worker threads 20, for example by aggregating the data into various jobs to be processed by one or more hard-wired processing engines, and supplying various input data, for example, in a first-come, first-serve format. first served" into one or more FPGA engines 13 and/or receives results from them.

[00542] For example, as shown in FIG. 39, in various embodiments, the worker thread generates various job data batches that can be compiled and/or streamed into larger job batches that can be queued and/or further aggregated in preparation for transmission, e.g. via DDR3 in 7, for example via a low latency broadband point-to-point protocol such as QPI 3. In specific cases, data can be buffered according to specific data sets being passed to the FPGA. Once the packaged data is received by the FPGA 7, for example with cache coherence support, it can be processed and sent to one or more specialized clusters 11, from where it can be forwarded further to one or more sets of processing engines for processing therein. in accordance with one or more pipeline operations described herein.

[00543] Once processed, the result data can be sent back to the cluster and queued to be sent back over the point-to-point tightly coupled interconnect to the CPU for further processing. In certain implementations, data may be sent to a deaggregator thread for subsequent processing. Once post-processing is complete, the data can be sent back to the original worker thread 20, which can wait for the data. This distributed processing is especially useful in the case of the functions described earlier in this document. In particular, these functions are distinguished by the fact that their algorithmic complexity (although very demanding of network computing resources) is quite limited and each of them can be implemented with the ability to have a fairly uniform computational cost of all its various sub-operations.

[00544] However, in various cases, instead of processing data in large batches, smaller subroutines or protocols of individual functions or elements may be executed, for example, related to one or more functions of a pipeline, rather than performing full processing functions for that pipeline on that data. Therefore, a useful strategy may be to identify one or more critical functions for a computationally intensive operation in any given operation and then implement that subfunction in custom FPGA (hardware acceleration) logic, for example, for the computationally intensive subfunctions, and the rest of the operation, and ideally most or a significant portion of the algorithmic complexity is implemented in software for execution on the CPU/GPU/CPU, as described herein, for example, in accordance with FIG. 39.

[00545] In general, many genomic data processing operations are characterized by the fact that a small percentage of the algorithmic complexity accounts for a large percentage of the total computational load. Here's a typical example: 20% of the algorithmic complexity for a given function might account for 90% of the computational load, while the remaining 80% of the algorithmic complexity might account for only 10% of the computational load. Therefore, in various cases, components of the system described herein may be configured to implement a large, e.g., 20% or more, portion of the complexity to be implemented with high efficiency in custom FPGA logic, which may be configured to be monitored and controlled in hardware design and thus can be configured to implement this in an FPGA; which in turn can reduce the processing load on the CPU by 90%, thereby providing a 10x overall speedup. Other typical examples may be even more extreme, such as where 10% of the logarithmic complexity may account for 98% of the computational load, in which case applying FPGA acceleration as described herein to the 10% portion of the complexity may be even simpler , but can also provide up to 50x net acceleration. In various cases where extremely accelerated processing is required, one or more functions can be performed by a quantum computing device.

[00546] However, such "fragmented" or distributed processing-based acceleration approaches may be more practical when implemented on a tightly integrated CPU/GPU + FPGA platform rather than a loosely integrated CPU/GPU + FPGA platform. In particular, on a loosely integrated platform, the portion, for example, the functions to be implemented in the FPGA logic, can be selected in such a way as to minimize data input to the FPGA engines and to minimize data output from the FPGA engines, for example, for each unit processed data, and could additionally be executed with the ability to maintain a software/hardware boundary capable of withstanding high latency. In such cases, the hardware/software boundary may be reinforced, for example on a loosely integrated platform, to pull through certain narrow-bandwidth/high-bandwidth junctions, where these splits may otherwise be undesirable when optimizing the partitioning algorithmic complexity and computational load. This can often result in elongated hardware boundaries, undesirably capturing much of the algorithmic complexity in a hard-wired format, or in compressed hardware boundaries, undesirably excluding computationally dense parts.

[00547] In contrast, on a tightly coupled CPU/GPU+FPGA platform, due to shared memory with cache coherence and a high-bandwidth, low-latency CPU/GPU/FPGA interconnect, low-complexity/high-computation-load parts of the genomic data processing operation can be selected very precisely for implementation in custom FPGA logic (e.g., using the hardware engines described herein), with optimized software/hardware boundaries. In such a case, even if a data unit at the desired software/hardware boundary is large, it can still be handed over to the FPGA hardware engine for processing, simply by passing a pointer to the specific data unit. Particularly in the case shown in FIG. 33B, the hardware engine 13 of the FPGA 7 can dispense with access to each element of the data unit stored in the DRAM 1014; instead, it can access needed entries, such as those in cache 1014a, using small, efficient, low-latency 3' interconnects serving the CPU/GPU cache, thereby consuming less overall bandwidth than it otherwise would have. was to access the entire data unit and/or transfer it to FPGA 7, for example, using the DMA of DRAM 1014 over weak interconnect 3, as shown in. 33A.

[00548] In such cases, the hardware engine 13 may annotate the processing results to a data unit in place in CPU/GPU memory 1014 without streaming a complete copy of the data unit via DMA to the CPU/GPU memory. Even if the required software/hardware boundary is not suitable for software thread 20 to make a high-latency, non-blocking, first-in, first-out handoff to hardware engine 13, it could potentially make a blocking function call to hardware engine 13 by going to sleep. for a short delay until the hardware engines complete their operation, which latency is dramatically reduced by cache-coherent shared memory, high-speed, low-latency interconnect, and a distributed software/hardware coordination model, as shown in FIG. 33B.

[00549] In specific cases, because the specific algorithms and requirements for signal/image processing and base determination vary from one sequencer technology to another, and because the amount of raw data from a sequencer sensor is usually enormous (it is reduced to enormous after signal/image processing and to moderately large after base determination), such signal/image processing and base determination can be efficiently performed within the sequencer itself or at a nearby computing server connected via a broadband link to the sequencer. However, the throughput of DNA sequencers has steadily increased at rates exceeding Moore's Law, such that existing central processing unit (“CPU”) and/or graphics processing unit (“GPU”) based signal/image processing and base detection are implemented separately and each on his own, became more and more insufficient to complete this task. However, since tightly integrated CPU + FPGA and/or CPU + FPGA and/or GPU/CPU + FPGA platforms can be made compact and easily implemented in such a sequencer, for example, in the form of a CPU chip and/or GPU and/ or an FPGA placed on the sequencer motherboard, or easily installed in a server near the sequencer, or a cloud server system with remote access from the sequencer, such a sequencer can be an ideal platform to provide the computational acceleration provided by the FPGA/ASIC hardware engines described herein .

[00550] For example, the system proposed herein may be configured to perform primary, secondary, and/or tertiary processing, or a portion thereof, so as to be implemented using an accelerated CPU, GPU, and/or FPGA platform; CPU + FPGA; GPU + FPGA; GPU/CPU + FPGA; KPU; CPU/CPU; GPU/CPU; CPU and/or GPU and/or CPU + FPGA. In addition, such accelerated platforms, for example, containing one or more FPGA and/or CPU hardware engines, are useful for implementation in cloud systems, as described herein. For example, signal/image processing, radixing, mapping, alignment, sorting, deduplication and/or variant detection algorithms, or parts thereof, typically require a large amount of floating point and/or fixed point mathematical operations, especially additions and multiplications. These functions may also be configured to implement one or more quantum processing circuits, for example, implemented on a quantum processing platform.

[00551] In particular, large modern FPGA/quantum circuits contain thousands of high-speed multiply and add resources. More specifically, these circuits may contain custom engines that can be implemented in or with them, and the custom engines can be configured to perform parallel arithmetic operations at speeds far beyond the capabilities of simple general purpose CPUs. Likewise, simple GPUs have more comparable resources for parallel arithmetic operations. However, GPUs often have inconvenient architectural and programming limitations that can make it impossible to use them to their full potential. Accordingly, these FPGA and/or quantum processing and/or GPU arithmetic resources can be assembled into a circuit or otherwise configured structure to operate in exactly the intended manner with nearly 100% efficiency, for example, to perform the calculations required by the functions described in this document. Accordingly, it is possible to add a GPU card into expansion slots on a motherboard with a tightly integrated CPU and/or FPGA, thereby allowing all three types of processors to interoperate, although the GPU may still interoperate with all of its own and loosely integrated limitations.

[00552] More specifically, with respect to graphics processing unit (GPU) devices, in various cases the GPU may be configured to implement one or more functions, as described herein, to accelerate the processing speed of the underlying calculations required to perform a given function, completely or partially. More specifically, the GPU may be configured to perform one or more tasks in the mapping, alignment, sorting, deduplication, and/or variant detection protocol, for example, to speed up one or more calculations, such as large numbers of floating point math operations and/or or fixed point, such as associated additions and multiplications, to work in conjunction with the server's CPU and/or FPGA to speed up application or processing execution and reduce the computation cycles required to implement such functions. The cloud servers described herein with GPU/CPU/FPGA cards can be designed to handle computationally intensive tasks with ease and provide a less problematic user experience when used for visualization. Such computationally intensive tasks could also be offloaded to the cloud, for example to be executed by a quantum computing device.

[00553] Accordingly, if tightly integrated CPU+FPGA or GPU+FPGA and/or CPU/GPU/FPGA shared memory platforms are used in a sequencer or on an attached or cloud server, for example, for signal/image processing, base detection functions, mapping, leveling, sorting. By removing duplicates and/or identifying variants, gains can be achieved, for example, in the incremental development process. For example, an initially limited portion of the computational workload, such as programming a function for radixing, mapping, alignment, sorting, deduplication, and/or variant detection, may be implemented in one or more FPGA engines, while the rest of the work may be performed in CPU expansion cards and /or GPU However, the tight CPU/GPU/FPGA shared memory integration model presented herein can be later further configured to facilitate the incremental selection of additional compute-intensive functions for GPU, FPGA, and/or quantum acceleration. accelerations, which can then be implemented as processing engines, and their various functions can be offloaded to be executed in the FPGA and/or in some cases can be offloaded to the cloud, for example, to be executed by a CPU, thereby speeding up signal/image processing /defining bases/mapping/aligning/identifying variants. Such incremental advances may be implemented as needed to keep up with the increasing throughput of various primary and/or secondary and/or tertiary processing technologies.

[00554] Consequently, mapping and alignment of reads, eg, one or more reads, to a reference genome, as well as sorting, duplicate removal, and/or variant detection, can benefit from such acceleration using a GPU and/or FPGA or CPU. Namely, mapping and alignment and/or variant determination, or parts thereof, can be implemented partially or entirely as custom FPGA logic, for example, by streaming "to be mapped and/or aligned and/or variant determination" reads from CPU/GPU memory to FPGA mapping/alignment/variant detection engines and stream back records of mapped and/or aligned and/or variant reads that can then be streamed to the board, e.g. when performing sorting and /or definition of options. FPGA acceleration only works on a loosely coupled CPU/GPU + FPGA platform, and in the configurations described herein can be extremely fast. However, there are some additional benefits that can be achieved by moving to a tightly integrated CPU/GPU/CPU + FPGA platform.

[00555] Accordingly, with respect to mapping, alignment, and variant determination, in some embodiments, the overall benefit of a tightly integrated CPU/GPU+FPGA and/or quantum processing platform as described herein is that faster mapping/alignment/ defining options, such as hardware acceleration, can be effectively broken down into several discrete computationally intensive operations, such as seed generation and/or mapping, seed chaining, paired-end recovery scanning, gapless alignment, and gapped alignment (Smith-Waterman or Needleman-Wunsch), generating a de Bruijn graph, performing an HMM calculation, etc., for example when CPU and/or GPU and/or quantum computing software performs lighter (but not necessarily less complex) tasks and can make calls acceleration from discrete hardware and/or other quantum computing engines as needed. Such a model may be less efficient on a typical loosely integrated CPU/GPU + FPGA platform, for example, due to large amounts of data to be transferred back and forth between stages and high latencies, but may be more efficient on a tightly integrated CPU + FPGA, GPU platform + FPGA and/or quantum computing with shared memory with cache coherence, high-bandwidth low-latency interconnect, and distributed software/hardware coordination model. In addition, for example, with regard to variant determination, Hidden Markov Model (HMM) and/or Dynamic Programming (DP) algorithms, including Viterbi and forward algorithms, can be implemented in conjunction with the base/mapping/alignment operation. sorting/removing duplicates, for example, to compare the most likely source sequence that explains the observed sensor measurements, in a configuration, for example, well suited for parallel mesh FPGA circuits and/or quantum circuits described herein.

[00556] Namely, efficient use of hardware and/or software resources in a distributed processing configuration can result from reduced acceleration using hardware and/or quantum computing for discrete computationally intensive functions. In such cases, several of the functions described herein may be executed in a monolithic strictly hardware engine to require less computationally intensive processing, but may nonetheless remain algorithmically complex and therefore may consume significant amounts of physical FPGA resources (lookup tables, flip-flops, block RAM, etc.). In such cases, moving some or all of the various discrete functions to software would take over existing CPU cycles in exchange for freeing up significant FPGA space. In certain such cases, the freed up FPGA area can be used to provide greater parallelism to computationally intensive mapping/alignment/variant detection subfunctions and thus enhance acceleration, or for other genomic acceleration functions. Such benefits can also be achieved by implementing computationally intensive functions in one or more dedicated quantum circuits to implement a quantum computing platform.

[00557] Therefore, in various implementations, the algorithmic complexity of one or more of the functions described herein may be somewhat reduced by performing them strictly in hardware or strictly in a quantum computing implementation. However, some operations, such as comparing pairs of candidate alignments for paired-end reads and/or performing subtle mapping quality assessments (MAPQ), represent very low computational loads and would therefore benefit from more sophisticated and accurate processing in CPU software /GPU and/or quantum computing. Therefore, as a rule, reducing hardware processing to specific computationally intensive operations will allow the use of more complex and accurate algorithms in the CPU/GPU parts,

[00558] Additionally, in various embodiments, all or part of the mapping/alignment/sorting/deduplication/variant detection operations described herein may be performed such that more algorithmically complex calculations may be performed at higher levels in hardware and/or in one or more quantum circuits, for example, when the computationally intensive hardware and/or quantum functions being invoked are configured to execute in a dynamic or iterative manner. In particular, a monolithic design of strictly hardware/quantum processing can be implemented with the ability to function more efficiently in a linear pipeline. For example, if during processing one Smith-Waterman alignment displayed indicates that the true alignment path extends beyond the evaluation band, such as the capture band, as described above, another Smith-Waterman alignment must be called to correct this. Therefore, these configurations can essentially reduce FPGA hardware acceleration/quantum acceleration to discrete functions, such as forming procedural abstractions that make it easier to build higher-level complexity on top of them.

[00559] In addition, in various cases, flexibility within variant mapping/alignment/identification algorithms and their functions can be improved by reducing software and/or quantum acceleration to discrete computationally intensive functions and running the system with the ability to implement others, such as less resource-intensive parts in CPU and/or GPU software. For example, although hardware algorithms can be modified and reconfigured in FPGAs, typically such changes to hardware designs, such as via firmware, can require several times the effort of such changes in software. In such cases, the computationally intensive parts of mapping and alignment and sorting and deduplication and/or variant detection, such as seed mapping, seed chaining, paired end recovery scanning, gapless alignment, gap alignment, and HMMs, which are relatively are well defined, are therefore stable functions and do not require frequent algorithmic changes. Therefore, these functions can be suitably optimized in hardware, while other functions that can be performed by CPU/GPU software are more suitable for incremental improvement of algorithms, which is much easier in software. However, once fully developed, they can be implemented in hardware. One or more of these functions may also be configured to be implemented in one or more quantum circuits of a quantum processing machine.

[00560] Accordingly, in various cases, variant detection (with respect to DNA or RNA, single sample or joint, generative or somatic, etc.) can also benefit from FPGA acceleration and/or quantum acceleration, e.g. its various computationally intensive functions. For example, haplotype-based determiners, which determine bases based on supporting data derived from the context provided within a window around a candidate, as described in frequencies above, are the most computationally intensive. These operations, including comparison of a candidate haplotype (e.g., a single-stranded nucleotide sequence representing the hypothesis of the truth of the sequence of at least one of the sample strands at the genomic locus under study) with each sequencer read, for example, to estimate the conditional probability of observing a read given the truth of a given haplotype.

[00561] Such an operation can be performed using one or more MRJD, pairwise hidden Markov model (pairwise-HMM) and/or partially defined Markov model (PD-HMM) calculations that summarize the probabilities of possible combinations of sequencing or sample preparation (PCR) errors etc.) using a dynamic programming algorithm. Therefore, in such cases, the system may be configured to accelerate the computation of a pair-HMM or PD-HMM using one or more, for example, parallel, FPGA hardware engines or quantum processing engines, where the CPU/GPU/CPU software may be executed with the ability to execute the rest of the parent haplotype-based variant detection algorithm, either on a loosely integrated platform, or on a tightly integrated CPU + FPGA, or GPU + FPGA, or CPU and/or GPU + FPGA, and/or CPU platform. For example, with weak integration, software threads can build and prepare a de Bruijn graph and/or assembly graph from reads that overlap a selected active region (a window or a contiguous subset of a reference genome), extract candidate haplotypes from the graph, and queue haplotype-read pairs for transfers using DMA to hardware FPGA engines, for example, for pair-HMM or PD-HMM comparison. The same or other software threads can then receive the pair-HMM results queued and DMAed back from the FPGA to CPU/GPU memory and perform genotyping and Bayesian probability calculations to perform the final variant determination. Of course, one or more of these functions may be capable of execution on one or more quantum computing platforms.

[00562] For example, as shown in FIG. 38, the CPU/GPU 1000 may include one or more, such as a plurality, of threads 20a, 20b, and 20c, each of which may have access to an associated DRAM 1014, the DRAM having workspaces 1014a, 1014b, and 1014c within which each thread 20a, 20b, and 20c may have access, respectively, to perform one or more operations on one or more data structures, such as large data structures. These memory portions and their data structures may be accessed, for example, through corresponding cache portions 1014a′, for example, by one or more processing engines 13a, 13b, 13c of the FPGA 7, wherein the processing engines may access the reference data structures, e.g. performing one or more of the operations described herein, such as mapping, alignment, sorting, and/or variant searching. Thanks to the wideband tight interconnect 3, data related to data structures and/or associated with processing results can be shared substantially seamlessly between the CPU and/or GPU and/or CPU and/or associated FPGA, for example while maintaining cache coherence , to optimize processing efficiency.

[00563] Accordingly, in accordance with one aspect, a system may be provided that may be configured to share memory resources among its constituent parts, for example, when performing certain computing tasks or subfunctions through software, such as executed by a CPU, and/ or GPU and/or CPU, and/or performing other computing tasks or routines through firmware, such as through associated integrated circuit hardware such as an FPGA, ASIC, and/or structured ASIC. This can be achieved in various ways, for example through direct weak or tight coupling between the CPU/GPU/CPU and the chip, such as an FPGA. Such configurations may be particularly useful when distributing operations related to the processing of large data structures, as described herein, that use time-consuming functions or subfunctions intended to be used and accessed by both the CPU and/or GPU and/or CPU and integrated circuit. In particular, in various embodiments, when processing data through a genomic pipeline as described herein, for example, to speed up overall processing, timing, and efficiency, a number of different operations may be performed on the data, which operations may involve both software and processing hardware components.

[00564] Consequently, it may be necessary to share or otherwise exchange data between a software component running on the CPU and/or GPU and/or CPU and an on-chip hardware component, such as an FPGA or ASIC. Accordingly, one or more of the various stages in a processing pipeline, or portions thereof, may be performed by one device, such as a CPU/GPU/CPU, and one or more of the various stages may be performed by another device, such as an FPGA or ASIC. In such a case, the CPU and FPGA must be coupled in a data-exchangeable manner, eg, via a point-to-point interconnect, in a manner that allows such data to be transferred efficiently, which coupling may include sharing memory resources. To achieve such distribution of tasks and sharing of information to perform such tasks, the CPU and/or GPU and/or GPU may be loosely or tightly coupled to each other and/or FPGA or other chipset, and a worker management system may be included threads to efficiently distribute the workload.

[00565] Therefore, in specific embodiments, a genomic analysis platform is provided. For example, the platform may include a motherboard, memory, a plurality of integrated circuits, such as forming one or more of a CPU/GPU/CPU, a mapping module, an alignment module, a sorting module, and/or a variant determination module. Namely, in specific embodiments, the platform may include a first integrated circuit, such as an integrated circuit forming a central processing unit (CPU) or graphics processing unit (GPU), that responds to one or more software algorithms that are configured to provide The CPU/GPU instructions to perform one or more sets of genomic analysis functions as described herein, for example, wherein the CPU/GPU includes a first set of physical electronic interconnects for connection to a motherboard. In other embodiments, a quantum processing device is provided, wherein the CPU includes one or more quantum circuits that are configured to perform one or more of the functions described herein. In various cases, memory is provided that may also be attached to the motherboard and may also be electronically coupled to the CPU and/or GPU and/or CPU, for example through at least a portion of the first set of physical electronic interconnects. In such cases, the memory may be configured to store multiple reads of genomic data, and/or at least one or more genetic reference sequences, and/or an index, for example, in a hash table, of one or more genetic reference sequences.

[00566] In addition, the platform may include one or more second integrated circuits, for example, where each of the second integrated circuits forms a field programmable gate array (FPGA), or ASIC, or structured ASIC having a second set of physical electronic interconnects for interconnection with the CPU and memory, for example via a point-to-point protocol. In such a case, the FPGA (or structured ASIC) may be programmable via firmware to configure a set of hard-wired digital logic circuits that are interconnected by a plurality of physical interconnects to perform a second set of genomic analysis functions, e.g., mapping, alignment, sorting, deletion duplicates, variant definitions, etc., HMM functions, etc. In particular, the hard-wired FPGA digital logic circuits may be implemented as a set of processing engines for implementing one or more preconfigured steps in a sequence analysis pipeline of a genomic analysis platform, for example, where the sets of processing engines include one or more of mapping modules, and /or alignment, and/or sorting, and/or removing duplicates, and/or identifying variants, and modules can be formed from separate or the same subsets of processing engines.

[00567] For example, with respect to variant determination, computing the pairwise-HMM or PD-HMM is one of the most computationally intensive steps in a haplotype-based variant determination protocol. Therefore, the speed of variant detection can be greatly improved by accelerating this step in one or more FPGA engines or quantum processing engines. However, additional benefits can be obtained by accelerating other computationally intensive steps in additional FPGA and/or QP engines to achieve greater acceleration of variant determination or part thereof, or reducing the CPU/GPU load and the number of required SPU/GPU cores, or both, as shown in FIG. 38.

[00568] Additional computationally intensive functions relevant to variant detection that may be implemented in FPGA and/or quantum processing engines include: region detection where regions of the reference genome covered by aligned reads are selected for processing of sufficient depth and/or quality; active region detection, where reference genome loci with non-trivial supporting evidence for possible variants are identified and windows with sufficient context around these loci are selected as active regions for further analysis; constructing a de Bruijn graph or other assembly graph, where reads overlapping the active region and/or K-mers from these reads are assembled into a graph; preparing the assembly graph, for example, pruning paths with low coverage or low quality, restoring dangling start and end sections of paths by connecting them at a reference skeleton in the graph, converting the graph representation from K-mers to a sequence, merging similar branches, and otherwise simplifying the graph; selection of candidate haplotypes from the collected graph; and the alignment of candidate reference gnome haplotypes, e.g., using Smith-Waterman alignment, e.g., to identify variants (SNPs and/or indels) in the reference represented by the haplotype, and synchronizing their nucleotide positions with the reference.

[00569] All of these functions can be implemented as high-performance hardware engines in FPGAs and/or using one or more quantum circuits of a quantum computing platform. However, calling such a variety of hardware acceleration functions from multiple integration points in the variant definition software may become ineffective on a loosely coupled CPU/GPU/CPU + FPGA platform, and therefore a tightly integrated CPU/GPU/CPU + FPGA platform may be appropriate. For example, various stepwise processing techniques such as constructing, preparing a de Bruijn graph or other assembly graph and extracting haplotypes from it can greatly benefit from a tightly coupled CPU/GPU/CPU + FPGA platform. Additionally, assembly graphs are large and complex data structures, and repeatedly transferring them between the CPU and/or GPU and FPGA can become resource-intensive and prevent significant speedup.

[00570] Therefore, the ideal model for such graph processing using a tightly integrated CPU/GPU/CPU and FPGA platform is to store such graphs in shared memory while maintaining cache coherence for alternate processing by the CPU and/or GPU and/ or CPU and FPGA hardware functions. In such a case, the software thread processing the graph data can iteratively issue commands to perform various computationally intensive steps in the hardware engine's graph processing, and the software can then examine the results and determine the next steps between hardware calls, such as in the example given the process depicted in FIG. 39. This processing model can be managed by a suitably configured workflow management system and/or can be configured to conform to programming paradigms such as a data structure API or an object-oriented method interface, but accelerate computationally intensive functions using custom hardware engines and/or quantum processing engines, which in practice are implemented through implementation on a tightly coupled CPU and/or GPU and/or CPU + FPGA platform with shared memory supporting cache coherence and high-bandwidth CPU/GPU/CPU/FPGA interconnects with low latency.

[00571] Accordingly, in addition to mapping and alignment of reads to a reference genome, reads may be “de novo” assembled, e.g., without a reference genome, e.g., by detecting apparent overlaps between reads, e.g., in a cluster where they are completely or mostly consistent , and combining them into longer sequences, contigs, scaffolds, or graphs. This assembly can also be performed locally, for example using all reads that are determined to map to a given chromosome or part thereof. When assembled in this way, it is possible to include a reference genome or a segment of it in the assembled structure.

[00572] In such a case, due to the difficulty of joining together read sequences that do not fully agree, a graph structure can be used, for example, where overlapping reads may agree on a single sequence in one segment, but branch out into multiple sequences in an adjacent segment, as explained above. Therefore, such an assembly graph may be a sequence graph, where each edge or node represents one nucleotide or sequence of nucleotides that are considered to be adjacent without gap to the sequences in the connected edges or nodes. In specific cases, such an assembly graph may be a k-mer graph, where each node represents a k-mer or nucleotide sequence of (usually) a fixed length k, and where connected nodes are considered to overlap each other in longer observed sequences, typically overlapping k - 1 nucleotides. In various methods, one or more transformations between one or more sequence graphs and k-mer graphs are possible.

[00573] Although assembly graphs are used to identify variants based on haplotypes, and some of the graph processing techniques used are similar, there are important differences. De novo assembly graphs are typically much larger and use longer k-mers. While assembly graphs for variant detection are limited to fairly structured and relatively simple graphs, such as those having no cycles and running from source to sink along the reference sequence backbone, de novo assembly graphs are typically less structured and more complex, with cycles, dangling paths, and other anomalies , not only permitted, but also subject to special analysis. De novo assembly graph coloring is sometimes used, assigning nodes and edges “colors” indicating, for example, what biological sample they are from or a matching reference sequence. Consequently, de novo assembly graphs require the use of a wider selection of graph analysis and processing functions, often iteratively or recursively, and in particular, due to the size and complexity of de novo assembly graphs, the processing functions tend to be extremely computationally intensive.

[00574] Therefore, as stated above, the ideal model for such graph processing on a tightly integrated CPU/GPU/CPU + FPGA platform is to store such graphs in shared memory while maintaining cache coherence for interleaved processing between the CPU, and/or GPU, and/or CPU and FPGA hardware functions. In such a case, the software thread processing the data graph can iteratively issue commands to perform various computationally intensive steps in the hardware engine's graph processing, and then examine the results to thereby determine the next steps to be performed by the hardware, for example by flagging the corresponding calls hardware equipment. As noted above, this processing model benefits greatly from implementation on a tightly coupled CPU+FPGA platform with shared memory maintaining cache coherence and a high-bandwidth, low-latency CPU/FPGA interconnect.

[00575] Additionally, as described later herein, tertiary analysis includes genomic processing, which may follow graph assembly and/or variant identification, which in clinical applications may include variant annotation, phenotype prediction, disease testing and/or predicting response to therapy as described herein. The reasons why it is useful to run tertiary analysis on such a tightly integrated CPU/GPU/CPU + FPGA platform is that this platform configuration provides efficient acceleration of primary and/or secondary processing that is very computationally intensive, and is ideal for continuing tertiary analysis on that same platform for convenience and to reduce turnaround time and to minimize the transfer and copying of large genomic data files. Therefore, a loosely or tightly coupled CPU/GPU/CPU + FPGA platform is a good choice, but a tightly coupled platform may have additional benefits since the tertiary analysis steps and methods vary widely depending on the application, and in any case where computationally intensive steps slow down tertiary analysis, acceleration of these steps can be realized in an optimized manner using a custom FPGA.

[00576] For example, tertiary analysis on a tightly integrated CPU/GPU/CPU and/or FPGA platform is particularly advantageous due to the ability to re-analyze genomic data in an iterative manner using CPU/GPU/CPU acceleration and/or FPGA secondary processing in response to partial or intermediate tertiary results, which may allow for additional benefit from a tight integration configuration. For example, after a tertiary analysis detects a possible phenotype or disease, but with limited confidence in the truth or falsity of that finding, a targeted secondary analysis can be repeated, focusing on specific reads and reference regions that influence that finding, thereby improving accuracy and the confidence of the corresponding variant definitions, and in turn improving the confidence of the detection definition. In addition, if the tertiary analysis identifies genotype information for a pedigree or structural variant, the secondary analysis can be repeated using a different or modified genome that is more suitable for a particular individual, thereby increasing the accuracy of variant definitions and improving the accuracy of further steps in the tertiary analysis.

[00577] However, if tertiary analysis is performed on a CPU-only platform after primary and secondary processing (possibly accelerated on a separate platform), re-analysis using the secondary analysis tools will likely be too slow to be useful on the tertiary analysis platform itself on its own, and the alternative is to transfer to a faster platform, which is also prohibitively slow. Thus, in the absence of any form of hardware or quantum acceleration on a tertiary analysis platform, primary and secondary processing must typically be completed before tertiary analysis begins, without the ability to simply re-analyze or iteratively secondary analyze and/or pipeline analysis functions. But on a platform accelerated by FPGA and/or quantum processing, and especially on a CPU and/or GPU and/or CPU and/or FPGA platform where secondary processing is most efficient, iterative analysis becomes practical and useful.

[00578] Accordingly, as discussed above, the modules described herein may be implemented in hardware of a chip, for example, hard-wired therein, and in such cases, their implementation may be such that they can operate at higher speeds and with greater accuracy compared to implementation in software, for example, when there are minimal instructions that need to be retrieved, read and/or executed. Moreover, in various cases, the functions to be performed by one or more of these modules may be distributed so that the various functions may be implemented by software of the main CPU and/or GPU and/or CPU, while in others In cases, various other functions could be performed by the hardware of the associated FPGA, for example when two or more devices perform their respective functions with each other, for example in a coordinated manner. For such purposes, a CPU, GPU, CPU and/or FPGA, or ASIC, or structured ASIC can be hard coupled, for example, through a low latency broadband connection such as QPI, CCVI, CAPI, etc. Accordingly, in some cases, computationally intensive functions to be performed by one or more of these modules may be performed by a quantum processor implemented by one or more quantum circuits.

[00579] Therefore, given a unique implementation of hardware and/or quantum processing, modules of the present invention can operate directly according to their operating parameters, for example, without the need to fetch, read and/or execute instructions as when implemented solely in software CPU. In addition, memory and processing time requirements can also be reduced, for example when on-chip data exchanges are carried out through files, for example stored locally in the FPGA/CPU/GPU/CPU cache, for example maintaining cache coherence, rather than through large-scale accesses to external memory. Of course, in some cases, the chip and/or board can be made to be sized to include more memory, such as as much as the memory board, to enhance parallel processing capabilities, resulting in even higher processing speeds. For example, in certain embodiments, a chip of the present invention may include an on-chip DRAM device so that the chip does not have to rely on external memory, thereby further increasing processing speed, such as when the Burrows-Wheeler algorithm or de Bruijn graph can be used instead of a hash table and a hash function, which may in various cases rely on external memory, such as main memory. In such cases, it is possible to achieve completion of part or all of the pipeline in 6, or 10, or 12, or 15, or 20 minutes or less, for example from start to finish.

[00580] As stated above, there are all sorts of different places where a module can be placed in hardware, or can be located remotely from it, such as on a server accessible in the cloud. When a given module is placed on a chip, for example hard-wired into a chip, its function may be performed by hardware, but if necessary the module may be placed remotely from the chip and the platform may then contain the necessary means to send the corresponding data to a remote location such as a server , for example, a quantum server accessible via the cloud so that certain functionality of the module can be used for further data processing in accordance with user-selectable required protocols. Accordingly, a portion of the platform may comprise a web interface for performing one or more tasks in accordance with the operation of one or more modules described herein. For example, when mapping, alignment, and/or sorting are all modules that may take place on a chip, in various cases one or more of local realignment, duplicate flagging, base quality assessment recalibration, and/or variant searching may occur on the cloud.

[00581] In particular, after genetic data has been generated and/or processed, for example, in one or more primary and/or secondary processing protocols, for example, mapped, aligned and/or sorted, for example, to create one or more files determining variants, for example, to determine how a subject's genetic sequence data differs from one or more reference sequences, in accordance with a further aspect, the present invention may relate to performing one or more other analytical functions on the generated and/or processed genetic data, for example, for further processing, such as tertiary processing, as shown in FIG. 40. For example, the system may be configured to further process the generated and/or secondary processed data, for example, by passing it through one or more tertiary processing pipelines 700, such as one or more of a microarray analysis pipeline, a genomic analysis pipeline, for example , whole-genome analysis pipeline, genotyping analysis pipeline, exome analysis pipeline, microbiome analysis pipeline, genotyping analysis pipeline including co-genotyping, variant analysis pipeline including structural variant pipelines, somatic variant pipelines, and GATK and/or MuTect2 pipelines, and RNA sequencing pipelines and other genetic analysis pipelines.

[00582] Additionally, an additional processing layer 800 may be provided in various cases, for example, for disease diagnosis, therapeutic intervention, and/or prophylactic warning, for example, including NIPT, NICU, cancer, LDT, agribiological, and other such diagnostic data diseases, prevention and/or therapies, using these data generated by one or more of the provided primary and/or secondary and/or tertiary pipelines. For example, specific bioanalytical pipelines include genome pipelines, epigenome pipelines, metagenome pipelines, variant pipelines such as GATK/MuTect2, and other such pipelines. Therefore, the devices and methods described herein can be used to generate genetic sequence data, which can then be used to generate one or more variant definition files and/or other associated information that can be further processed by other tertiary pipelines. processing in accordance with the devices and methods described herein, for example, for the diagnosis of specific and/or general diseases, as well as for preventive and/or therapeutic measures and/or developmental methods. See, for example, FIG. 41 B, C and 43.

[00583] As described above, the methods and/or systems provided herein may include generating and/or otherwise obtaining genetic sequence data. Such data may be generated or otherwise obtained from any suitable source, such as an STS or a “chip-based sequencer.” The methods and systems described herein may include performing further processing on this generated and/or received data, for example, using one or more secondary processing 600 protocols. Secondary processing protocols may include one or more of mapping, aligning, and sorting generated genetic sequence data, for example, to create one or more variant definition files, for example, to determine how a subject's genetic sequence data differs from one or more reference sequences or genomes. According to another aspect, the invention may relate to performing one or more analytical functions on generated and/or processed genetic data, for example, the resulting secondary processing data, for example, for additional processing, for example, tertiary processing 700/800, which can be performed on or together with the same chip or chipset on which the above sequencer technology is provided.

[00584] Accordingly, in the first case, for example, regarding the generation, acquisition and/or transmission of genetic sequence data, as shown in FIG. 37-41, such data may be generated either locally or remotely, and/or the results thereof may then be processed directly, for example by local computing resource 100, or may be transmitted to a remote location, such as remote computing resource 300, to further processing, for example, for secondary and/or tertiary processing, see FIG. 42. For example, generated genetic sequence data can be processed locally and directly, such as when sequencing and secondary processing functionality resides on the same chipset and/or within the same field device 10. Similarly generated Genetic sequence data can be processed locally and indirectly, for example when sequencing and secondary processing functionality is performed separately by different devices that share the same equipment or location, but may be spatially separated, although connected with data exchange capability , for example, over a local network 10. In the following case, genetic sequence data can be produced remotely, for example, using a remote SNP, and the resulting data can be transmitted over a 30/50 cloud network to an autonomous remote location 300, for example, geographically separated from the sequencer.

[00585] Namely, as shown in FIG. 40A, in various embodiments, a data generation device, such as a nucleotide sequencer 110, may be provided at the site of operation, such as where the sequencer is a sequencer on a chip, or SOC, wherein the sequencer is coupled to the local computing resource 100 either directly or indirectly. eg using a 10/30 LAN connection. Local computing resource 100 may include or be otherwise associated with one or more data generation engines 110 and/or data acquisition engines 120. Such mechanisms may be mechanisms configured to generate and/or otherwise obtain data, such as analog, digital and/or electromagnetic data, related to one or more genetic sequences of a subject or group of subjects, for example, when the genetic sequence data is represented in the format BCL or FASTQ file.

[00586] For example, such data generation engine 110 may be a primary processor, such as a sequencer, such as a sequencer, an on-chip sequencer, or other similar mechanism for generating genetic sequence information. In addition, such data acquisition mechanisms 120 may be any mechanism configured to receive data, such as generated genetic sequence information; and/or together with the data generator 110 and/or the computing resource 100 of applying one or more secondary processing protocols to this information, for example, conveyor secondary processing devices configured to execute matcher, equalizer, sorter and/or variant determiner protocols on the generated and /or obtained sequence data as described herein. In various cases, the data generating device 110 and/or the data receiving device 120 may be connected by a network to each other, for example, a local network 10 such as for local storage 200; or may be linked by a network, such as a local and/or cloud network 30, such as for transmitting and/or receiving data, such as digital data related to the primary and/or secondary processing of genetic sequence information, for example, to a remote location or therefrom, for example, for remote processing 300 and/or storage 400. In various embodiments, one or more of these components may be communicatively connected using a hybrid network as described herein.

[00587] The local computing resource 100 may also include or be otherwise associated with a compiler 130 and/or a processor 140, such as a compiler 130 configured to compile generated and/or received data and/or data associated therewith, and a processor 140 configured to process generated and/or received data and/or compiled data and/or control data of the system 1 and its components as described herein, for example, to perform primary, secondary and/or tertiary processing. For example, any suitable compiler can be used, but in certain cases additional efficiency can be achieved not only by implementing a tightly coupled configuration such as discussed above to efficiently and coherently transfer data between system components, but it can also be achieved by implementing a configuration " just-in-time (JIT) computer language compiler. Additionally, in certain cases, processor 140 may include a workflow management system to control the operation of various system components with respect to data generated, received, and/or to be processed through various stages of platform pipelines.

[00588] More specifically, as used herein, the term just-in-time (JIT) refers to a device, system, and/or method for converting received and/or generated files from one format to another. In general use, the JIT system described herein may include a compiler 130 or other computing architecture, such as a processing program, that can be implemented to convert various codes from one form to another. For example, in one implementation, the JIT compiler may be configured to convert bytecode, or other program code, containing instructions that need to be interpreted into instructions suitable for sending directly to the associated processor 140 for almost immediate execution, for example, without having to interpret the instructions using specific machine language. In particular, after a coding program, such as a Java program, has been written, the source language statements may be compiled by a compiler, such as a Java compiler, into bytecode instead of being compiled into code that contains instructions corresponding to to any given specific hardware platform processing language. Therefore, this bytecode that compiles the action is platform-independent code that can be sent to any platform and executed on that platform regardless of the underlying processor. Therefore, a suitable compiler may be a compiler that is configured to compile bytecode into platform-specific executable code, which can then be executed immediately. In this case, the JIT compiler can be configured to immediately convert one file format to another, for example on the fly.

[00589] Therefore, a suitably configured compiler, as described herein, is able to overcome various shortcomings in the art. Namely, after compilation, programs that were written in a particular language must be recompiled and/or rewritten depending on each specific computer platform on which they were to be implemented. In modern compilation systems, the compiler may be configured to write and compile a program only once, and once written in a particular form, it can be converted to one or more other forms almost immediately. More specifically, compiler 130 may be a JIT or other similar dynamic translation compiler format that is configured to write instructions in a platform-independent language that does not need to be recompiled and/or rewritten depending on the particular computer platform on which it is implemented. For example, in a particular use model, the compiler may be configured to interpret compiled bytecode and/or other encoded instructions into instructions that are understandable to that particular processor to convert one file format to another regardless of the computing platform. Essentially, the JIT system described herein is configured to take one genetic file, for example representing genetic code, such as a BCL or FASTQ file generated by a genetic sequencer, and quickly convert it to another form, for example, a SAM, BAM and /or CRAM file, for example, using the methods described herein.

[00590] In particular, in various cases, the system described herein may include a first and/or second compiler 130a and 130b, such as a virtual compiler machine, that performs the conversion of one or multiple bytecode instructions at a time. For example, the use of a just-in-time compiler such as Java or other appropriately configured second compiler within a given system platform will allow instructions to be compiled into bytecode, which can then be converted to specific system code, for example, as if the program had originally been compiled on that platform . Accordingly, after the code is compiled and/or recompiled, for example, using the JIT compiler(s) 130, it will execute faster on the computer processor 140. Therefore, in various implementations, just-in-time (JIT) compilation may be configured to occur during execution of a given program, for example, at runtime rather than before execution. In such a case, this may include steps of translating into machine code or translating into another format that can then be executed directly, thereby enabling one or more of compilation before execution (AOT) and/or interpretation.

[00591] More specifically, as implemented in the present system, a typical data stream typically produces data in one or more formats derived from one or more computing platforms, such as BCL, FASTQ, SAM, BAM, CRAM, and/or VCF file formats or their equivalents. For example, a typical DNA sequencer 110, e.g., SNP, produces raw signals representing specific bases, referred to herein as reads, e.g., in the form of a BCL and/or FASTQ file, which can optionally be subjected to further processing, e.g. image processing, and/or compressed 150. Similarly, the reads of the generated BCL/FASTQ files can then be subjected to further processing in the system, as described herein, to create mapped and/or aligned data, and these resulting data, e.g., mapped and aligned reads may be in SAM or BAM file format, or alternatively in CRAM file format. Further, the SAM or BAM file can then be processed, for example, using a variant definition procedure, to obtain a variant definition file, such as a VCF file or a gVCF file. Accordingly, once created, all of these resulting BCL, FASTQ, SAM, BAM, CRAM and/or VCF files are (extremely) large files that need to be stored, for example, in the system memory architecture, locally 200 or remotely 400. Storing any one of these files are expensive. Storing all files in these formats is extremely expensive.

[00592] As stated, just-in-time (JIT) compilation or other double compilation or other dynamic translation compilation analysis can be performed and deployed on a given system with the potential to reduce such high storage costs. For example, a JIT analysis scheme may be implemented in a given system such that data is stored in only one format (e.g., FASTQ or BAM compressed file format, etc.), while simultaneously providing access to one or more file formats (e.g. , BCL, FASTQ, SAM, BAM, CRAM and/or VCF, etc.). This rapid file conversion process can be accomplished by fast genomic data processing using appropriate hardware and/or quantum acceleration platforms described herein, such as those for mapping, aligning, sorting, and/or identifying variants (or their constituent functions, such as duplicate removal, HMM and Smith-Waterman algorithm, compression and decompression, etc.) in hardware engines on integrated circuits such as FPGA, or using a quantum processor. Therefore, by implementing JIT or similar analysis along with such acceleration, genomic data can be processed into required file formats on the fly at speeds comparable to conventional file access. Thus, significant storage savings can be realized through JIT-type processing with little or no loss in access speed.

[00593] In particular, there are two options that are useful for a core genomic data store created as described herein to be available for JIT processing, namely storage of unaligned reads (e.g., this could include compressed FASTQ file or unaligned compressed SAM, BAM or CRAM files), and storage of aligned reads (for example, this could include compressed BAM or CRAM files). However, since the accelerated processing described herein allows any of the mentioned file formats to be quickly obtained, for example, on the fly, the base file format for storage can be selected to achieve the smallest compressed file size, thereby reducing storage costs. Therefore, given the comparatively smaller file size of raw, eg, original unaligned, read data, there is an advantage to storing unaligned reads so that data fields are kept to a minimum. Likewise, there is an advantage to storing processed and compressed data, such as in a CRAM file format.

[00594] More specifically, given the high processing speeds that can be achieved with the systems devices and methods of use described herein, in many cases it may not be necessary to store mapped and/or aligned data for every single read, since this information can be easily retrieved as needed, for example, on the fly. Additionally, although the compressed FASTQ file format (e.g., FASTQ.gz) is typically used to store genetic sequence data, such unaligned read data may also be stored in more advanced compressed formats, such as after mapping and/or alignment in SAM, BAM, or CRAMs that can further reduce file size, for example by using a compact binary representation and/or more targeted compression techniques. Consequently, these file formats can be compressed before storage, decompressed after storage, and quickly processed, for example on the fly, to convert one file format to another.

[00595] An advantage of storing aligned reads is that the sequence content of most or all of the reads can be omitted. Namely, the system can be efficiently improved and storage space saved simply by storing the difference between the read sequences and the selected reference genome, for example, which is indicated by the alignment positions of the read variant. More precisely, since differences from the reference are usually infrequent, the aligned position and list of differences can often be stored more compactly than the original read sequence. Therefore, in various cases, storing in an aligned read format, for example when storing data related to differences between aligned reads, may be preferable to storing unaligned read data. In such a case, if the aligned read and/or variant definition format is used as the base storage format, for example in a JIT procedure, other formats such as SAM, BAM and/or CRAM compressed file formats may also be used.

[00596] Along with the aligned and/or unaligned read file data to be stored, a variety of other data may also be stored, such as metadata derived from the various calculations defined herein. Such calculated data may include mapped, aligned and/or further processed data such as alignment scores, mapping confidence, editorial distance from reference, etc. In certain cases, such metadata and/or other additional information does not need to be kept in the underlying storage for JIT analysis, such as in cases where it can be rendered on the fly, such as using the accelerated data processing described herein.

[00597] In terms of metadata, the data may be a small file that instructs the system in order to navigate back or forth from one file format to a conversion to another file format. Therefore, the metadata file will allow the system to create a binary compatible version of any other file type. For example, when moving forward from the original data file, the system only needs to access the instructions in the metadata and implement them. Along with fast file format conversion, JIT also enables fast compression and/or decompression and/or storage, for example, in a genome cloud memory cache.

[00598] As discussed in more detail below, once sequence data is generated 110, it may be stored locally 200 and/or made available for remote storage, for example, in a cloud-accessible cache 400 such as a Dropbox cloud storage device. For example, once in Dropbox's genomic storage, the data may appear to be available on the cloud 50, and therefore may be subject to further processing, for example, essentially immediately. This is particularly useful when there are multiple mapping/alignment/sorting/variant identification systems 100/300, for example on one of the two sides of the cloud interface 50, facilitating automatic loading and processing of data that can be further processed, for example, using JIT technology described in this document.

[00599] For example, a basic storage format for JIT compilation and/or processing may contain minimal data fields such as read name, base quality scores, alignment position and/or orientation in the reference, and a list of differences from the reference, for example where each field may be compressed optimally for its data type. Various other metadata may be included or otherwise associated with the storage file. In such a case, the underlying storage for JIT analysis may be on a local file system 200, such as hard disk drives and solid state drives, or on a network storage resource, such as a storage system 400 such as a NAS object store or Dropbox storage. In particular, when storing files of various formats such as BCL, FASTQ, SAM, BAM, CRAM, VCF, etc., created for a genomic dataset that can be JIT processed and/or stored by JIT or other similar system compilation and/or analysis may be performed with the ability to convert the data into one basic storage format for storage. Additional data may be associated and stored with the file, such as metadata and/or other information (which may be small) necessary to reproduce all other required formats using accelerated genomic data processing. Such additional information may include one or more of the following: a list of formats to be reproduced, data processing commands for reproducing each format, a unique identifier (e.g. URL or MD5/SHA hash function) of the reference genome, settings for various parameters, e.g. for mapping, alignment, sorting, variant determination, and/or any other processing as described herein, randomization seeds for processing steps, e.g., using pseudorandomization to deterministically reproduce the same results, user interface, and the like.

[00600] In various cases, data to be stored and/or retrieved in a JIT or similar dynamic processing and/or translation analysis system may be presented to the user or other applications in a variety of ways. For example, one option is to have a JIT analysis store in a standard or custom JIT object file format, such as storing and/or retrieving as a SAM, BAM, CRAM, or other custom file format, and providing custom means for quickly converting a JIT object to the required format (eg, in local temporary storage 200) using the accelerated processing described herein. Another option is to present the appearance of multiple file formats such as BCL, FASTQ, SAM, BAM, CRAM, VCF, etc. to the user and user applications such that the file system accesses the various file formats using a JIT procedure, and thus Thus, it was necessary to save only one type of file, and from this file it was possible to generate all other files on the fly. Another option is to create a custom tool that otherwise accepts special file formats (BCL, FASTQ, SAM, BAM, CRAM, VCF, etc.) that can instead be represented as a JIT object, and can automatically invoke JIT analysis. so that when called, receive data in the required data format, for example, BCL, FASTQ, SAM, BAM, CRAM, VCF, etc. automatically.

[00601] Accordingly, JIT routines are useful for providing access to multiple file formats, such as BCL, FASTQ, SAM, BAM, CRAM, VCF, etc., from a single file format by quickly processing the underlying stored compressed file format. In addition, JIT technology remains useful even if only one file format needs to be accessed, since compression is still achieved directly against the storage format being accessed. In such a case, the underlying file storage format may be different from the file format being accessed, and/or may contain less metadata, and/or may be compressed more efficiently than the format being accessed. In addition, in such a case, as discussed above, the file is compressed before saving and decompressed when extracted, for example automatically.

[00602] In various cases, the JIT analysis methods presented herein can also be used to transmit genomic data over the Internet or other network to minimize transmission time and consume less network bandwidth. In particular, one compressed base file format may be stored in a single storage application and/or one or more formats may be accessed through decompression and/or accelerated processing of genomic data. Similarly, in a transfer application, only one compressed base file format needs to be transferred, for example, from a source network host to a destination network host, for example, when the base format may be selected first to produce the smallest compression size, and/or when all required formats files can be generated at the destination node by or for processing genomic data, for example on the fly. This ensures that only one data format needs to be used for storage and/or transmission, from which various other file formats can be derived.

[00603] Accordingly, as shown in FIG. 40A, hardware and/or quantum accelerated processing of genomic data as described herein can be used both at the source node of the network to form and/or compress the base format for transmission, and at the destination node to decompress and/or form other required file formats using accelerated processing of genomic data. However, JIT or other dynamic translation analysis continues to be useful in a transmission application even if only one of the source or destination nodes uses hardware and/or quantum-accelerated genomic data processing. For example, a data server that sends large volumes of genomic data may use hardware and/or quantum-accelerated processing of genomic data to generate a compressed base format for transmission to various destinations. In such cases, each destination may use slower software processing of genomic data to generate other required file formats. Therefore, although the speed benefits of JIT analysis are weakened at the destination node, transmission time and network utilization are still effectively reduced, and the source node is able to efficiently service many such transmissions through its associated hardware and/or quantum-accelerated genomic data processing engine.

[00604] Additionally, in another example, a data server that accepts downloads of large volumes of genomic data, for example, from various sources, may use hardware and/or quantum-accelerated processing and/or storage of genomic data, while various source nodes may use slower software running on the CPU/GPU to generate a compressed base file format for transmission. In an alternative implementation, hardware and/or quantum-accelerated genomic data processing may be used by one or more intermediate network nodes, such as a gateway server, between the source nodes and destination nodes to transmit and/or receive genomic data in a compressed base format according to with JIT or other methods of dynamic translation analysis, thus taking advantage of reduced transmission time and network usage without overloading these intermediate network nodes with excessive software processing.

[00605] Therefore, as shown in FIG. 40A, in certain cases, local computing resource 100 may include a compiler 130, such as a JIT compiler, and may also include a compressor 150 that is configured to compress data, such as generated and/or received primary and/or secondary processing data (or tertiary data) that may be compressed, for example, before transmission over local network 10 and/or cloud network 30, and/or hybrid network 50, such as in a JIT parsing procedure, and which may be decompressed after transmission and/or before use .

[00606] As described above, in various cases, the system may include a first integrated circuit and/or quantum circuit 100, for example, to perform a mapping, alignment, sorting, and/or variant determination operation to generate one or more mapped, aligned, sorted, containing duplicates and/or quantified variants of these results. In addition, the system may include another integrated circuit and/or quantum circuit 300, for example, to use the results data to perform one or more analyzes using genomic and/or bioinformatics pipelines, such as tertiary processing. For example, the resulting data generated by the first integrated and/or quantum circuit 100 can be used, for example, by the first or second integrated and/or quantum circuit 300 when performing a further genomic and/or bioinformatics processing pipeline. Namely, secondary processing of the genomic data may be performed by the first hardware and/or quantum-accelerated processor 100 to generate the result data, and tertiary processing may be performed on the result data, such as where further processing is performed by a CPU, and/or GPU , and/or CPU 300, which is operatively connected to the first integrated circuit. In such a case, the second circuit 300 may be configured to perform tertiary processing of the genomic variation data obtained by the first circuit 100. Accordingly, the result data obtained from the first integrated circuit acts as an analysis engine driving the further processing steps described herein document regarding tertiary processing, for example, using a second integrated and/or quantum processing circuit 300.

[00607] However, the data generated at each of these primary and/or secondary and/or tertiary processing steps can be enormous, requiring very large resource and/or memory costs, for example for storage, either locally 200 or remotely 400. For example, in the first stage of primary processing, generated nucleic acid sequence data 110, for example in a BCL and/or FASTQ file format, can be received 120, for example from SNP 110. Regardless of the file format of this sequence data, it can be used in the protocol secondary processing as described herein. The ability to ingest and process raw sequence data directly from SNP, for example in BCL and/or FASTQ file format, is very useful. In particular, instead of converting a sequence data file from the SNP, for example in the BCL file format, to a FASTQ file, the file can be received directly from the SNP, for example as a BCL file, and can be processed, for example, by being accepted and processed by the JIT system. for example, on the fly, into a FASTQ file and can then be processed as described herein to produce mapped, aligned, sorted, non-duplicate and/or variant-qualified results that can then be compressed, for example, into a file SAM, BAM and/or CRAM, and/or may be further processed, for example, using one or more of the described genomic tertiary processing pipelines.

[00608] Accordingly, once such data is created, it must be stored in some way. However, such storage is not only resource-intensive, but also expensive. Namely, in a typical genomic protocol, the resulting sequenced data is stored as a large FASTQ file. Then, after processing, for example by applying a mapping and/or alignment protocol, a BAM file is created, which is also typically stored, increasing the cost of storing genomic data, for example, due to the need to store both FASTQ and BAM files. In addition, after processing the BAM file, for example by applying a variant definition protocol, a VCF file is created, which also usually needs to be saved. In such a case, to adequately provision and use the generated genetic data, it may be necessary to store all three files, FASTQ, BAM and VCF, locally 200 or remotely 400. Additionally, the original BCL file may also be stored. This storage is inefficient, memory intensive and expensive.

[00609] However, the processing power of the hardware and/or quantum processing architecture implemented in the present invention, along with JIT compilation, compression, and storage, greatly mitigates inefficiencies, resource costs, and overheads. For example, given the implementation methods and processing speeds achieved by the proposed accelerated integrated circuits, for example, to convert a BCL file to a FASTQ file and then convert the FASTQ file to a SAM file or a BAM file, and then convert the BAM file to a CRAM file and/or VCF file and vice versa, the proposed system greatly reduces the amount of computing resources and/or file sizes required to efficiently process and/or store such data. The advantages of these systems and methods are further enhanced by the fact that only one file format needs to be stored, eg BCL, FASTQ, SAM, BAM, CRAM and/or VCF, from which all other file formats can be retrieved and processed. In particular, only one file format needs to be stored, and from such a file can be quickly, for example, on the fly, generated by any of the other file formats, in accordance with the methods described herein, for example, in a just-in-time compilation format, or JIT.

[00610] For example, typical prior art methods require large amounts of computing resources, such as server farms and large memory banks, to process and store the FASTQ files generated by the SNP sequencer. In particular, in a typical scenario, once a large FASTQ file has been generated by the CPS, a server farm will need to be used to ingest and convert the FASTQ file into a BAM file and/or a CRAM file, which may take up to a day or more to process. However, the created BAM file itself must also be saved, which requires additional time and resources. Similarly, a BAM or CRAM file may be processed to generate a VCF file, which may also take another day or more, and this file will also need to be saved, thereby expending additional resources and incurring further costs. More specifically, a typical FASTQ file for the human genome consumes about 90 GB of memory per file. Likewise, a typical human genome BAM file can be around 160 GB. You may also need to save a VCF file, although these files are quite small compared to FASTQ and/or BAM files. Secondary processing may also produce SAM and CRAM files, and these may also need to be saved.

[00611] Prior to the technologies proposed herein, it was computationally intensive to move from one step to another, for example, from one file format to another, and therefore typically had to store all the data for those file formats. Part of the reason for this is that if the user ever wanted to go back and restore one or more of these files, it would require a huge amount of computing resources and time to rewind the processes involved in restoring the various files, thereby incurring high monetary costs. Additionally, these files are compressed before being stored, and such compression can take from about 2 to about 5 to about 10 hours or more, with about the same amount of time required to decompress before reuse. Due to these high costs, typical users will not compress such files before saving and will also tend to save all two, three or more file formats, e.g. BCL, FASTQ, BAM, VCF, incurring increased costs over a longer period of time .

[00612] Accordingly, JIT protocols used in the present invention can take advantage of the increased processing speeds achieved by proposed hardware and/or quantum accelerators to realize improved efficiency while reducing both processing and storage time and costs. Instead of storing 2, 3, or more instances of the same shared data in different file formats, only one file format needs to be stored, and any of the other file types can be recovered on the fly, such as using the accelerated processing platforms discussed in this document. In particular, the proposed devices and systems can easily revert from storing a FASTQ file to a BCL file, or progressing to a BAM file and then on to a VCF file, for example, in less than 30 minutes, for example within 20 minutes, or about 15 or 10 minutes or less.

[00613] Therefore, through the use of pipelines and the processing speed provided by the hardware/quantum processing engines described herein, one file format needs to be stored, and other file formats can be easily and quickly generated from it. Therefore, instead of having to store all three file formats, one needs to store one file format from which the remaining file formats can be generated, for example, on the fly, just in time for further processing steps required by the user. Therefore, the system can be easily used in such a way that the user simply interacts with a graphical user interface, for example, presented on an associated display of the device, for example, so that the user can obtain the desired format representation by pressing a button FASTQ, BAM, VCF, etc. .d. in the GUI, where one or more acceleration engines could, in the background, perform the acceleration steps necessary to restore the requested file in the requested file format from the stored file.

[00614] Typically, one or more compressed versions of a BCL, FASTQ, SAM, BAM, CRAM, and/or VCF file will be stored along with a small metafile that contains all of the configurations used by the system to create the compressed and/or saved file. Such metafile data contains details of how a particular file format, such as FASTQ and/or BAM, was generated, and/or what steps would be needed to go back or forward to generate any of the other file formats. This process is described in more detail later in this document. Likewise, a process can be advanced or reverted using configuration stored in a metafile. This can reduce storage needs and economic costs by 80% or more if the computing function is linked to storage functions.

[00615] Accordingly, in view of the above, and as shown in FIG. 40A, a cloud server system for analytics and data storage is proposed. For example, using a cloud-accessible northbound system as described herein, a user can connect to a storage device, for example, to store input data. For example, a remote user may access the system to enter genomic and/or bioinformatics data into the system, for example, to store and/or process it. In particular, a remote user of the system, for example using local computing resource 100, can access the system 1 to download genomic data, such as one or more sequenced genomes of one or more individuals. As described in detail below, the system may comprise a user interface, for example having access to a suitably configured API, which will allow the user to access the BioIT platform to upload data to be processed, control processing parameters and/or download output data, such as result data , from the platform.

[00616] Namely, the system may contain an API, for example, of an S3 object or "S3 type" that allows access to one or more system memories for storing 400 and/or receiving stored files. For example, there may be a cloud accessible API object, such as where the API is configured to store data files in the cloud 50, such as one or more storage buckets 500, such as an S3 bucket. Accordingly, the system may be configured to provide a user with access to remotely stored files, for example, through an S3 API or an S3 type, for example, by accessing the API through a cloud interface on a personal computing device.

[00617] Such an API may therefore be configured to provide access to a cloud 50 to thereby connect the user to one or more cloud servers 300 described herein, for example, to upload and/or download a given storage file, for example, to make it is accessible between the cloud server 300 and the local hard drive 100. This may be useful, for example, to allow a remote user to provide, access, and/or download data to or from server 300 and then perform one or more applications and/or computations on that data, either locally 100 or on server 300. and then calling an API to send the converted data back to or from the cloud 50, for example, for storage 200 and/or further processing. It is especially useful for extracting, analyzing and storing genomic data.

[00618] However, typical cloud data storage, such as "S3" storage, is expensive. This cost increases when storing large quantities of genomic and/or bioinformatics data, where such costs become prohibitive. In addition, the time required to record, upload and/or download data for use, for example, either locally 100 or remotely 300 and/or storage costs 400 also make such expensive cloud storage systems less attractive. The proposed solutions described in this document overcome these and other shortcomings.

[00619] Specifically, rather than moving to a typical "S3" storage object or other typical cloud object, the API proposed herein provides an alternative S3-compatible API that can be implemented with the ability to reduce transfer speed and/or cost data storage. In such a case, when a user wishes to save a file, rather than moving to a typical cloud, e.g., S3, API, an alternative service API system, e.g., the proprietary S3-compatible API described herein, will launch a computation instance, e.g. an instance of a CPU and/or FPGA system that will compress the file, generate a metadata index to indicate what the data is and/or how the file was generated, etc., and then store the compressed file using S3-compatible bucket 400 storage Accordingly, this document presents a cloud 50 service that uses a compute instance 300 that can be launched by an alternative API to compress the data before storing 400. In such a case, what is saved is therefore not the actual file, but rather a compressed version of the original file. file.

[00620] Namely, in such a case, the initial file may be in a first format that can be loaded into the system via a proprietary S3-compatible API that accepts a file, for example an F1 file, and can then perform a calculation function on that file and/ or compress the file, for example, through a suitably configured CPU/GPU/FPGA processing engine 300, which then prepares the compressed file for storage as a compressed file, such as an F1 compressed file. However, when a compressed and stored file needs to be retrieved, it can be decompressed and then the decompressed file can be sent to the user. The advantage of this accelerated compression and decompression system is that storing a compressed file 400 results in incredible storage cost savings, a benefit made possible by the computation and/or compression functionality achieved by the systems described herein.

[00621] Therefore, due to the fast and efficient computation and/or compression functionality achieved by the present systems, the user does not even need to know that the file is compressed before storage and subsequently decompressed after storage and presented in the user interface. In particular, the system operates so quickly and efficiently that the user does not need to be aware of the multiple compression, computation and/or decompression steps that take place when storing and/or retrieving the user's requested data, all of which occurs in a seamless and timely manner. However, the fact that the presented storage system will be less expensive and more efficient than previous storage systems will be obvious.

[00622] Accordingly, in view of the above, object-oriented storage services, where storage services can be provided at a lower cost by combining instance computation and/or compression together with storage functionality. In such a case, the typical storage cost can be replaced by a computation cost that is offered at a much lower level because, as discussed herein, the computation cost can be reduced by an accelerated implementation such as an FPGA and/or quantum computing platform 300 as described herein. Therefore, the accelerated platforms described herein can be implemented as a fast and efficient storage and retrieval system that allows compressed data to be quickly stored, which can be both compressed and stored, and quickly decompressed and retrieved at significantly lower cost and cost. greater efficiency and speed. This is particularly useful when it comes to storing 400 genomic data, and is compatible with the just-in-time processing functionality described above herein. Therefore, in accordance with the devices, systems and methods described herein, an object storage service may be provided, where the storage service implements fast compression functionality, such as genomics-specific compression, for storing genomic processing result data.

[00623] More specifically, as shown in FIG. 40A, in one embodiment, the BioIT systems proposed herein may be configured such that the pipeline server system 300, such as a portion thereof, receives a request to an API, such as an S3-compatible API, that is configured to be operably connected to a database 400 data, which is configured to associate the initial file (F1) with a compressed version of the file (CF1), for example, based on associated metadata. Likewise, after the CF1 source files are decompressed and processed, the resulting results data file (F2) can then be compressed and saved as a CF2 file. Accordingly, when a file needs to be retrieved from the database 400, the server 300 has an API that has already associated the source file with the compressed file through appropriately configured metadata, so that when retrieval is requested, the system's Workflow Management Controller (WMS) will launch a compute instance 300 that will launch the appropriate compute instance to perform any necessary computations and/or decompress the file for further processing, transmission, and/or presentation to the requesting user 100.

[00624] Therefore, in various implementations, an example method may include one or more of the following steps in any logical order: 1) The request is received through an API, for example, an S3-compatible API, 2) the API communicates with the WMS, 3) The WMS populates the database and initiates the calculation instance(s), 4) the calculation instance(s) performs the necessary compression on the F1 file and generates characteristic metadata and/or other relevant file associations (X), e.g. file CF1 X1, 4) thereby preparing the data for storage 400. This process can then be repeated for files F2, F3, ..., Fn, for example, other processed information so that WMS knows how the compressed file was formed, as well as where and how it was preserved. It should be noted that a unique feature of this system is that access to stored data 400 can be provided to multiple other users 100 substantially simultaneously. For example, the compression systems and methods described herein are useful in conjunction with the BioIT platforms described herein such that, at any time during the processing process, result data can be compressed and stored in accordance with the methods described herein , and made available to other users with the appropriate permissions.

[00625] With respect to performing genomic analysis, user 100 may access the system 300 described herein, for example, through a genomic analysis API, such as S3 or an S3-compatible API, upload genomic data, for example, in a BCL file format, and/or or FASTQ or other file format, and thereby request the performance of one or more genomic operations, such as mapping, alignment, sorting, deduplication, variant identification, and/or other operations. System 300 receives a request to the Workflow Manager API, then the Workflow Manager system evaluates the incoming requests, indexes jobs, builds a queue, allocates resources, such as allocating instances, and builds a pipeline thread. Accordingly, once a request has been received, preprocessed, and queued, an instance allocator, such as an API, will spin up various job-specific instances, described in more detail later herein, according to the work designs. Therefore, after a job has been indexed, queued, and/or stored in the appropriate database 400, the worker thread manager will then retrieve data from storage 400, such as S3 or S3-compatible storage, restart the appropriate instance that retrieves the file, and will run the appropriate process on the data to complete one or more requested jobs.

[00626] In addition, in the case of a request to perform multiple jobs on the data that require multiple instances to be executed, after the first instance has completed its operations, the result data may be compressed and stored, for example, in a corresponding memory instance, for example, in a first database , such as an elastic or flexible storage device, to wait for other instances of the pipeline to spin up and retrieve result data for further processing, for example, in accordance with the systems and methods described herein. Next, as new requests arrive and/or current jobs continue to execute, the workflow management system will continually update the queue to allocate jobs to the proper instances through the instance allocator API to maintain efficient data flow through the system and execution of system processes.

[00627] Likewise, the system 300 may continuously receive the results data and store the data 200/400, for example, in the first and second databases before further processing and transmission, such as transmission back to the original requester 100 or a designated party. In certain cases, result data may be compressed, as described herein, before being stored 400 and/or sent. Further, as stated above, the generated results data files after compression may contain appropriate metadata and/or other associated data, and the results data may be labeled differently as it flows through the system, for example, going from an F1 file to an F1C file to a F2, to file F2C, etc. as data is processed and moved through the platform's pipeline, for example as specified by the File Associations API.

[00628] Accordingly, thanks to the proprietary, specialized APIs that are described in this manual, the system can have a common framework to which other services can be attached, and/or additional resources, such as instances, can be brought into play to provide seamless and efficient execution of conveyor operations. Likewise, compressed and stored result data files can be invoked as needed, causing the workflow manager to spin up the appropriate compute and/or database instances to decompress the result data for presentation to the requester. It should be noted that in various cases, said compute and compression instance, as well as said compute and decompression instance, may be one or multiple instances, and may be implemented as a CPU, FPGA, or hard-coupled CPU/FPGA, hard-coupled CPU/CPU, or hard-coupled FPGA/FPGA. In certain cases, one or more of these and other instances described herein may be implemented as a quantum processing device.

[00629] Accordingly, in view of what is described herein, in accordance with one aspect, an apparatus is provided for performing one or a plurality of functions in performing genomic sequence analysis operations. For example, after data has been received, for example, by a remote user 100, and/or stored 400 in a cloud system, the input data may be made available to the WMS and may be prepared for further processing, such as secondary analysis, the results of which may then be transmitted back to the local user 100, for example after compression, stored 400 and/or subjected to additional processing, such as tertiary processing, by the system server 300.

[00630] In certain cases, the secondary processing steps described herein for specific implementations may be performed by the local computing resource 100 and may be implemented by software and/or hardware, such as an additional external computing resource 200, where the computing resource 200 includes a core A CPU, such as from about 4 to about 14, has about 24 or more CPU cores, and may further comprise one or more FPGAs. The local additional external resource 100 may be configured to access a large storage unit 200, such as 120 GB of RAM, which access may be direct, such as through a direct connection between them, or indirect, such as through a communication-capable connection between them via local cloud network 30.

[00631] Namely, in a local system, data may be transferred to or from memory 200 via suitably configured SSD drives that are configured to write processing task data, such as genomics-related tasks to be processed, and read processed ones. results data from memory 200. In various implementations, local computing resource 100 may be communicatively coupled to sequencer 110, from which a BCL and/or FASTQ file may be retrieved, e.g., from the sequencer, and written to SSD drives directly, e.g. , through a suitably configured interconnect. Local computing resource 100 may then perform one or more secondary processing operations on the data. For example, in one embodiment, the local computing resource is a LINUX® server having 24 CPUs that can be coupled to a suitably configured FPGA configured to perform one or more of the secondary processing operations described herein.

[00632] Therefore, in various cases, local computing device 100 may be a "workstation" computing solution having a BioIT chipset that is configured to perform one or more of secondary and/or tertiary processing of genetic data. For example, as described herein, computing resource 100 may be associated with a PCIe card that is inserted into the computing device to thereby be associated with one or more internal CPU cores, GPUs, CPUs, and/or associated memories. In particular, components of the computing device 100 comprising processing devices, associated memories, and/or associated PCIe cards with one or more FPGA/ASIC chipsets therein may communicate with each other, all of which may be provided within a package, for example , in the form of a boxed version, which is typical in the art. More specifically, the boxed version may be configured for use in a workstation or, in various cases, it may be configured for use in a remotely accessible server rack. In other embodiments, CPU/FPGA/memory chipsets and/or associated ExpressCard interconnect cards may be interconnected within a next-generation sequencing device to form a single unit therein.

[00633] Accordingly, in one particular case, a desktop boxed version may contain multiple CPUs/GPUs/GPUs associated with one or more FPGA arrays, such as 4 CPUs/GPUs or 8, or 12, 16, 20, 22, 24 CPUs or more , which can be connected to 1 or 2 or 3 or more FPGAs, for example, in one package. Namely, in one specific case, a boxed version of a computing resource is proposed that contains 24 CPU cores, configured to change the configuration of the FPGA matrix, a database, for example, 128x8 RAM, one or more SSDs, for example, when the FPGA matrix is configured to at least partially changing the configuration between operations, for example, between performing mapping and alignment. Therefore, in such a case, the BCL and/or FASTQ files generated by the sequencing device 110 can be read into the CPU and/or transferred to the FPGA for processing, and its result data can be returned to the associated CPUs via SSD drives. Accordingly, in this embodiment, the local computing system 100 may be configured to offload various computationally intensive functions to the associated FPGA, thereby improving the speed, accuracy, and efficiency of bioinformatics processing. However, while the boxed solution 100 is useful, for example, on local equipment, it may not be suitable for access by multiple users who may be located remotely from the boxed version.

[00634] In particular, a cloud server 50 may be provided in various cases, such as when the server 300 can be accessed remotely. Accordingly, in specific cases, one or more integrated circuits (CPU, FPGA, CPU) described herein may be provided that can be accessed via a cloud-based interface 50. Therefore, in specific cases, it may be provided a boxed version of a computing resource in the form of a boxed version of an automated workstation, as described above, and the configuration of the boxed version can be made with the ability to transfer to the cloud and remote access to it. However, this configuration may not be sufficient to handle large volumes of traffic from remote users. Accordingly, in other cases, one or more integrated circuits described herein may be implemented as a server based solution 300, which may be configured as part of a server rack, for example, when a system accessible on the server is configured to be remotely accessible , for example via cloud 50.

[00635] For example, in one embodiment, there may be a computing resource, or local server 100, having one or more, for example, multiple, CPU and/or GPU and/or GPU cores and associated memories along with one or more FPGAs /ASICs described in the present invention. In particular, as stated above, one implementation may include a boxed version that includes from 18 to 20 to 24 or more CPU/GPU cores, the boxed version having an SSD, 128x8 RAM, and one or more BioIT - FPGA/ASIC circuits, and also includes a suitably configured communication module having transmitters, receivers, antennas, as well as capabilities for data exchange via WIFI, Bluetooth and/or cellular communications, which are configured to provide remote access to the out-of-the-box version . In this implementation, for example, when a single FPGA is provided, the FPGAs may be configured to be reconfigurable, eg, partially reconfigured, between one or a plurality of different stages of the genomic analysis pipeline.

[00636] However, in other cases, a server system is provided that may include from about 20 to 24, to 30, to 34, to 36 or more CPU/GPU cores and about 972 GB or more of RAM that may be associated with one or more, such as about two, or about four, or about six, or about eight or more FPGAs, which may be configured as described herein. For example, in one implementation, one or more FPGAs may be configured to change configuration, such as partial reconfiguration, between one or more different stages of the genomic analysis pipeline. However, various other implementations may provide a set of specialized FPGAs, for example, each specifically designed to perform a specific BioIT operation such as mapping, alignment, variant detection, etc., thereby eliminating the reconfiguration step.

[00637] Accordingly, one or more FPGA arrays may be provided in various cases, for example, when the FPGA arrays are configured to change configuration between different pipeline operations. However, in other cases, one or more FPGAs may be configured to specialize in performing one or more functions without the need to partially or completely configure them. For example, the FPGA arrays proposed herein may be configured to specialize in performing one or more computationally intensive operations in a BioIT pipeline, such as where one FPGA is provided that is specifically designed to perform a mapping operation, and another FPGA is provided that performs with the ability to perform the alignment operation, although, in some cases, a single FPGA may be provided that is configured to be partially reconfigurable between performing both mapping and alignment operations.

[00638] Additionally, other operations in the pipeline that may also be performed by configurable or dedicated FPGAs may include performing a BCL transform/permutation operation, a Smith-Waterman operation, an HMM operation, a local realignment operation, and/or various other variant determination operations. Likewise, various pipeline operations can be configured to be executed by one or more of the system's associated CPU/GPU/CPU. Such operations may be one or more computationally intensive pipeline operations, for example, to perform sorting, deduplication, and other variant determination operations. Therefore, it is possible to implement a comprehensive system with the ability to perform a combination of operations, partly using the CPU/GPU/CPU, and partly using hardware such as FPGA/ASIC systems.

[00639] Accordingly, as shown in FIG. 40B, in various implementations of cloud system 50, it may include a plurality of computing resources, including multiple instances and/or layers of instances, for example, where the instances and/or layers of instances are configured to perform one or more of the BioIT pipeline operations described in this document. For example, various instances of CPU/GPU/CPU and/or hard-wired integrated circuits may be provided to perform the specialized genome analysis pipeline functions proposed herein. For example, there may be different FPGA instances for performing specialized genomic analysis operations, such as an FPGA instance for performing mapping, another for performing alignment, another for performing local realignment and/or other Smith-Waterman operations, another for performing HMM operations, and etc.

[00640] Likewise, various CPU/GPU/CPU instances may be provided to perform specialized genomic analysis operations, such as CPU/GPU/CPU instances to perform signal processing, sorting, deduplication, compression, various variant detection operations, and the like. . In such cases, associated memory or memories may be provided, for example between different stages of a pipeline computation, to receive result data as they are computed, compiled and processed throughout the system, for example between different CPU and/or FPGA instances and/or layers thereof . In addition, it should be noted that the size of the various CPU and/or FPGA instances may vary depending on the computing needs of the cloud system and may vary from small to medium, large and very large, as well as the number of CPU/GPU/CPU instances may vary and FPGA/ASIC.

[00641] Additionally, as shown in FIG. 40B, the system may also include a workflow manager that is configured to schedule and direct the movement of data throughout the system and from one instance to another and/or from one memory to another. In some cases, the memory may be a plurality of memories that are instance-specific memories, and in other cases, the memory may be one or more memories configured to be elastic and therefore capable of switching from one instance to another, such as a switchable elastic block storage device. In still other cases, the memory may be non-instance specific and therefore non-connectivity capable of communicating with multiple instances, for example for elastic file storage.

[00642] In addition, the worker thread manager itself may be a specialized instance, for example a CPU/GPU/CPU core, that is specifically designed for and/or configured to determine which jobs need to be executed and when and what resources will be used in the process. execution of these jobs, as well as queuing jobs and routing them from resource to resource, for example, from instance to instance. The worker thread manager may comprise or may otherwise be implemented as a load estimator, and/or form an elastic control node, which is a dedicated instance that can be executed by a processor, such as a CPU/GPU/CPU core. In various cases, the workflow manager may have a database connected to it, which may be configured to manage all of the jobs that are to be processed, are being processed, or have been processed. Therefore, the WMS manager can be configured to detect and manage data flows throughout the system, determine how to allocate system resources, and when to deploy more resources.

[00643] As stated above, in certain cases, a workstation and/or server solution may be provided simultaneously, where the computing device contains a plurality of servers of X CPU cores of size Y, which are configured to feed data to one or more FPGAs size Z, where X, Y and Z are numbers that may vary depending on the processing needs of the system, but should be selected and/or otherwise configured to be optimal, e.g. 10, 14, 18, 20, 24, 30 etc. For example, typical system configurations are optimized to perform the BioIT operations of the system described herein. Namely, certain system configurations are optimized to maximize data flow through various CPU/GPU/CPU instances into various integrated circuits, such as FPGA, systems, where the size of the CPU and/or FPGA may vary depending on each other, based on from the processing needs of the system. For example, one or more of the CPU and/or FPGA may have a size that is relatively small, medium, large, very large, or extra large. More specifically, the system architecture can be designed such that the CPU/FPGA hardware is sized and implemented in such a way that it operates in an optimally efficient manner to keep both instance platforms busy throughout operation, such as when CPUs outnumber FPGAs in a ratio of 4 to 1, 8 to 1, 16 to 1, 32 to 1, 64 to 2, etc.

[00644] Therefore, while it is generally good for FPGAs to have great capabilities, data processing using high-performance FPGAs is inefficient and the system is fed with insufficient data to process. In such a case, one FPGA or part thereof can be implemented. In particular, ideally, the workflow management system directs data flow to specific CPUs and/or FPGAs that are configured to keep the system and its components constantly busy computing. For example, in one example configuration, one or more, such as 2, 3, or 4 or more, CPU/GPU/CPU cores may be configured to supply data to a small, medium, large, or very large FPGA or portion thereof. Specifically, in one embodiment, a specific CPU instance may be provided, for example, to perform one or more of the BioIT processing operations described herein, for example, where the CPU instance is accessible from the cloud and contains up to 4, 8, 16, 24, 30, 36 CPU cores, which cores may or may not be operably coupled to a portion of one or more FPGAs.

[00645] For example, a cloud-accessible server rack 300 may be provided, where the server contains a CPU core instance having from about 4 CPU cores to about 16 or about 24 CPU cores that are operably coupled to the FPGA instance. For example, an FPGA instance may be provided, such as where the average FPGA size is X, and the included FPGAs may be about 1/8X, X, 2.5X, and up to 8X in size, or even about 16X or larger. In various cases, additional CPU/GPU/CPU cores or FPGAs may be included and/or provided as a consolidated instance, such as when there is a large amount of data to process and when multiple CPU cores are selected to keep the FPGA busy at all times. Therefore, the ratio between CPU and FPGA can be adjusted by combining them in such a way as to optimize data flow, and thus the system can be designed to scale up or down depending on needs, for example, to minimize costs for optimal use in depending on worker threads.

[00646] However, when the CPUs do not generate enough work to keep the FPGA fully occupied and/or fully utilized, the configuration will not be ideal. Therefore, this document proposes a flexible architecture of one or more instances that can be directly linked together, or configured to be linked together in such a way as to adapt to the efficient operation of CPU/FPGA software/hardware so that the CPU/GPU/CPU optimally supplied the available FPGAs and/or part of them so that the platforms of both instances were constantly occupied. Accordingly, allowing access to such a system via the cloud will ensure that data fed into it is queued by the worker thread manager and routed to specific CPU/FPGA resources that are configured to receive and process the data in an optimally efficient manner.

[00647] For example, in some configurations, cloud-accessible instances may contain CPUs/GPUs/CPUs of different sizes in different quantities, and in addition there may be cloud-accessible instances that contain FPGAs (or ASICs) and/or CPUs of different sizes in different quantities. It can even be instances having combinations of these instances. However, in various iterations, the provided CPU/GPU/CPU and/or FPGA/CPU and/or mixed instances may have too many of one instance and/or too few of another instance to effectively operate the presented BioIT pipeline platforms described herein. Accordingly, this document proposes systems and architectures, flexible combinations thereof and/or methods for implementing them for efficiently forming and using genomic and/or bioinformatics processing pipeline platforms, such as those made available through the cloud 50.

[00648] In such systems, CPUs/GPUs/CPUs can be selected in such numbers and configured to handle less computationally intensive operations, and the number and configurations of FPGAs and/or CPUs can be tailored to solve computationally intensive tasks, such as when data is unobstructed are transmitted back and forth between CPU/GPU/CPU and FPGA/CPU instances. In addition, one or more memories may be provided for sorting data, eg result data, between various different procedural steps and/or between different types of instances to avoid significant instance latency periods. Namely, very little CPU/GPU usage is used during mapping and alignment because, due to the computationally intensive nature, these tasks are designed to be performed by hardware implementations. Likewise, during case definition, tasks can be broken down so that they are roughly equally distributed among the CPU/FPGA instances as their tasks, such as where Smith-Waterman and HMM operations can be performed by hardware and various other operations can be executed by software executed on one or more instances of the CPU/GPU/CPU.

[00649] Accordingly, the architecture parameters specified herein are not necessarily limited to the architecture once established, but rather the system is designed to have greater flexibility in the organization of its implementations and relies on a workflow manager that will determine which instances are active, when, how and for how long, and specifies which calculations are performed on which instances. For example, the number of CPUs and/or FPGAs that need to be activated and operatively coupled together must be selected and configured such that the activated CPUs and FPGAs, and their accompanying software/hardware, are kept optimally occupied. In particular, the number of CPUs and their operation must be configured to ensure that a given number of FPGAs, or a portion thereof, are always occupied so that the CPUs optimally and efficiently supply the FPGAs to skillfully support the operation of both instances and their components.

[00650] Therefore, in this way, the system's workflow management controller can be configured to access the workflow and organize and divide it so that tasks that can be more optimally performed by the CPU/GPU are directed to the number of CPUs required to perform these operations optimally, and those tasks that could be more optimally performed by an FPGA/ASIC/CPU were routed to a number of FPGAs required to perform those operations optimally. Elastic and/or efficient memory may also be included to efficiently transfer data of the results of these operations from one instance to another. In this way, a combination of machines and memories can be configured and combined so that it is optimally scaled based on the amount of work that needs to be done, as well as the configuration and use of instances to best perform the work efficiently and cost-effectively.

[00651] Namely, the cloud architectures described herein demonstrate that various known shortcomings of previous architectures can lead to inefficiencies that can be overcome by flexibly allowing more CPU/GPU/CPU cores to access all sorts of different hardware instances , for example, FPGAs, or parts thereof, that are organized more purposefully to be able to specifically allocate the proper instance to perform the intended functions optimally by being implemented in such a format. For example, the system may be configured to provide remote access to a larger portion of the available CPU/GPU instances so that they are kept busy, producing result data that can be optimally fed to the available FPGA/GPU instances to keep selected FPGA instances always busy. Therefore, it is desirable to provide a structured architecture that is as efficient as possible and is always occupied. It should be noted that configurations where too few CPUs supply too many FPGAs such that one or more FPGAs are underutilized are inefficient and should be avoided.

[00652] In one implementation, as shown in FIG. 40B, the architecture may be configured to virtually include several different layers or layers, such as a first layer having a first number of X CPU cores, such as 4 to about 30 CPU cores, and a second layer having 1 to 12 or more instances FPGA matrices, the size of which can vary from small to medium or large, etc. You can enable a third layer of CPU cores and/or a fourth layer of additional FPGAs, etc. Thus, on the cloud server 300 there are multiple instances available, for example, instances that simply contain a CPU or GPU, and/or instances that contain FPGAs, or combinations thereof, for example, at one or more of the layers described herein. Accordingly, in a similar way, the architecture can be flexibly or elastically organized so that the most intensive special computing functions are performed by hardware instances or CPUs, and those functions that can be performed by CPUs are sent to the appropriate level for general processing purposes, with a number of instances as necessary The CPU/FPGA can be scaled up or down by the system as needed.

[00653] For example, the size of the architecture can be elastic to minimize system overhead while maximizing optimal use of the system. Namely, the architecture can be designed to maximize efficiency and reduce latency by combining different instances at different virtual layers. In particular, a plurality, for example a significant portion and/or all, of layer 1 CPU/GPU instances may be configured to supply different layer 2 FPGA instances that are specifically configured to implement special functions, such as a mapping FPGA and an alignment FPGA . At another level, one or more additional (or as many as at level 1) CPUs may be provided, for example, to perform sorting and/or duplicate removal operations and/or various variant determination operations. Moreover, one or more additional FPGA layers may be configured to perform Needleman-Wunsch, Smith-Waterman, and HMM variant determination operations, and the like. Therefore, the first level of CPU can be used to form the initial level of genomic analysis, for example, performing general processing steps, including queuing and preparing data for further pipeline analysis, and the data, after processing by one or multiple CPUs, can be fed to one or more further layers of specialized FPGAs, for example, where the FPGA instance is configured to perform computationally intensive functions.

[00654] Thus, in a particular implementation, the CPU/GPU instances in the pipeline send their data, once prepared, to one or two Layer 2 FPGA instances for mapping and alignment. Once the mapping is complete, the resulting data may be stored in memory and/or then provided to an alignment instance, where alignment can be performed, for example, by at least one layer 2 FPGA instance dedicated to alignment. Similarly processed mapped and aligned data can then be stored in memory and/or sent to a Level 3 CPU instance for further processing, which may be the same as Level 1 or a different instance, for example to perform less resource-intensive functions processing genomic analysis, for example, to perform a sorting function. Additionally, after the Tier 3 CPUs have completed their processing, the resulting data can be routed either back to the Tier 2 FPGA instances or to the Tier 4 FPGA instances, for example, for resource-intensive further genomic processing functions such as Needleman processing functions -Wunsch (NW), Smith-Waterman (SW), for example, in an FPGA instance specifically designed for NW or SW. Likewise, after SW analysis is performed, for example, by an FPGA dedicated to the SW, the processed data can be sent to one or more associated memories and/or further down the processing pipeline, for example, to an instance of the CPU and/or FPGA layer 4 or 5, or back to level 1 or 3, for example to perform HMM analysis and/or variant determination, for example in a dedicated FPGA and/or a further level CPU processing core.

[00655] Likewise, latency and efficiency issues can be overcome by combining all kinds of different instances at one or more different layers to provide a pipeline platform for genomic processing. Such a configuration may involve more than scaling and/or combining instances; instances may be configured to specialize in performing specific functions. In such a case, the mapping FPGA instance only performs mapping, and similarly, the alignment FPGA instance only performs alignment, etc., instead of one instance performing the processing in the pipeline from start to finish. Although in other configurations, one or more FPGAs can be at least partially reconfigured, for example between executing pipeline tasks. For example, in certain embodiments, because the genomic analysis to be performed therein is a multi-step process, the FPGA code may be configured to change mid-processing, e.g., when the FPGA completes a mapping operation, it may be reconfigured to perform one or more of the alignments, variant definitions, Smith-Waterman, HMM, etc.

[00656] Consequently, a pipeline manager, such as a workflow management system, can manage a queue of genomic processing requests generated by tier 1 CPU instances to break them down into discrete jobs, aggregate them, and route them to the appropriate job-specific CPU, and then to specific to specify FPGA instances for further processing, e.g., mapping and/or alignment, e.g., at layer 2, wherein the processed mapped and aligned data can be sent back or forth to the next CPU/FPGA result data processing layer, e.g., to perform various steps in the option definition module.

[00657] For example, a case determination function can be broken down into multiple operations that can be performed in software, then routed to Smith-Waterman and/or HMM processing in one or more FPGA hardware instances, and then sent to the CPU for continuing variant determination operations, such as when the entire platform is elastically and/or efficiently sized and implemented to minimize the cost of expensive FPGA instances while maximizing utilization, maximizing latency, and therefore optimizing operations. Accordingly, fewer hardware instances are thus required given their absolute processing capabilities and hardware implementation specificity, and therefore the number of FPGAs relative to the CPU can be minimized and their utilization, e.g., FPGAs can be maximized, and therefore the system can be optimized to keep all instances always busy. This configuration is an optimal design for genomic processing analysis, especially for mapping, alignment, and variant detection.

[00658] An additional structural element that may be included, such as an auxiliary device, in the pipeline architecture described herein is one or more effective memory modules that may be configured to function to provide storage of blocks of data, such as result data , as they are transferred throughout the conveyor. Accordingly, one or more elastic block data stores (EBDS) and/or more efficient (flexible) block data storage modules can be inserted between one or more processing layers, for example, between different instances and/or instance levels. In such a case, the storage device may be configured such that as data is processed and information is retrieved, the processed results can be directed to the storage device for storage before being sent to the next processing level, for example by a dedicated FPGA processing module. The same storage device may be used between all instances, or multiple storage devices may be used between different instances and/or levels of instances, for example, to store and/or compile and/or queue result data. Accordingly, one or more memories may be provided so that different instances of the system can be associated with and/or have access to the same memory so that they can view and access the same or similar files. Therefore, there may be one or more elastic memories (memories configured to be associated with multiple instances sequentially) and/or effective memories (memories configured to be associated with multiple instances simultaneously) through which different instances of the system can be readable and writing to the same or similar memory.

[00659] For example, in one implementation example relevant to configurations using such elastic memories, before sending data directly from one instance and/or from one processing layer to another, the data may be directed to an EBDS or other storage device or structure, for example, a block of effective memory to be stored and then directed to the appropriate hardware processing module. Namely, a block storage module can be attached to a node as a storage device, where data can be written to a block data store (BDS) for storage in one layer, the BDS can be switched to another node to forward the data to the next processing layer. Thus, one or more, eg multiple, BDS modules may be included in a pipeline and configured to switch from one node to another to participate in the movement of data throughout the pipeline.

[00660] Additionally, as noted above, a more flexible file storage device may be used, such as a device that is configured to communicate with one or more instances simultaneously, for example, without having to switch from one to another. In a similar way, the system can be elastically scaled at each level of the system, for example, where each level in it can differ in the number of nodes for processing data at that level, and after processing, the result data can be written to one or more associated EBDS devices, which can then be switched to the next level of the system to make the stored data available to the next level of processors to perform their specific tasks at that level.

[00661] Accordingly, there are multiple stages in a processing pipeline, for example at its service nodes, at which data is prepared for processing, such as pre-processing, and once prepared, the data is sent to appropriate processing instances at one level where result data can be generated , the resulting data can then be stored, for example, on an EDS device, queued and prepared for the next stage of processing by switching to the next node of instances, and forwarded to the next instance for processing using next-order FPGA and/or CPU processing instances where they can further result data can be generated, and again once generated, the result data can be sent back to the previous or forward to the next EDS level for storage before moving on to the next processing stage.

[00662] Specifically, in one particular implementation, the flow through the pipeline may look like this: CPU (eg, 4 CPU cores, or C4 instance): data prepared (queued and/or stored); FPGA (for example, 2XL FPGA - 1/8 of the entire server, or F1 instance): Mapping, temporary storage; FPGA (e.g. 2XL FPGA - 1/8 of the entire server, or F1 instance): alignment, temporary storage; CPU: sorting, temporary storage; CPU: removal of duplicates, temporary storage; CPU: determination of options 1, temporary storage; FPGA (e.g. F1 or 16XL, or F2 instance): Smith-Waterman, temporary storage; FPGA (e.g. instance F1 or F2): HMM, temporary storage; CPU: determination of options 2, temporary storage; CPU: VCGF, temporary storage, etc. Additionally, a workflow management system may be included to control or direct the flow of data through the system, for example, where the WMS may be implemented on a CPU core, such as a 4-core CPU, or a C4 instance. It should be noted that one or more of these steps may be performed in any logical order and may be implemented by any suitably configured resource, for example, implemented in software and/or hardware in any number of different combinations. And it should be noted that any of these operations may be performed on one or more CPU instances and one or more FPGA instances at one or more theoretical processing levels, for example, to form the BioIT processing described herein.

[00663] As noted, a worker thread manager may be included, for example, when WMS is implemented on one or more CPU cores. Therefore, in various cases, a WMS may have a database associated with it during operation. In such a case, the database contains various operations and jobs to be queued, pending jobs, and a history of all jobs, previous or currently to be executed. Therefore, the WMS monitors the system and database to identify any new jobs that need to be completed. Consequently, when pending jobs are identified, the WMS initiates a new parsing protocol on the data and sends it to the corresponding instance node. Accordingly, the worker thread manager keeps track of and knows where all the input files are located, or are stored, processed, or need to be stored, and therefore directs and commands instances of the various processing nodes to access the corresponding files at a given location, to begin reading the files, to begin implementing processing instructions and where to record result data. And, therefore, the WMS guides the systems regarding the transfer of data to subsequent processing nodes. The WMS also determines where new instances need to be launched and brought into production to enable dynamic scaling of each stage or level of processing. As a result, the WMS identifies, organizes and routes discrete jobs to be executed at each level and then routes the result data to be written to memory for storage, and upon completion of the job, starts another node, reads the next job and performs the next iterative operation.

[00664] Likewise, input jobs can be distributed across many different instances, which can be scaled, for example, independently or collectively to include fewer or more instances. These instances can be used to create nodes to better balance resource usage, where such instances may represent a partial or full resource. The workflow manager may also direct and control the use of one or more memories, for example, between processing steps described herein. The different instances may also include complementary programming so that they can exchange data with each other and/or different memories for server virtualization. The WMS may also include a load estimator for elastic management of node utilization.

[00665] Additionally, with respect to the use of memories, one or more EBDS or other suitably configured data and/or file storage devices may be attached to one or more of different nodes, for example, between different instance levels, for example, for temporary storage between all sorts of different processing steps. Therefore, the storage device may be a single storage device configured to associate with all of the different instances, such as an effective memory unit such as elastic file storage, or may be a plurality of storage devices, such as one device per instance type, that is configured to switching between instances, such as elastic block storage. Accordingly, in a similar manner, each level of processing instances and/or memory can be elastically scaled as needed, for example, between each of different nodes or levels of nodes, for example, to process one or more genomes.

[00666] In the form of architecture described herein, one or multiple genomes may be introduced into the system for processing, for example, from one or more lanes of a next-generation sequencer flow cell, as shown in FIG. 1. Namely, the provision of a cloud server system 300, which is described herein, will allow multiple jobs to be accumulated and/or queued for processing, which can be processed by all sorts of different instances of the system simultaneously or sequentially. Therefore, the pipeline may be configured to support multiple jobs processed by the virtual matrix processor, which are associated with suitably configured storage devices to facilitate efficient processing of data from one instance to another. In addition, as indicated, a simple storage device may be provided that is capable of being associated with multiple different instances, for example, simultaneously. In other cases, the memory device may be an elastic type memory device that may be configured to associate with the first instance, for example, at one time, and then reconfigure and/or unlink the first instance and switch to the second instance.

[00667] Therefore, in one implementation, one or more elastic block storage devices may be included, and the system may be configured to include a switch control mechanism. For example, a switch controller may be included configured to control the operation of such storage devices as they are switched from one instance to another. The configuration may be configured to allow data to move through a pipeline of specialized processors, thereby increasing system efficiency, e.g., across all instances, e.g., by passing data flow through the system, allowing each layer to scale independently and trigger as needed to scale efficiently .

[00668] In addition, the workflow management system algorithm may be configured to determine the number of jobs, the number of resources to process those jobs, the order of processing, and direct the flow of data from one node to another by tossing or switching one or more flexible switching devices, and where required, can deploy additional resources to cope with growth in the work process. It should be noted that this configuration can be done to avoid copying data from one instance to the next, which is inefficient and takes too much time. Instead, by switching elastic storage from one set of instances to another, for example by dragging it from one node and attaching it to a second node, system efficiency can be significantly improved. Additionally, in various cases, one or more elastic file storage devices, such as a single storage device, capable of being associated with multiple instances without having to switch from one to another, can be used instead of EBSD, to further improve data transfer between instances, making the system even more effective. Additionally, it should be noted that, as stated earlier in this document, in another configuration the CPU architectures may be directly coupled to each other. Likewise, different FPGAs can be directly connected together. And, as stated above, the CPUs can be directly coupled to the FPGA, for example when the coupling is via a hard-coupling interface as described above.

[00669] Accordingly, with regard to the user storing and accessing generated result data, all generated result data does not need to be stored, and this applies to the entire system. For example, the generated result data will typically be in a specific file format, such as BCL, FASTQ, SAM, BAM, CRAM, VCF. However, each of these files is large in size, and storing them all will take up a lot of memory, which can be a huge expense. However, the advantage of the presented devices, systems and ways of using them is that all these files do not need to be stored. Instead, given the high processing speeds and rapid rates of compression and decompression achievable by the system components and methods, only one file format needs to be stored, such as a compressed file format in, for example, a cloud database 400 . Namely, you need to store only one data file format, from which, by implementing devices and methods of the system, you can obtain all other file formats. And, due to the fast rates of compression and decompression achieved by the system, it is typically a compressed file, such as a CRAM file.

[00670] Specifically, as shown in FIG. 40A, in one implementation, a user of local computing resource 100 may upload data, such as genomic data, such as a BCL and/or FASTQ file, to the system via cloud 50 for ingestion by cloud computing resource, such as server 300. Server 300 will then either temporarily store the data 400 or will begin processing data in accordance with a job request from user 100. While processing input data, computing resource 300 will thereby generate result data, for example in a SAM or BAM and/or VCF file. The system may then save one or more of these files, or may compress one or more of these files and save them. However, to reduce costs and make more efficient use of resources, the system may store a single, for example compressed, file from which all other file formats can be generated, for example, using the devices and methods described herein. Accordingly, the system is configured to generate data files, eg, results data, that can be stored in an associated server 300 database 400 that is accessible via the cloud 50 in a cost effective manner.

[00671] Accordingly, using local computing resource 100, a user of the system can log in and access server 300 through cloud 50, upload data to server 300 or database 400, and request that one or more jobs be performed on that data. The system 300 will then execute the requested jobs and store the results data in the database 400 . As noted, in particular cases, the system 300 will store the generated results data in a single file format, such as a CRAM file. Additionally, by clicking a button, the user can access the saved file, and after that, with another click of the button, can access all other file formats. For example, in accordance with the methods described herein, and given the system's fast processing capabilities, these files will then be processed and generated behind the scenes, for example, on the fly, thereby reducing both processing time and load, as well as storage costs , for example, when the computation and storage functions are coupled to each other.

[00672] In particular, there are two parts of this efficient and fast storage process that are enabled by the accelerated operations described herein. More specifically, since various mapping, alignment, sorting, deduplication, and/or variant detection processing operations can be implemented in a hardware and/or quantum processing configuration, rapid generation of result data in one or more file formats can be achieved. In addition, thanks to the tightly coupled architectures described in this document, seamless compression and storage of result data, for example in FASTQ, SAM, BAM, CRAM, VCF file format, is also achieved.

[00673] Moreover, due to the accelerated processing provided by the system devices, and due to their seamless integration with associated storage devices, data resulting from system processing operations that are to be stored can be efficiently compressed before storage and decompressed after storage. Due to this efficiency, storage costs and/or losses associated with unpacking files before use are reduced. Accordingly, due to these advantages, the system can be configured to seamlessly compress and store only one type of file, with on-the-fly recovery of any other type of file as needed or required by the user. For example, a BAM file, or an associated compressed SAM or CRAM file, can be stored, and the rest can be generated from that file, for example, in the forward or backward direction, for example, to play a VCF file or a FASTQ file, or a BCL file, respectively.

[00674] For example, in one embodiment, a FASTQ file may be initially entered or otherwise generated into the system and then stored. In this case, when moving in the forward direction, you can obtain the checksum of the file. Similarly, after receiving the resulting data, another checksum can be generated when going back. These checksums can then be used to ensure that any further file formats to be generated and/or recreated by the system back and forth will exactly match yet another and/or its compressed file format. Likewise, one can ensure that all the required data is stored in the most efficient manner and the WMS knows exactly where this data is stored, in what file format, what was the original file format and from this data the system can restore any file format in an identical manner by moving forward or back between file formats (after the template is initially generated).

[00675] Therefore, the advantage in just-in-time compilation speed is provided in part by hardware or quantum implementation of the generation of the corresponding files, for example, when generating a BAM file from a previously generated FASTQ file. In particular, compressed BAM files, including SAM and CRAM files, are typically not stored in the database due to the added time involved in decompressing the compressed stored file before processing. However, the JIT system allows you to do this without significant losses. More specifically, by implementing the devices and processes described herein, not only can generated sequence data be quickly compressed and decompressed, for example, almost instantly, but it can also be stored efficiently. In addition, from a saved file, no matter what format it was saved in, you can restore any other file format in a matter of seconds.

[00676] Therefore, as shown in FIG. 40C, when hardware- or quantum-accelerated platforms perform various secondary processing procedures such as mapping and alignment, sorting, deduplication, and variant detection, an additional compression step may be performed, such as an all-in-one process, before storing in condensed form. Thereafter, when the user needs to parse or otherwise use the compressed data, the file can be extracted, decompressed and/or converted from one file format to another, and/or parsed, for example, using JIT engines loaded into a hard-mounted processor or configured in a quantum processor, and applying one or more JIT pipeline procedures to the compressed file.

[00677] Accordingly, in various cases where a system contains an associated FPGA, it is possible to completely or partially reconfigure the FPGA and/or orchestrate the quantum processing engine to perform a JIT procedure. In particular, the JIT module may be loaded into the system and/or implemented as one or more engines, which may include one or more compression engines 150 that are configured to run in the background. Therefore, when a given file format is called, the JIT type system can perform the necessary operations on the requested data to produce a file in the requested format. These operations may include compression and/or decompression, as well as conversion to produce the requested data in a recognized file format.

[00678] For example, when generating genetic data, it is typically created in a raw data format, such as a BCL file, which can then be converted to a FASTQ file, for example, by a next generation sequencer that generates the data. However, with the present system, raw data files, such as BCL or other raw data file format, can be streamed or otherwise transferred to a JIT module, where it can then be converted into data in a FASTQ file format and/or other format file. For example, once a FASTQ file has been generated, it may be processed as described herein, and a corresponding BAM file may be generated. And, similarly, a corresponding VCF file can be generated from a BAM file. In addition, SAM and CRAM files may also be generated during appropriate steps. Each of these steps can be completed very quickly, especially if the corresponding file format has already been generated once. Therefore, after receiving the BCL file, for example, directly from the sequencer, the BCL file can be converted to a FASTQ file or directly converted to a SAM, BAM, CRAM and/or VCF file, for example, using a hardware or quantum-implemented mapping/alignment/ sorting/identifying options.

[00679] For example, in one use model in a typical sequencing instrument, a single lane may be loaded with a large number of different genomes from a subject for parallel processing by a single sequencing instrument. Therefore, upon completion of the analysis, a large number of different BCL files are generated, as a multiplex complex, derived from all the different bands and representing the complete genomes of each of the different subjects. Accordingly, these multiplexed BCL files can then be demultiplexed to generate corresponding FASTQ files representing the genetic code for each individual subject. For example, if N BCL files are generated in a single sequencing run, these files would need to be demultiplexed, layered, and stitched together for each subject. This stitching is a complex process where each subject's genetic material is converted into BCL files, which can then be converted into FASTQ files or used directly for mapping, alignment and/or sorting, variant identification, and the like. This process can be automated to significantly speed up its various steps.

[00680] Additionally, as shown in FIG. 40A, once this data is generated 110, and therefore needs to be stored, for example, regardless of which format is selected, it may be stored in a password-protected and/or encrypted cache memory, such as dedicated genomic memory 400 like Dropbox. Accordingly, as the generated and/or processed genetic data leaves the sequencer, it can be processed and/or stored and made available to other users in other systems, for example, in a Dropbox cache 400. In this case, an automated bioinformatics analysis pipeline can then access the data in the cache and automatically begin processing it. For example, the system may include a management system, such as workflow management system 151, having a controller, such as a microprocessor or other intelligent system, such as artificial intelligence, that controls the retrieval of BCL and/or FASTQ files, such as from cache memory , and then directs processing of this information to generate a BAM, CRAM, SAM and/or VCF file, thereby automatically generating and outputting various processing results and/or sorting them into Dropbox memory 400.

[00681] A unique advantage of JIT processing, which is implemented in this use model, is that JIT allows the various generated genetic files to be compressed, for example, before storing the data, and quickly decompressed before use. Therefore, JIT can compile and/or compress and/or store data as it exits the sequencer and stores it in a secure Dropbox genomic cache. This secure "Dropbox" genomic cache 400 may be a cloud-accessible cache 50 that is configured to store genomic data from one or more automated sequencers 110, such as when the sequencers are located remotely from the cache 400.

[00682] In particular, after the sequence data is generated 110, for example, by the CNS, it can be compressed 150 for transmission and/or storage 400 to reduce the amount of data uploaded to the cloud 50 and stored there. Such upload, transfer and storage can be performed quickly due to data compression 150 that occurs in the system, for example, before transfer. Additionally, once downloaded and stored in the cloud cache 400, the data can then be retrieved, locally 100 or remotely 300, for processing in accordance with the BioIT pipeline devices, systems, and methods described herein to create a mapping file, alignment , sorting and/or defining variants, such as a SAM, BAM and/or CRAM file, which can then be stored along with a metafile that specifies information about how the generated file was created, such as SAM, BAM, CRAM, etc. .

[00683] Therefore, in conjunction with the metadata, the compressed SAM, BAM and/or CRAM file can then be processed to produce other file formats such as FASTQ and/or VCF files. Accordingly, as noted above, using JIT it is possible to recover a FASTQ or VCF file on the fly from a compressed BAM file and vice versa. The BCL file can also be recovered in a similar way. It should be noted that SAM and CRAM files can be similarly compressed and/or stored and used to create one or more other file formats. For example, a CRAM file that can be converted back from the CRAM format can be used to create a variant definition file and, likewise, to create a SAM file. Therefore, you need to save the SAM, BAM and/or CRAM file, other file formats such as VCF, FASTQ, BCL can be played from these files.

[00684] Accordingly, as shown in FIG. 40A, a mapping and/or leveling and/or sorting and/or variant determination instrument 110, such as a workstation computer, may be located at the operating location 100, and/or a second corresponding instrument 300 may be located remotely and be accessible from the cloud 50. This configuration, together with the devices and methods described herein, are configured to allow a user to quickly perform a BioIT analysis "on the cloud" as described herein to obtain result data. The results data may then be processed for compression, and once compressed, may be configured to be transmitted, for example, back to the local computing resource 100, or may be stored on the cloud 400 and, through a cloud interface, made available to the local computing resource 100. In such a case, the compressed the data may be a SAM, BAM, CRAM and/or VCF file.

[00685] In particular, the second computing resource 300 may be another workstation solution, or it may be a server-based configurable resource, for example, where the computing resource is accessible from the cloud 50 and is configured to perform mapping and/or alignment and/ or sorting and/or identifying options. In this case, the user can request the cloud server 300 to run BioIT jobs on the uploaded data, such as data in BCL and/or FASTQ format. In this case, the server 300 will then access the stored and/or compressed files to quickly process the data and generate one or more result data that can then be compressed and/or stored. Additionally, one or more other BCL, FASTQ, SAM, BAM, VCF, or other file formats can be generated from the results data file on the fly using JIT processing. This configuration thereby widens the typical transmission speed bottleneck.

[00686] Therefore, in various embodiments, system 1 may include a first mapping and/or alignment and/or sorting and/or variant determination apparatus 100, which may be located locally 100, for example, for local data acquisition, compression 150, and /or storage 200; and the second device 300 may be remotely located and associated with the cloud 50, wherein the second device 300 is configured to receive the generated and compressed data and store it, for example through an associated storage device 400. Once saved, the data may be available for decompression and conversion of the saved files into one or more other file formats.

[00687] Therefore, in one implementation of the system the data. for example, raw sequence data, e.g., in BCL or FASTQ file format, that is generated by a data generation device, e.g., sequencer 110, can be uploaded and stored on a cloud 50, e.g., in an associated Dropbox genomic cache 400. This data may then be directly accessed by the first mapping and/or leveling and/or sorting and/or variant determination engine 100 as described herein, or may be accessed indirectly by the server resource 300, which may then process sequence data to produce mapped, aligned, sorted and/or qualified variants of the data results.

[00688] Accordingly, in various embodiments, one or more of the storage devices described herein may be configured to be accessible, if appropriate, through the cloud. For example, various system results data may be compressed and/or stored in memory or other suitably configured database, where the database is implemented as a "Dropbox" type genomic cache 400, for example, where various results data may be stored in a SAM file, BAM, CRAM and/or VCF, which can be accessed remotely. In particular, it should be noted that, as shown in FIG. 40A, a local instrument 100 may be provided, which local instrument may be associated with or remote from the sequencing instrument 110 but associated with the sequencing instrument 110 via a local cloud 30, and the local instrument 100 may also be associated with local storage 200 or remote cache 400, for example, when the remote cache is in the form of a Dropbox genome storage. Additionally, in various instances, a second mapping and/or alignment and/or sorting and/or variant determination instrument 300, such as a cloud instrument, may, if appropriately authorized, be connected to the Dropbox genome repository 400 to gain access to files, eg, compressed files, stored there by local computing resource 100, and may then decompress those files to make the results available for further, eg, secondary or tertiary processing.

[00689] Accordingly, in various cases, the system can be streamlined so that as data is generated and exits sequencer 110, for example in raw data format, it can either be immediately uploaded to cloud 50 and stored in genome storage 400. Dropbox" or transferred to the BioIT processing system 300 for further processing and/or compression before beginning the upload and storage 400. Once stored in the cache 400, the system can immediately queue the data for extraction, compression, decompression and/or further processing, for example by another associated BioIT processing device 300, which, once processed into result data, can be compressed and/or stored 400 for use at a later date. At this point, the tertiary processing pipeline can be initiated, and thereby the stored result data from the secondary processing can be unpacked and used, for example, for tertiary analysis, in accordance with the methods described herein.

[00690] Therefore, in various implementations, the system may be pipelined such that all data that exits the sequencer 110 may either be compressed, for example, by local computing resource 100, before being transmitted and/or stored 200, or the data may be transmitted directly into a genome "Dropbox" folder for storage 400. Once received there, the stored data can then be substantially immediately queued for extraction and compression and/or decompression, such as by a remote computing resource 300. Once decompressed, the data can be substantially immediately made available for processing such as mapping, alignment, sorting and/or variant determination to create processed result data that can then be compressed again for storage. The compressed secondary result data may then be made available, for example, in a Dropbox genome storage 400, decompressed, and/or used in one or more tertiary analysis procedures. Because data can be compressed when stored and essentially immediately decompressed when retrieved, it is available for use by many different systems and in many different bioanalytical protocols at different times—it just needs to be accessed in the Dropbox storage cache 400.

[00691] Therefore, in a similar manner, the BioIT platform pipelines presented herein can be configured to provide incredible flexibility in data generation and/or analysis and are configured to accept form-specific input of genetic data in a variety of formats for processing to create output formats that are compatible with various downstream analyses. Accordingly, as shown in FIG. 40C, devices, systems and methods are provided herein for performing genetic sequencing analysis, which may include one or more of the following steps: First, receiving an input file, which may be in one or more FASTQ or BCL file formats or other genetic sequence file formats, such as a compressed file format, after which the file can be decompressed and/or processed through a series of steps described herein to form a VCF/gVCF file, which can then be compressed and/or processed, and/or transferred. Such compression and decompression may occur at any suitable stage throughout the process.

[00692] For example, after receiving a BCL file, it may be processed in a parsing pipeline, for example, sequentially, as described herein. For example, upon receipt, the BCL file may be converted and/or demultiplexed, for example, into a FASTQ and/or FASTQgz file format, which can then be sent to a mapping and/or alignment module, for example, server 300 to be mapped and/or aligned in accordance with the device and its uses described herein. Additionally, in various embodiments, the mapped and aligned data, for example in a SAM or BAM file format, may be sorted by position and/or marked and duplicates removed. The files may then be converted, for example to a CRAM file, for transmission and/or storage, or may be sent to a variant definition module, such as an HMM, for processing to produce a variant definition file, VCF or gVCF.

[00693] More precisely, as shown in FIG. 40C and 40D, in certain cases, a file intended to be received by the system may be streamed or otherwise transferred to the system directly from a sequencing device, such as the SNP 110, and therefore the transferred file may be in a BCL file format. When a received file is in the BCL file format, it may be converted and/or otherwise demultiplexed into a FASTQ file for processing by the system, or the BCL file may be processed directly. For example, platform pipeline processors may be configured to receive BCL data that is streamed directly from the sequencer, as described with reference to FIG. 1, or they can accept data in the FASTQ file format. However, receiving the sequence data immediately upon exiting the sequencer is useful because it allows the data to be translated directly from raw sequencing data into directly processed data, eg, one or more of SAM, BAM, and/or VCF/gVCF formats, for output.

[00694] Accordingly, upon receipt of a BCL and/or FASTQ file by, for example, computing resource 100 and/or 300, it may be mapped and/or aligned by the computing resource, where the mapping and/or alignment may be performed on single or paired reads ends. For example, once ingested, sequence data may be compiled into reads for analysis, for example, with read lengths that may range from about 10 or about 20, such as 26, or 50, or 100, or 150 base pairs or less, to about 1K, or about 2.5K, or about 5K, even about 10K base pairs or more. Likewise, after mapping and/or alignment, the sequence can then be stored, eg sorted by position, eg by grouping according to a reference range and sorting groups by reference position. In addition, the sequenced data can be processed by marking duplicates, for example, based on the starting position and CIGAR string, to generate a high-quality duplicate report, and all marked duplicates can be removed at this stage. The result can be a mapped and aligned SAM file, which can be compressed to form a BAM/CRAM file, for example, for storage and/or further processing. In addition, after the BAM/CRAM file is retrieved, the mapped and/or aligned sequence data can be sent to a system variant detection module, such as a reassembled haplotype variant detector, which, in some cases, can use one or more of Smith-Waterman alignment and/or hidden Markov model, which may be implemented in a combination of software and/or hardware, to generate a VCF file.

[00695] Therefore, as shown in FIG. 40D, the system and/or one or more of its components are configured to convert BCL data into FASTQ or SAM/BAM/CRAM data formats, which can then be sent throughout the system for further processing and/or data reconstruction. For example, after BCL data has been received and/or converted to a FASTQ file and demultiplexed and/or deduplicated, it can be sent to one or more pipeline modules described herein, for example, for mapping and/or alignment, which, depending on the number of samples processed, will result in the creation of one or more, for example several, SAM/BAM files. These files can then be sorted, de-duplicated, and sent to the variant determiner to produce one or more VCF files. These steps can be repeated to provide more context and improve accuracy. For example, after the sequence data has been mapped or aligned, for example to produce a SAM file, the SAM file can then be compressed into one or more BAM files, which can then be passed to the VCF engine for conversion by system processing into a VCF file. gVCF, which can be compressed into a CRAM file. As a result of traversing the entire system, the output may be a Gzip and/or CRAM file.

[00696] Specifically, as shown in FIG. 40C and 40D, one or more of the generated files may be compressed and/or transferred from one system component to another, such as from a local resource 100 to a remote resource 300, and may be decompressed upon receipt, such as if it had previously been compressed, or converted/demultiplexed. More specifically, upon receipt of a BCL file by either local resource 100 or remote resource 300, it may be converted to a FASTQ file, which may then be processed by the system integrated circuits for mapping and/or alignment purposes, or may be transmitted to remote resource 300 for such processing. Once mapped and/or aligned, the resulting sequence data, for example in a SAM file format, can undergo further processing, such as being compressed one or more times, for example into a BAM/CRAM file, after which the data can be processed by sorting by positions, marking duplicates and/or identifying variants, the results of which, for example in VCF format, can then be compressed again and/or stored and/or transferred, for example, from remote resource 300 to local resource 100.

[00697] More specifically, the system may be configured to process BCL data directly, thereby eliminating the conversion step to a FASTQ file. Likewise, BCL data can be fed directly into the pipeline to create a unique VCF output file for the sample. Intermediate SAM/BAM/CRAM files can also be generated upon request. Therefore, the system may be configured to receive and/or transmit one or more data files, such as BCL or FASTQ, containing sequence information, and process them to create a compressed data file, such as a SAM/BAM/CRAM data file.

[00698] Accordingly, as shown in FIG. 41A, the user may wish to obtain the compressed file and convert it to the original version of the generated BCL 111c and/or FASTQ 111d file, for example, to subject the data to further, eg, more advanced signal processing 111b, eg, for error correction. In an alternative implementation, the user may access raw sequence data, such as in a BCL or FASTQ file format 111, and subject the data to further processing, such as mapping 112 and/or alignment 113, and/or other related functions 114/115. For example, the output data from these procedures may then be compressed and/or stored and/or subjected to further processing 114 such as sorting 114a, deduplicating 114b, recalibrating 114c, local realignment 114d, and/or compression/decompression 114e. The same or another user may then wish to access a compressed form of the mapped and/or aligned results data and then perform other analysis on it, for example to perform one or more of the variant determinations 115, for example, through an HMM, a Smith-Waterman algorithm, transformations, etc., and the results are then compressed and/or saved. Another user of the system can then access the compressed VCF file 116, decompress it, and subject the data to one or more tertiary processing protocols.

[00699] Next, the user may want to perform a pipelined comparison. Variant mapping/alignment/sorting/identification is useful for performing various genomic analyses. For example, if further DNA or RNA analysis or analysis of a slightly different kind is subsequently required, the user may wish to run the data through a different pipeline and therefore having access to the recovered original data file is very useful. Likewise, this process may be useful, for example, when a SAM/BAM/CRAM file may need to be created or recreated, such as when a new or different reference genome has been generated and therefore the mapping or alignment may need to be repeated to a new reference genome.

[00700] Storing compressed SAM/BAM/CRAM files is also useful as it will allow the user of system 1 to take advantage of the fact that the reference genome forms the backbone of the result data. In such a case, what is important is not the data that agrees with the reference, but rather how the data does not agree with the reference. Therefore, it is important to retain only data that is inconsistent with the reference. Therefore, system 1 can benefit from this fact of storing only what is important and/or useful to users of the system. Thus, the entire genomic file (showing a match and mismatch to the reference) or a portion thereof (showing only the match or mismatch to the reference) can be compressed and stored. Therefore, it can be noted that since only the differences and/or variations between the reference and the genome under study are most useful for research, only these differences need to be preserved in different implementations, since everything that matches the reference does not need to be examined anew. Accordingly, since any given genome differs little from the reference genome, for example, 99% of human genomes are usually identical, then after creating a BAM file, only variations relative to the reference genome need to be considered and/or saved.

[00701] Accordingly, as shown in FIG. 41B, another useful component of the cloud-accessible system 1 proposed herein is a workflow controller 151 that can be used to automate the flow of the system. Such system automation may involve the use of various system components to access data, either locally 100 or remotely 300 when and/or where it becomes available, followed by substantially automatic application of further processing steps, such as those related to BioIT, to that data - the conveyors described in this document. Accordingly, the workflow controller 151 is the underlying automation technology for managing the system's various pipelines, such as 111, 112, 113, 114 and/or 115, and in various cases may utilize artificial intelligence component 121a.

[00702] For example, the system 1 may include an artificial intelligence (AI) module that is capable of analyzing various system data and, in response, communicates its results using the workflow management system 151. In particular, in various cases, the AI module may be configured to analyze various genomic data presented in the system, as well as outcome data that is generated by data processing, to identify and determine various relationships between this data and/or any other data that may be entered into the system. More specifically, the AI module may be configured to analyze various genomic data according to a variety of other factors to determine any relationship, e.g., cause and effect, between various factors, e.g., data points that may be informative regarding the impacts of the ones in question. factors on certain genomic data, such as data variation, and vice versa.

[00703] Namely, as described in more detail later herein, the AI module may be configured to correlate a subject's genomic data generated by the system with any electronic medical records for that subject or other individuals to determine relationships between them and/or or other relevant factors and/or data. Accordingly, such other data that may be used in determining any relevant effects and/or relationships that these factors may have on the subject and/or his genomic data and/or health include: NIPT data, NICU data, cancer data, LDT data, environmental and/or agro-biological data and/or other such data. For example, further data for analysis may be derived from other factors such as environmental data, clade data, microbiome data, methylation data, structural data such as chimeric or conjugate read data, generative variant data, allele data, RNA data and other such data relating to the subject's genetic materials. Therefore, the AI module can be used to associate various relevant data flowing through the system with variants identified in the genome of one or more subjects, along with other possible associated factors-based consequences.

[00704] In particular, the AI engine may be configured to execute on a CPU/GPU/CPU and/or it may be configured to execute as an accelerated AI engine that may be implemented on an FPGA and/or a quantum processing device. Namely, the AI engine may be coupled to one or more, for example all, of the system's different databases so that the AI engine can use and process various data flowing through the system. In addition, when the subject whose genome is being processed gives appropriate permission to access genomic data and medical history, the system is configured to correlate the various data sets with each other and can then extract the data to determine various significant matches, associations and/or relationships .

[00705] More specifically, the AI module may be configured to implement a machine learning protocol on the input data. For example, genomic data of a plurality of subjects, which is generated based on the assays described herein, may be stored in a database. Likewise, with appropriate permissions and authentications, the electronic medical records/electronic health record (EMR) of the subject whose genomic DNA is being processed can be accessed and similarly stored in a database. As described in more detail below, processing engines may be configured to analyze subjects' genomic data as well as their EMR data to determine any correlations between them. These correlations will then be used to confirm the observed relationships, and the results can be used to perform the various functions of the system more efficiently and more successfully.

[00706] For example, an AI processing engine can access genomic data from subjects, correlate it with known diseases and conditions of those subjects, and based on this analysis, the AI engine can learn to perform predictive correlations based on that data to improve its ability to predict the presence of disease and/or other similar conditions in other individuals. In particular, by identifying such correlations between the genomes of other people with their EMR, for example regarding the presence of disease markers, the AI module can learn to identify such correlations, for example, system-defined disease markers, in the genomes of other people, thereby gaining the ability to predict the possibility of disease or other identifiable conditions. More specifically, by analyzing a subject's genome in comparison to known or defined genetic markers, and/or by identifying variations in the subject's genome, and/or, additionally, by identifying potential relationships between genomic data and the subject's health status, such as an EMR, the AI module can be able to draw conclusions not only about the subject being studied, but also about other people from whom samples will be taken in the future. This can be done, for example, on a systematic basis, for each subject separately, or within populations and/or in different geographical areas.

[00707] More specifically, in relation to the presented systems, a collection of reads is created. The cluster may overlap areas known to have a higher likelihood of significant variation. Accordingly, on the one hand, the system will analyze the cluster to determine the presence of variation, and, at the same time, based on previous results, it will already know the likelihood of the presence or absence of variation there, for example, it will have an initial prediction regarding whether what should the answer be? Regardless of whether the expected variation is present or not, this information will be useful when analyzing this region of the genomes of other subjects. For example, it may be one data point in a sum of data points used by the system to improve variant definitions and/or better associate those variants with one or more disease states or other health conditions.

[00708] For example, in an example study protocol, the AI analysis may include obtaining an electronic image of a cluster of one or more regions in the genome, for example, those regions suspected of encoding one or more health conditions, and associating this image with known definitions of variations from other collections, for example where the relationship of these variations to disease states may or may not be known. This can be done over and over again, training the system to process information, make appropriate associations, and make correct choice determinations faster and faster with higher accuracy. Having done this for different, for example all, known regions of the genome suspected of causing a disease, the same can be repeated with the rest of the genome, for example, until the entire genome has been studied. Likewise, this can be repeated over and over again for multiple genome samples over and over again to train a system, such as a variant detector, to perform variant detection more accurately, faster, and more efficiently, and/or to allow the tertiary processing module to better identify disease states.

[00709] Accordingly, the system receives a set of inputs with known answers, performs an analysis and calculates the answer, and thereby learns through this process, for example, rendering an image of a cluster related to one genome and then learning to make a determination based on another genome, faster and faster, since it is easier for her to determine that subsequent clusters resemble previously obtained images with a known relationship with disease states. Thus, the system may be configured to learn to predict the presence of a variant, for example, based on pattern recognition, and/or to predict the relationship between the presence of that variation with one or more medical conditions.

[00710] More specifically, the more a system analyzes a partial or full genome and determines relationships between variations and different conditions, for example across multiple samples, the better it makes predictions, for example based on partial or full genome images of clusters. This is useful in predicting disease states based on cluster images and/or other read analysis, and may involve creating a correlation between one or more EMRs (containing phenotypic data), the cluster image and/or known variants (genotypic data) and/or diseases and disease states, for example, on the basis of which predictions can be made. In various cases, the system may include a transcription function to be able to transcribe any physical notes that may be part of the patient's medical record to include these data in associations.

[00711] In one use model, a subject may have a mobile tracker and/or sensor, such as a cell phone or other computing device, that may be configured to both track the subject's location and sense environmental and/or physiological conditions of the user in this place. Other perceptual data may also be collected. For example, the mobile computing device may contain a GPS tracker and/or its location may be determined by triangulating cell towers, and may be configured to transmit the data it collects, for example, via cellular WIFI, Bluetooth, or other suitably configured protocol communications. Thus, the mobile device can monitor and categorize environmental data related to geographic locations, environmental conditions, physiological state, and other perceptual data of the mobile computer user that he encounters in his daily life. The collected location, environmental, physiological, health and/or other related data, such as ZNA data, may then be transmitted, for example, on a regular or periodic basis, to one or more databases described in this manual, wherein the collected ZNA data may be correlated with the subject's medical history, such as EMR records, and/or genomic data, as determined by the system described herein.

[00712] Likewise, in various cases, one or more of these data may be directed from the ZNA data collection and analysis platform to a central repository, for example, in a government agency, for analysis on a larger, for example, national, scale, for example, in accordance with artificial intelligence described in this document. For example, a database, such as a government-run database, may have recorded environmental data against which the subject's environmental data can be compared. For example, in one illustrative case, an NIPT test may be performed on a mother, father, and their child, and then over the course of their entire lives, their environmental, genomic, and medical record data may be collected and correlated with each other and/or the same or more patterns, for example, over the life course of individuals, especially with respect to the occurrence of mutations, for example due to environmental factors. This data collection can be carried out throughout the life of an individual and can be done on a family basis to better build a data collection database and better predict the effects of such factors on genetic variation and vice versa.

[00713] Accordingly, the workflow controller 151 allows the system 1 to receive input data from one or more sources, such as multiple sequencing instruments, such as 110a, 110b, 110c, etc., and multiple input data from a single instrument 110 sequencing, where the incoming data represents the genomes of multiple subjects. In such cases, the workflow controller 151 not only monitors all incoming data, but also efficiently organizes and facilitates secondary and/or tertiary processing of the received data. Accordingly, the workflow controller 151 allows system 1 to seamlessly connect to both small and large sequencing centers where genetic material of any kind can be received by one or more sequencing machines 110 simultaneously, all of which can be transmitted to system 1 via the cloud 50.

[00714] More precisely, as shown in FIG. 41A, in various cases, one or more of a plurality of samples may be received by the system 1, and therefore the system 1 may be configured to receive and efficiently process the samples, either sequentially or in parallel, for example, in a multi-sample processing mode. Accordingly, to streamline and/or automate the processing of multiple samples, the system can be managed by a comprehensive Workflow Management System (WMS) 151 or Laboratory Information Management System (LIMS). The WMS 151 allows users to easily schedule multiple workflows for any pipeline, as well as adjust or accelerate analysis algorithms for SNP, platform pipelines and their accompanying applications.

[00715] In such a case, each sequence being processed may have a barcode indicating the sequence type, file format, and/or what processing steps have been performed and what processing steps are to be performed. For example, a barcode may contain a declaration indicating "this is the processing of the genome of subject X in file format Y, so this data needs to be passed through pipeline Z", or, similarly, it may state "this is the resulting data A, which should go to the reporting system " Accordingly, as data is received, processed, and transmitted through the system, barcodes and results will be uploaded to a workflow management system 151, such as a LIMS (Lab Information Management System). In this example, the LIMS could be a standard tool used for laboratory management, or it could be a custom tool used for process control.

[00716] In either case, the workflow controller 151 tracks the barcoded sample from its arrival at the site, for example for storage and/or processing, until the results are sent to the user. In particular, the workflow controller 151 is configured to track all data as it flows through the system from start to finish. More specifically, when a sample arrives, the associated barcode is read, and based on that read, the system determines how to perform the requested work and prepares the sample for processing. Such processing may be simple, such as performed by a single genomic pipeline, or it may be more complex, such as performed by multiple pipelines, such as five pipelines that must be stitched together. In one particular model, the generated or received data can be run through the system to produce processed data, which can then be run through an equivalent GATK module and the results compared, and then the sample can be passed on to another pipeline for further, such as tertiary, processing 700 (see FIGURE 41B).

[00717] Therefore, the overall system may operate in accordance with several different processing pipelines. Indeed, many of the system's processes may be interconnected, with the workflow manager 151 notifying or otherwise determining that a job is waiting, quantifying the job's matrices, identifying available resources to perform the required analysis, loading the job into the system, receiving incoming data, e.g. from the sequencer 110, loads them and then processes them. Specifically, once a workflow is configured, it can be saved, and then that workflow receives the modified barcode and the automated process occurs as directed by the workflows.

[00718] Before the advent of true automated workflow management system 151, many bioinformatics specialists had to spend a long time configuring and tuning the system and its component parts, and then had to spend more time actually performing the analysis. To further complicate matters, the system had to be reconfigured before the next sample could be accepted for analysis, and reconfiguring the system to analyze a new set of samples required even more time. Thanks to the technology described in this document, the system can be fully automated. The present system is particularly configured to automatically accept multiple samples, map them to multiple different work streams and pipelines, and process them on the same or multiple different motherboards.

[00719] Accordingly, the workflow management system 151 reads job requirements from the barcodes, allocates resources to execute the jobs, e.g., regardless of location, updates the sample barcode, and routes the samples to allocated resources, e.g., processing units, for processing. Therefore, it is the workflow manager 151 that determines the secondary analysis 600 and/or tertiary analysis 700 protocols that will be performed on the received samples. These processing units are the resources that are available to delimit and perform the operations assigned to each data set. In particular, the workflow controller 151 manages various operations related to receiving and reading a sample, defining jobs, allocating resources to perform those jobs, such as secondary processing, connecting all system components, and moving a set of samples through the system from one component to another. Thus, the controller 151 acts to control the entire system from start to finish, for example, from sample reception to VCF file generation and/or up to and including tertiary processing (see FIG. 41B).

[00720] In additional cases, as shown in FIG. 41C, system 1 may include an additional tier of processing modules 800, for example, configured to perform additional processing, for example, of secondary and/or tertiary processing data, for example, for diagnosis, discovery of new diseases and/or treatments and/or preventing them. For example, an additional level of processing 800 may be provided in various cases, such as for disease diagnosis, therapeutic intervention, and/or prophylactic warning 70, for example, including NIPT 123a, NICU 123b, cancer 123c, LDT 123d, agri-biological 123e, and other such data for disease diagnosis, prevention and/or therapy, using these data generated by one or more of the provided primary and/or secondary and/or tertiary pipelines.

[00721] Accordingly, a system 1 is presented herein for creating and using a local network 30 and/or a wide area hybrid network 50. For example, local cloud 30 is currently used primarily as a private data store, such as at a remote storage location 400 . In such a case, data computation is performed locally 100 using the local computing resource 140, and when storage needs are large, the local cloud 30 can be accessed to store the data generated by the local computing resource 140, for example, using a remote private storage resource 400. Consequently, the data generated is typically managed entirely locally 100. In other implementations, the data is generated, computed, and managed entirely externally through a secure connection to the remote computing resource 300 via a private cloud interface 30.

[00722] In particular, the bioinformatics analysis platform implementation, local compute 140 and/or storage 200 functions are typically supported locally at the site 100. However, when storage needs exceed local storage capacity, the data can be offloaded by accessing the local cloud 30 for private storage outside the place of 400 operation. In addition, when there is a need to provide other remote users with access to the stored data 400, such data can be transferred and made available through the global cloud interface 50 to the remote storage 400, but for global access. In such a case, when the computing resources 140 required to perform the computing functions are minimal, but the storage requirements are large, the computing function 140 can be supported locally 100, while the storage function 400 can be supported remotely, for example, for either private or global access. wherein the fully processed data is moved back and forth between a local processing function 140, for example for local processing only, and a storage function 400, for example for remotely storing the processed data 400, for example using JIT protocols described above herein.

[00723] For example, this can be illustrated by the example of a sequencing function 110, such as a typical NGN, where the data generation and/or computation resource 100 is configured to perform functions necessary for sequencing genetic material to obtain genetic sequenced data, such as reads, wherein the data is generated locally 100 and/or transmitted to the operating location locally 30. Once generated, these reads, for example by the on-site NPS, can then be transmitted, for example as a BCL or FASTQ file, over the cloud network 30, for example , to store 400 , in a remote location 300 so that they can be recalled from the cloud 30 if necessary, for example for further processing. For example, after sequence data has been generated and stored, for example, 400, it may thereafter be called up, for example, for local use, for example, to perform one or more of the secondary processing 600 and/or tertiary processing 700 functions, i.e. in a location remote from storage 400, such as on-premises 100. In such a case, local storage resource 200 simply serves as a storage cache where data is placed pending movement to or from the 30/50 cloud, such as to or from remote storage 400 .

[00724] Likewise, when the computing function is large, for example, requiring one or more cores 300 of remote computing servers or computing clusters to process the data, and when the storage requirements for storing the processed data 200 are relatively minimal compared to the computing resources 300 required for processing data, the data to be processed may be sent, for example, through the cloud 30, for processing by a remote computing resource 300, which may contain one or more cores or clusters of computing resources, for example, one or more supercomputing resources. In such a case, after the data has been processed by the cloud computer core 300, it can be moved via the cloud network 30 to be stored locally 200 and made readily available for use by the local computing resource 140, such as for local analysis and/or diagnostics. Of course, the remotely generated data 300 may also be stored remotely 400.

[00725] This can also be illustrated by a typical secondary processing function 600, for example, where pre-processed sequence data, such as read data, is stored locally 200 and accessed, for example, by a local computing resource 100, and transmitted over a cloud network 30 to Internet to the remote computing facility 300 for processing thereby, for example, using a secondary processing function 600 or a tertiary processing function 700 to obtain processed results data that can then be sent back to the local computing facility 100 for storage 200 thereby. This may occur when a local practitioner generates sequenced read data using a local data generation resource 110, such as an automated sequencer, to produce a BCL or FASTQ file, and then sends this data over the network 50 to a remote computing facility 300, which then applies one or more functions to this sequence data, for example a Burrows-Wheeler transform function, or a Needleman-Wunsch and/or Smith-Waterman alignment function, to generate result data, for example in a SAM file format, which can then be compressed and transmitted over the Internet 30/50, for example, as a BAM file, to a local computing resource 100 for examination thereby in one or more locally controlled processing protocols, for example, to create a VCF file, which can then be stored locally 200. In various cases, the data can also be saved remotely 400.

[00726] However, seamless integration is required for interaction between local 100 and remote 300 computer processing, as well as between local 200 and remote storage 400, such as in the hybrid cloud system 50 presented herein. In such a case, the system may be designed such that the local 100 and remote 300 computing resources can work together seamlessly such that the data to be processed can be allocated in real time to either the local 100 or the remote computing resource 300 , without significant overhead due to transmission speed and/or operational efficiency. This may be the case, for example, when software and/or hardware and/or quantum processing must be deployed or otherwise implemented using computing resources 100 and 300 that are configured to match each other and/or have the same or similar functionality capabilities, for example, the hardware and/or software are designed in the same way to execute the same algorithms on generated and/or received data in the same way.

[00727] For example, as shown in FIG. 41A, local computing resource 100 may be configured to generate or receive generated data and, therefore, may include a data generation engine 110, e.g., to generate raw data, and/or analysis 500, e.g., to generate a BCL sequence file and/or FASTQ. This data generation engine 110 may be or be associated with a local computer 100, as described throughout herein, having a processor 140 that may be configured to execute one or more software applications and/or may be implemented in hardware to execute one or more more algorithms, for example, in the mounting configuration, on generated and/or received data. For example, the data generation engine 110 may be configured to obtain one or more generated data, such as sequencing the data 111. In various embodiments, the generated data may be detected data 111a, such as data that can be detected as a change in voltage, ion concentration, electromagnetic radiation, and the like; and/or the data generation engine 110 may be configured to generate and/or process a signal, such as analog or digital signal data, such as data representing one or more nucleotide identities in a sequence or chain of related nucleotides. In such a case, the data generation engine 110, such as the sequencer 111, may also be configured to perform pre-processing on the generated data to process a signal 111b or perform one or more base determination operations 111c, such as on the data, to obtain sequence identity data. , for example, file 111d BCL and/or FASTQ.

[00728] It should be noted that in this case, the generated data 111 can be generated locally and directly, for example, using a local data generation resource 110 and/or a computing resource 140, such as an NTS or an on-chip sequencer. In an alternative implementation, the data may be created locally and indirectly, for example, using a remote computing and/or generation resource, such as a remote SNS. Once data 111, e.g., in BCL and/or FASTQ file format, is created, it may be transferred indirectly via local cloud 30 to local computing resource 100, e.g., for secondary processing 140 and/or storage in local storage resource 200 , for example, while awaiting further local processing 140. In such a case, when the data generation resource is remote from the local processing resources 100 and/or storage 200, the corresponding resources can be configured so that remote and/or local storage, remote and local the processing and/or communication protocols used by each resource could be designed to be seamlessly and/or seamlessly integrated with each other, for example by running the same, similar and/or equivalent software, and/or having the same, similar and /or equivalent hardware configurations, and/or the use of the same communication and/or transmission protocols, which, in some cases, may have been implemented at or after the time of manufacture.

[00729] Specifically, in one implementation, these functions may be implemented in a hardware configuration, for example, where the sequencing function and the downstream function are supported on the same or related chip or chipset, for example, where the sequencer and downstream processor are interconnected directly on the chip as described in this manual. In other implementations, these functions may be implemented on two or more separate devices through software, such as a quantum processor, CPU, or GPU, that are optimized to allow the two remote devices to communicate seamlessly with each other. Other implementations may also use a combination of hardware and software implementations to perform the above functions.

[00730] More specifically, the same configuration may be implemented to perform mapping, alignment, sorting, variant determination, and/or other functions that may be deployed by local 100 and/or remote 300 computing resources. For example, the local computing 100 and/or remote 300 resources may include software and/or hardware configured to implement one or more of the secondary tier 600 processing functions 112-115 and/or the tertiary tier 700/800 processing functions locally and /or remotely generated data, such as genetic sequence data, such that the processing and its results can be seamlessly shared with each other and/or stored thereby. In particular, the local computing function 100 and/or the remote computing function 300 may be configured to generate and/or receive raw data, such as genetic sequence data, for example, in a BCL and/or FASTQ file format, and execute one or more protocols secondary 600 and/or tertiary 700 processing on these generated and/or received data. In such a case, one or more of these protocols may be implemented in software, hardware, or a combination format, such as running on a quantum processor, CPU, and/or GPU. For example, data generation resource 110 and/or local resource 100 and/ or remote processing resource 300 may be configured to perform one or more of a mapping operation 112, an alignment operation 113, a variant determination operation 115, or other related function 114 on the received or generated data in software and/or hardware.

[00731] Accordingly, in various embodiments, a data generation resource such as a sequencer 111, e.g., an SNP or an on-chip sequencer, whether implemented in software and/or hardware or a combination thereof, may also be configured to include an initial tier of processors 500, such as a scheduler, various analytics, comparison tools, graph builders, release tools, and the like, to assist the data generator 111, such as a sequencer, in converting biological information into raw read data in BCL file format 111d or FASTQ. In addition, local computing resource 100, whether implemented in software and/or hardware or a combination thereof, may also be configured to include an additional tier of processors 600, such as may include a mapping engine 112 or may otherwise include programming to perform an algorithm on genetic sequence data, for example, to perform a Burrows-Wheeler transform and/or other algorithm to construct a hash table and/or to perform a hash function 112a on said data, for example, to map a hash seed to generate mapped data sequences. Moreover, local computing resource 100, whether implemented in software and/or hardware or a combination thereof, may also be configured to include an initial tier of processors 600, for example, may also include an alignment engine 113 as described herein , or may otherwise involve programming to perform an alignment algorithm on genetic sequence data, e.g., a mapped sequenced sequence, e.g., to perform a gapped and/or ungapped Smith-Waterman alignment, and/or a Needleman-Wunsch algorithm or similar algorithm 113a evaluation to generate aligned sequence data.

[00732] Local computing resource 100 and/or data generation resource 110 may also be configured to include one or more other modules 114, whether implemented in software and/or hardware or combinations thereof, which may be configured to performing one or more other processing functions on the genetic sequence data, for example, on the mapped and/or aligned sequence data. Thus, one or more other modules may comprise a suitably configured engine 114 or otherwise include programming to perform one or more other processing functions, such as sorting 114a, duplicate removal 114b, recalibration 114c, local realignment 114d, marking functions 114f duplicates, base quality assessment recalibration 114g, and/or compression function 114e (eg, to obtain a SAM file, a reduced BAM, and/or CRAM compression and/or decompression) in accordance with the methods described herein. In various cases, one or more of these processing functions may be implemented as one or more system 1 pipelines.

[00733] Likewise, system 1 may be configured to include a module 115, whether implemented in software and/or hardware or a combination thereof, which may be configured to process data, for example, sequenced, mapped, aligned and/or or sorted data, so as to obtain a variant definition file 116. In particular, system 1 may include a case determination module 115 for performing one or more case determination functions, such as a Hidden Markov Model (HMM) and/or GATK function 115a, for example, in an installation configuration and/or through one or more software applications. for example, either locally or remotely, and/or the converter 115b for them. In various cases, this module can be made in the form of one or more conveyors of system 1.

[00734] In specific embodiments, as shown in FIG. 41B, system 1 may include a local computing function 100 that may be configured to use computer processing resource 150 to perform one or more further processing functions on data, such as BCL and/or FASTQ data, generated by system data generator 110 and /or received by the system data acquisition engine 120 (as described herein), for example, transferred to it, for example, by a third party 121, for example, through a cloud 30 or a hybrid cloud network 50. For example, a third-party analyzer 121 may use a remote computing resource 300 to generate corresponding data requiring further processing, such as genetic sequence data and the like, which can be transferred to system 1 via the 30/50 network for further processing. This may be useful, for example, when the remote computing resource 300 is a RAS configured to receive raw biological data and convert it into a digital representation of that data, for example, in the form of one or more FASTQ files containing reads of genetic sequence data; and when further processing is required, for example, to determine how the generated sequence of the individual differs from one or more reference sequences, as described herein, and/or the results of this processing are desired to be subjected to further, for example, tertiary processing.

[00735] In such a case, the system 1 is configured to provide one or more parties, for example, a primary and/or a secondary and/or a third party user, access to associated local processing resources 100 and/or correspondingly configured remote resources associated therewith processing 300 so as to allow the user to perform one or more quantitative and/or qualitative processing functions 152 on the generated and/or received data. For example, in one configuration, system 1 may include, for example, in addition to the primary processing pipelines 500 and/or secondary processing 600, a third tier of processing modules 700/800 that may be configured to perform one or more processing functions on the generated and /or received data and/or secondary processed data.

[00736] Specifically, in one embodiment, system 1 may be configured to generate and/or receive processed genetic sequence data 111 that is remotely or locally mapped 112, aligned 113, sorted 114a, and/or further processed 114 to generate a file 116 determination of variants, which can then be subjected to tertiary processing, for example, in system 1, for example, in response to queries 121 from second-party and/or third-party analytics. More specifically, system 1 may be configured to receive processing requests from a third party 121 and further configured to perform such requested secondary 600 and/or tertiary 700/800 processing on the generated and/or received data. In particular, system 1 may be configured to create and/or obtain genetic sequence data 111, may be configured to take that genetic sequence data and map 112, align 113 and/or sort 114a thereof, and process it to create one or more files. 116 variant definition (VCF), and further the system 1 may be configured to implement a tertiary processing function 700/800 on the data, for example, with respect to one or more VCF files generated or received by the system 1.

[00737] In particular, the system 1 may be configured to perform some form of tertiary processing 700 on the generated and/or received data, such as applying one or more pipeline functions 700 thereto to generate genome data 122a, e.g. complete, epigenome data 122b, metagenome data 122c, etc., including genotyping data 122d, including co-genotyping, variant assay data, including GATK 122e and/or MuTect2 assay data 122f, among other potential analytical pipeline data such as pipeline microarray analysis pipeline, exome analysis pipeline, microbiome analysis pipeline, RNA sequencing analysis pipeline and other genetic analysis pipelines. In addition, system 1 may be configured to implement an additional tier of processing 800 on generated and/or processed data, including including one or more of non-invasive prenatal testing (NIPT) 123a, NICU 123b, cancer-related diagnostics and/or therapeutic exposures 123c, various laboratory developed tests (LDTs) 123d, agricultural biological (Ag Bio) applications 123e, or other such health-related processing functions 123f (see FIG. 41C).

[00738] Therefore, in various implementations where the primary user can access and/or configure the system 1 and its various components directly, for example, by directly accessing them, such as through local computing resource 100 as presented herein, the system 1 may also be configured to provide access to a second party, for example connected to the system 1 via a local area network or intranet connection 10, to configure and use the system 1 in a local environment. Additionally, in certain embodiments, the system may be configured to be accessed and/or configured by a third party 121, for example, through an associated hybrid cloud network 50 connecting the third party 121 to the system 1, for example, through an application program interface (API) ), which can be accessed using one or more graphical user interface (GUI) components. Such a GUI may be configured to provide a third party user with access to the system 1 and use an API to configure various system components, pipeline-related modules, and other associated data generation and/or processing functionality to use only those system components that are needed and/or useful to and/or requested or desired by a third party.

[00739] Accordingly, in various cases, the system 1 that is presented herein may be configured by a primary, secondary, or third-party user of the system. In such a case, the system 1 may be configured to allow a user to configure the system 1 and thereby arrange its components so as to deploy one, all, or a portion of the analytics system resources, for example, 152, for execution on data that is generated, received, or otherwise communicated to the system, for example, by a primary, secondary, or third party user, such that system 1 uses only those parts of the system that are necessary or useful to perform the analytics requested by the user to thereby obtain the desired results. For example, for these and other purposes, system 1 may include an API that is configured to include or otherwise be functionally linked to a graphical user interface (GUI) containing an operating menu and/or an accompanying list of system function calls from which the user can select and/or perform other actions to configure and use the system and its components as required.

[00740] In such a case, user-selectable operations from among one or more of the first tier of operations 600: sequencing 111, mapping 112, alignments 113, sorts 114a, variant determination 115, and/or can be controlled using GUI menus and/or system function calls. other related functions 114 in accordance with the ideas set forth herein, for example, with respect to the primary and/or secondary processing functions described herein. In addition, if desired, operations from among one or more of the second tier of operations 700 can be controlled using GUI menus and/or system function calls: genome analysis pipeline 122a, e.g., genome-wide analysis, epigenome pipeline 122b, metagenome pipeline 122c, genotyping pipeline 122d , e.g., collaborative, variant analysis pipelines, e.g., 122e GATK and/or 122f MuTect2, including structural variant pipelines, exome analysis pipeline, microbiome analysis pipeline, RNA sequencing pipelines, and other genetic analysis pipelines. Additionally, if desired, GUI menus and system function calls can direct user-selectable operations from one or more of the third tier operations 800, including: non-invasive prenatal testing (NIPT) 123a, cancer-related NICU 123b diagnostic and/or therapeutic interventions 123c, various laboratory developed tests (LDTs) 123d, agricultural biological (Ag Bio) applications 123e, or other such healthcare-related processing functions 123f.

[00741] Accordingly, menus and system function calls may contain one or more primary, secondary and/or tertiary processing functions so that the system and/or its component parts can be configured to execute one or more data analysis pipelines that are selected and configured by the user . In such a case, the local computing resource 100 may be configured to match and/or mirror the remote computing resource 300 and/or, similarly, the local storage resource 200 may be configured to match and/or mirror the remote storage resource 400 to various components of the system could be executed and/or the data so generated could be stored either locally or remotely in a seamless manner as chosen by system 1. Additionally, in particular embodiments, system 1 may be configured to provide access to third parties to execute proprietary analysis protocols 121a on the generated and/or processed data, for example, by using an artificial intelligence interface designed to search for correlations between them.

[00742] System 1 may be configured to perform some form of tertiary processing on the generated and/or received data. Therefore, in various embodiments, a primary, secondary, or third party user can access and/or configure any layer of system 1 and its various components either directly, such as by directly accessing the computing resource 100, or indirectly, such as through a local network connection 30 or an associated hybrid cloud network 50 connecting that party to system 1, for example through a suitably configured API, if properly authorized. In such a case, the components of the system may be presented in a menu-driven GUI where the user can select from among all the different processing and storage options to perform on the user-submitted data. In addition, in various cases, the user can upload his own system protocols so that they are accepted and executed by the system to process various data in the manner provided and chosen by the user. In such a case, the GUI and associated API will allow the user to access System 1 and use the API to configure various system components, pipeline-related modules, and other associated data generation and/or processing functionality to use only those system components that are needed and/or are useful to that party and/or the performance of which is requested or desired by that party.

[00743] As shown in FIG. 41C, one or more of the above modules and their respective functions and/or associated resources may be configured to be implemented remotely, such as by remote computing resource 300, and may also be configured to be communicated to system 1, such as by using a protocol for seamless transmission over a global cloud connection 50 over the Internet, for example, through a suitably configured data acquisition engine 120. Accordingly, in such a case, local computing resource 100 may include a data acquisition engine 120, for example, configured to transmit and/or receive such acquisition data and/or related information.

[00744] For example, system 1 may include a data acquisition engine 120 that is configured to enable continuous processing and/or storage of data in a seamless and resilient manner, such as through a cloud network 50 where processing functions are distributed locally 100 and/or remotely 300 Likewise, when one or more of the results of such processing can be stored locally 200 and/or remotely 400, the system seamlessly designates to which local or remote resource a given job should be sent for processing and/or storage, regardless of where the job is physically located. resource. Such distributed processing, transmission and reception may include one or more of the functions of sequencing 111, mapping 112, alignment 113, sorting 114a, duplicate labeling 114c, duplicate removal, recalibration 114d, local realignment 114e, base quality assessment recalibration 114f and/ or a compression function 114g, as well as a variant determination function 116, as described herein. If stored locally 200 or remotely 400, the processed data, whatever its state in progress, may be made available to either the local 100 or remote 300 processing resources, for example, for further processing before retransmission and/or re-storing .

[00745] In particular, system 1 may be configured to create and/or receive genetic sequence data 111, may be configured to take that genetic sequence data and process it locally 140, or transmit that data via a suitably configured cloud 30 or hybrid cloud network 50, for example, to a remote processing facility for remote processing 300. In addition, the system 1 may be configured to, after processing, store the processed data remotely 400 or transmit it back for local storage 200. Accordingly, the system 1 may be configured to the possibility of either local or remote generation and/or processing of data, for example, when the stages of generation and/or processing can be from the first tier of functions 600 of primary and/or secondary processing, which may include one or more functions: sequencing 111, mapping 112 , alignment 113 and/or sorting 114a to create one or more variant definition files (VCFs) 116.

[00746] In addition, system 1 may be configured to either locally or remotely generate and/or process data, for example, where the generation and/or processing steps may be from a second tier of tertiary processing functions 700, which may include one or more of generating and/or obtaining data related to genome pipeline 122a, epigenome pipeline 122b, metagenome pipeline 122c, genotyping pipeline 122d, variant analysis pipeline such as GATK 122e and/or MuTect2 122f, as well as other tertiary analysis pipelines such such as the microarray analysis pipeline, the microbiome analysis pipeline, the exome analysis pipeline, as well as RNA sequencing pipelines and other genetic analysis pipelines. In addition, system 1 may be configured to either locally or remotely generate and/or process data, for example where the generation and/or processing steps may be from a third tier of tertiary processing functions 800, which may include one or more of generating and/or obtaining data that relate to or include: non-invasive prenatal testing (NIPT) 123a, NICU 123b, cancer-related diagnostics and/or therapeutic interventions 123c, various laboratory developed tests (LDT) 123d, agricultural biologicals ( Ag Bio) applications 123e or other such healthcare-related processing functions 123f.

[00747] In specific embodiments, as shown in FIG. 41C, system 1 may also be configured to allow one or more parties to access the system and transmit information to or from associated local processing resources 100 and/or remote processing 300, as well as storing information either locally 200 or remotely 400 such in a manner that allows the user to choose what information is processed and/or stored and where in the system. In this case, the user can not only decide which functions of primary, secondary and/or tertiary analysis to perform on the generated and/or received data, but also how to deploy these resources, and/or where to store the results of such processing. For example, in one configuration, the user can choose whether to generate data locally or remotely or use a combination of approaches, whether to secondary process it, and if so, which secondary processing modules are used and/or which resources are used in those processes, and can also determine whether the generated or received data is then subjected to tertiary processing, and if so, using which modules and/or tiers of tertiary processing and/or using which resources in these processes, and similarly where to store the results these processes for each stage of operations.

[00748] Specifically, in one embodiment, a user may configure the system 1 depicted in FIG. 41A, so that the generation of genetic sequence data 111 occurs remotely, for example, using a CNS, but secondary data processing 600 occurs locally 100. In such a case, the user can then determine which of the secondary processing functions to perform locally 100, for example, by selecting processing functions, such as mapping 112, alignment 113, sorting 114a, and/or creating VCF 116, in the menu of available processing options. The user can then determine whether locally processed data should undergo tertiary processing, and if so, which modules to activate for further data processing, and whether this tertiary processing should occur locally 100 or remotely 300. Likewise, the user can select different options for different tiers of tertiary options processing and where any generated and/or received data needs to be stored, locally 200 or remotely 400, at each given stage or moment of the operation.

[00749] More specifically, the primary user may configure the system to receive requests for processing from a third party, wherein the third party may configure the system to perform such requested primary, secondary, and/or tertiary processing on the generated and/or received data. In particular, the user or a second and/or third party may configure the system 1 to create and/or obtain genetic sequence data either locally 100 or remotely 300. In addition, the user may configure the system 1 to take this genetic sequence data and map, align and/or sorting it, either locally or remotely, to create one or more variant definition files (VCFs). In addition, the user can configure the system to perform a tertiary processing function on the data, for example, on one or more VCF files, either locally or remotely.

[00750] Even more specifically, the user or other party may configure the system 1 to perform some form of tertiary processing on the generated and/or received data, and where in the system the processing should occur. Therefore, in various embodiments, a primary, secondary, and/or third party user 121 may access the system 1 and/or configure the system itself and its various components, for example, by directly accessing the local computing function 100 through a local network connection 30 or a connected hybrid connection. cloud network 50 connecting party 121 to system 1, for example, through an application program interface (API) that can be accessed using one or more graphical user interface (GUI) components. In such a case, a third party user can access system 1 and use the API to configure various system components, pipeline-related modules, and other related data generation and/or processing functionality to use only those system components that are needed and/or useful to a third party and/or the execution of which is requested or desired by it, and also to allocate the computing resources that will provide the processing and where the results of the processing will be stored.

[00751] Accordingly, in various cases, system 1 may be configured by a primary, secondary, or third-party user of the system, who may configure system 1 to arrange its components to deploy one, all, or selected analytical system resources for execution on data that the user generates directly, or instructs system 1 to generate it, or commands it to be transmitted to system 1, for example, over a network associated with it, for example, through data acquisition engine 120. Thus, the system 1 is configured to use only those parts of the system that are necessary or useful for the analytics required by or requested by the requestor. For example, for these and other purposes, an API may be included that is configured to include an actual GUI menu and/or an accompanying list of system function calls from which the user can select to configure and use the system in the desired manner.

[00752] Additionally, in specific embodiments, system 1 may be configured to provide access to a primary user and/or third parties, such as government regulatory agencies, such as the Federal Drug Administration (FDA) 70b, or to provide primary users or the ability for third parties to collate, compile and/or access a database of genetic information extracted and/or otherwise obtained and/or compiled by system 1 to form an electronic medical record (EMR) database 70a, and/or enable government agencies access and/or oversight of the system, for example by the FDA to evaluate drug development. System 1 may also be configured to collect, compile and/or annotate data 70c and/or make it accessible to high-level users.

[00753] Accordingly, the system 1 or its components may be configured to be accessible by a remote user, such as a primary user or a third party, and therefore one or more of the computer resources 100 and/or 300 may include a user interface and /or may also comprise a display device having a graphical user interface for allowing a potential user of the system to access the system to transmit sample data for input into and/or to receive result data from one or more of the BioIT pipelines described herein . A GUI or other interface may be configured to allow a user to control components of the system, for example, through a suitably configured web portal, and monitor the progress of processing, regardless of whether the involved computing resources are available locally 100 or remotely 300. Accordingly, The GUI may contain a list of sets of tasks that can be performed, for example, mapping 112, leveling 113, etc., and/or sets of resources for performing tasks, and the user can select which tasks he wants to perform and using which ones. resources. Therefore, in such a case, each individual user can therefore create a unique, or can use a predetermined, analysis workflow, such as by clicking, dragging, or otherwise selecting, specific work projects that he wishes to complete.

[00754] For example, one use model proposes a dashboard with a GUI interface that may contain a plurality of icons representing various processes that can be implemented and executed by the system. In this case, the user can click or drag the icons of the selected workflows into the workflow interface to build the desired workflow process, which, once built, can be saved and used to create control instructions for sample set barcodes. Once the required work projects are selected, the work flow controller 151 may configure the required work flow processes (eg, secondary analysis) and then identify and select resources to perform the selected analysis.

[00755] Once the workflow analysis process has begun, you can monitor its progress through the system by monitoring it on a dashboard. For example, an information monitoring panel indicates how much data is passing through the system, what processes are being executed on this data, what has already been done, how much remains to be processed, which work streams have been completed and which ones still need to be accessed, the latest projects to be completed and which ones to complete already completed. Essentially, the desktop allows you to access everything that is running on the system or part of it.

[00756] In addition, in various cases, the desktop may contain all sorts of different user interfaces that can be accessed using one or more tabs. For example, one of the tabs for accessing system controls could be a "local resources 100" tab, which, when selected, allows the user to select control functions that can be implemented locally. Another tab may be configured to access "cloud resources 300" which, if selected, allows the user to select other control functions that can be implemented remotely. Accordingly, by interacting with the dashboard, the user can select which resources to perform which tasks, and thereby can increase or decrease resource utilization as needed to meet project requirements.

[00757] Therefore, as computational complexity increases and/or the need for increased speed, the user (or the system itself, e.g., WMS 151) can bring into play more and more resources as needed, for example, by simply pressing a button instructing the workflow manager deploy additional local 100 and/or cloud 300 resources that are needed to complete the task within the required time frame. Thus, although the system is automated and/or controlled by workflow controller 151, the user of the system can still set control parameters and can bring cloud resources 300 into action as needed. Accordingly, controller 151 can extend to cloud 50/300 as needed to power additional processing and/or storage resources 400 are put into action.

[00758] In various cases, the desktop interface may be implemented as a mobile application or “app” that is accessible via a mobile device and/or desktop computer. Therefore, in accordance with one aspect, a genomic marketplace, or cohort, may be provided to enable multiple users to collaborate on one or more research projects to create an electronic cohort marketplace that is accessible through a dashboard application, such as a web browser interface. Therefore, the system may provide an Internet forum for performing collaborative research and/or a marketplace for the development of various analytical tools for analyzing genetic data, and the system may be accessible directly through the system interface or through an application to allow remote control of the system by the user.

[00759] Accordingly, in various embodiments, as shown in FIG. 42A, a hybrid cloud 50 is provided that is configured to connect a local computing resource 100 and/or storage resource 200 with a remote computing resource 300 and/or storage resource 400, for example, when the local and remote resources are separated from each other by distance, space, geography, and etc. In such a case, local and remote resources can be configured to communicate with each other so as to share information, such as digital data, seamlessly among themselves. In particular, local resources may be configured to perform one or more types of processing on the data, for example, prior to transmission over the hybrid network 50, and remote resources may be configured to perform one or more other types of processing on the data.

[00760] For example, in one particular configuration, the system 1 may be configured such that the generation and/or analysis function 152 is capable of being performed locally 100 by a local computing resource, for example, to perform a primary and/or secondary processing function for generating and/or processing genetic sequence data as described herein. Additionally, in various embodiments, the local resources may be configured to perform one or more tertiary processing functions on the data, such as one of genome, exome, and/or epigenome, or cancer, microbiome, and/or other DNA/RNA processing assays . In addition, when such processed data is intended to be transmitted, for example, to a remote computing resource 300 and/or a storage resource 400, the data may be transformed, for example, using a suitably configured converter, which may be configured to index, convert, compress and/or data encryption, for example, prior to transmission over the hybrid network 50.

[00761] In specific cases, such as when generated or processed data is transferred to a remote computing resource, such as a server 300, for further processing, such processing may be global in nature and may include receiving data from a plurality of local computing resources 100, ordering such data sets, annotating the data and comparing them, for example, to interpret the data, determine their trends, analyze the data for various biomarkers and assist in the development of diagnostics, therapeutics and/or prevention. Accordingly, in various cases, the remote computing resource 300 may be implemented as a data processing hub, such as when data from various sources may be transmitted, processed, and/or stored pending transformation and/or transmission, such as by being made available to the local computing resource 100. More specifically, the remote processing hub 300 may be configured to receive data from a plurality of resources 100, process that data, and distribute the processed data back to various local resources 100 to enable collaboration among researchers and/or resources 100. Such collaboration may include various sharing protocols and may further include preparing the data to be sent, for example, by allowing the user of system 1 to select among various security protocols and/or privacy settings to control the preparation of the data for transmission.

[00762] In one specific case, which is presented in FIG. 42B, local computing resource 100 and/or storage resource 200 are provided, for example, directly at the user's location. The computing resource 100 and/or the storage resource 200 may be coupled to a data generation resource 121, such as an on-chip sequencer or sequencer as described herein, for example, through a direct or intranet connection 10, wherein the sequencer 121 is configured to generating genetic sequence data such as BCL and/or FASTQ files. For example, sequencer 121 may be part of or reside within the same device as computing resource 100 and/or storage resource 200 so as to have direct data exchange and/or operative connection therewith, or sequencer 121 and computing resource 100 and/or resource The storage 200 may be part of separate devices, but located in the same institution and thus connected via a cable or connection to the internal network 10. In some cases, the sequencer 121 may not be located in the same institution as the computing resource 100 and/or or storage resource 200 and may therefore be connected via the Internet 30 or hybrid cloud network 50.

[00763] In such cases, the genetic sequence data may be processed 100 and stored 200 locally before conversion by a suitably configured converter, or the generated sequence data may be transmitted directly to one or more converters and/or analyzers 152, for example, through a suitably configured connection local network 10, internal network 30, or hybrid cloud network 50 as described above, for example before processing locally. In particular, like data generation resource 121, converter 151 and/or analyzer 152 may be part of or reside within the same device as compute resource 100 and/or storage resource 200 so as to have direct data exchange and/or operative connection therewith. , or the converter and/or analyzer 152 and the computing resource 100 and/or the storage resource 200 may be parts of separate devices from each other, but located in the same institution and thereby connected via a cable or connection to the internal network 10. In some cases, the converter 151 and/or analyzer 152 may not be located in the same institution as the compute resource 100 and/or storage resource 200 and therefore may be connected via the Internet 30 or hybrid cloud network 50.

[00764] For example, the converter may be configured to prepare data to be transmitted either before analysis or after analysis, for example, using a suitably configured computing resource 100 and/or analyzer 152. For example, analyzer 152 may perform a secondary and/or tertiary function. processing on the data as described herein, for example, to analyze the generated sequence data in terms of determining its genomic and/or exome characteristics 152a, its epigenomic characteristics 152b, any various DNA and/or RNA markers of interest and/or cancer indicators 152c, and their relationships with one or more microbiomes 152d, as well as one or more other secondary and/or tertiary processing processes as described herein.

[00765] As indicated, the generated and/or processed data can be converted, for example, using a suitably configured converter, for example, before being transmitted throughout the system 1 from one component thereof to another, for example, over a direct connection, a local network connection 10, Internet 30, or hybrid cloud network 50. Such conversion may include one or more of conversion 151d, such as when data is converted from one form to another; difficulty understanding 151c, including encoding, decoding and/or otherwise converting data from an incomprehensible form to an understandable form; indexing 151b, for example, including compiling and/or organizing generated data from one or more resources and bringing it into a state suitable for locating and/or searching its elements, for example, using a generated index; and/or encryption 151a, for example, creating lockable and non-lockable password-protected data sets, for example, prior to transmission via the Internet 30 and/or hybrid cloud 50.

[00766] Therefore, as shown in FIG. 42C, in these and/or other such cases, the hybrid cloud 50 may be configured to enable seamless and secure transmission of data to all components of the system, for example, when the hybrid cloud 50 is configured to allow different users of the system to configure its component parts and/or the system itself. system to meet user needs for research, diagnostic, therapeutic and/or preventative discovery and/or development. In particular, the hybrid cloud 50 and/or the various components of the system 1 can be configured to be operably connected to compatible and/or appropriate APIs that are configured to allow users to remotely configure the various components of the system 1 to deploy the required resources in a desired manner, locally. , remotely or in a combined manner, for example, based on the needs of the system and the characteristics of the analyzes being performed, while ensuring data exchange in a secure environment with the ability to encrypt.

[00767] In specific cases, system 1 may include a processing architecture 310, such as an interpreter, that is configured to implement an interpreting function 310. Interpreter 310 performs one or a series of analytical functions on the generated data, such as annotation 311, interpretation 312, diagnosis 313, and/or detection and/or analysis functions to determine the presence of one or more biomarkers, for example, in genetic data. The interpreter 310 may be part of or separate from the local computing resource 100, such as when the interpreter 313 is coupled to the computing resource 100 through a cloud interface, such as a hybrid cloud 50.

[00768] Additionally, additional processing architecture 320 may be included, for example, when architecture 320 is implemented as a collaborator. The collaborator 320 may be configured to perform one or more functions related to ensuring the security and/or confidentiality of the data to be transmitted. For example, the collaborator may be configured to secure the data sharing process 321, ensure confidentiality of the transmission 322, set control parameters 323, and/or initiate a security protocol 324. The collaborator 320 is configured to enable data sharing, for example, to facilitate shared processing, for example, the collaborator 320 may be part of or separate from the local computing resource 100, for example, when the collaborator is connected to the computing resource 100 through a cloud interface, such as a hybrid cloud 50. Interpreter 310, collaborator 320, and/or local computing resource 100 may also be associated with remote computing resource 300, for example, to improve system efficiency by offloading compute 300 and/or storage 400 functions to cloud 50. In various cases, the system 1 may be configured to allow secure analysis to be performed by a third party 121, for example where the third party can connect to and exploit the system, for example through a suitably configured API.

[00769] As shown in FIG. 43, system 1 may be a multi-tiered and/or multiplexed bioanalytical processing platform that contains layers of data generation and/or processing units, each of which has one or more processing pipelines that can be deployed systematically and simultaneously or sequentially to process genetic information , starting from the stage of its primary processing and ending with the stage of secondary and/or tertiary processing. In particular, presented herein are devices configured to perform bioanalysis in one or more hardware and/or software and/or quantum processing implementations, as well as methods for using them and systems containing them. For example, in one embodiment, a genomic analysis platform may be provided as a plurality of integrated circuits that may be configured as or otherwise included in one or more central or graphics processing units, such as a general purpose CPU and/or GPU. ,hardware implementation and/or quantum processing device. Specifically, in various embodiments, one or more of the pipelines of the genomic processing platform may be configured by one or more integrated circuits and/or quantum processor devices.

[00770] Accordingly, the platforms presented herein can be configured to leverage the enormous power of optimized software and/or hardware and/or quantum processing implementations to perform various genetic sequencing and/or secondary and/or tertiary processing functions. the processing described herein, which may be performed on one or more integrated circuits. Such integrated circuits may be seamlessly coupled together and may also be seamlessly coupled to various other integrated circuits, such as a CPU and/or GPU and/or GPU of a system that is configured to execute various software and/or hardware applications of tertiary bioanalytical functions .

[00771] Specifically, in various embodiments, these processes may be performed by optimized software running on a CPU, GPU, and/or GPU, and/or may be implemented as firmware-configurable integrated circuits, such as FPGAs, which may be part of the same device or separate devices, which may be located on the same motherboard, different PCIe cards within the same device, separate devices in the same institution, and/or in different institutions. Accordingly, one or more processor devices and/or integrated circuits may be directly coupled together, e.g., hardwired, e.g., by being physically incorporated into the same motherboard or separate motherboards housed in the same package and/or otherwise linked together in a manner, or may be located on separate motherboards or PCIe cards that can communicate with each other remotely, e.g., wirelessly and/or through a network interface, e.g., through a local cloud 30, and in various embodiments one or more processing devices and/or integrated circuits may be located geographically distant from each other, but communicate via the hybrid cloud 50. In specific cases, the integrated circuits that form or are part of the CPU, GPU and/or CPU may be designed as and /or be part of a secondary and/or tertiary analytics platform, may be configured to form one or more analysis pipelines where various generated data may be fed back and forth between different processing devices and/or integrated circuits, e.g., seamlessly or in streaming mode, to enable rapid data transfer between multiple integrated circuits and, more specifically, to speed up analyzes within them.

[00772] For example, in some cases, various devices for use in accordance with the methods described herein may contain or be otherwise coupled to one or more sequencing devices to perform a sequencing protocol, which may be accomplished by executing software on a remote sequencer, such as a next-generation sequencer such as Illumina's HiSeq Ten, located at the main sequencing center to make it accessible via a cloud interface. In other cases, sequencing may be performed in a hard-wired on-chip configuration, such as that implemented in Thermo Fisher's Ion Torrent sequencer, or in other on-chip sequencing technologies where sequencing is performed using semiconductor technology that enables next-generation benchtop sequencing, and /or by an integrated circuit configured as or otherwise including a field effect transistor using a graphene channel layer. In such cases, when sequencing is performed using one or more integrated circuits configured as or containing a semiconductor sequencing chip, they may be located remotely from one or more processing devices and/or integrated circuits described herein, which may be configured to perform secondary and/or tertiary analytics on sequenced data. In an alternative embodiment, the chips and/or processing devices may be located relatively close to each other so as to be directly coupled together, or at least in general proximity to each other, such as within the same institution. In this and other cases, sequencing and/or BioIT analytics pipelines can be configured such that the raw sequencing data generated by the sequencer can be quickly transferred, for example in streaming mode, to other analytical components of the pipeline for direct analysis, for example in streaming mode.

[00773] In addition, once raw sequencing data (eg, BCL data) or read data (eg, FASTQ data) is generated by a sequencing instrument, it may be transferred to and received by an integrated circuit, where the integrated circuit is configured to perform various bioanalytical functions on genetic and/or protein sequences, for example, for the purpose of analyzing generated and/or received DNA, RNA and/or protein sequence data. Sequence analysis may involve comparing a generated or received nucleic acid or protein sequence against one or more sequence databases, for example, to perform secondary analysis on the received data, and/or in some cases to perform a diagnosis of a disease, such as when a database of known sequences for performing the comparison may be a database containing data of morphologically different and/or incompatible sequences, which are data of genetic samples related or believed to be related to one or more disease states.

[00774] Accordingly, in various cases, isolated and sequenced genetic, e.g., DNA and/or RNA, data may be subject to secondary analyzes that may be performed on the received data, for example, to perform mapping, alignment, sorting, variant identification, etc. .P. for the purpose of generating mapped and/or aligned data, which can then be used to produce one or more VCF files detailing the difference between the mapped and/or aligned genetic sequence and the reference sequence. In particular, upon completion of secondary processing, the genetic information can be transferred to one or more tertiary processing modules of the system, for example, for further processing there, for example, to obtain results regarding therapy and/or prevention. More specifically, once variants have been identified, the mapper/aligner/variant definer can produce a standard VCF file that is ready and can be transferred to a complementary integrated circuit to perform tertiary analyses, such as genome-wide analyses, e.g., genome-wide analyses, genotyping analyses, e.g. , co-genotyping, microarray analysis, exome analysis, microbiome analysis, epigenome analysis, metagenome analysis, co-genotyping analysis, variation analysis, such as GATK analysis, structural variant analysis, somatic variant analysis, etc., as well as RNA sequencing analysis and others genomic analyses.

[00775] Therefore, the bioanalytical, e.g., BioIT, platform provided herein may contain highly optimized algorithms for mapping, alignment, sorting, duplicate tagging, haplotype variant detection, compression and/or decompression, e.g., in a hard-wired and/or or quantum processing. For example, while one or more of these functions may be configured to run in a fully or partially hard-wired configuration, in specific cases the secondary and/or tertiary processing platform may be configured to run one or more software applications and/or quantum processing applications , for example, one or more of the functions described herein. In particular, the sequenced and/or mapped and/or aligned and/or otherwise processed data may then be subjected to further processing using one or more optimized algorithms for one or more of whole genome analysis, genotyping analysis, microarray analysis, exome analysis, microbiome analysis, epigenome analysis, metagenome analysis, co-genotyping analysis and/or variant analysis, for example GATK, for example, implemented using software running on a CPU and/or GPU and/or general purpose CPU, although in certain cases, one or more of these functions may be at least partially implemented in hardware.

[00776] Accordingly, as shown in FIG. 43, in various embodiments, multiplexed bioanalytical processing platforms are configured to perform one or more of primary, secondary, and/or tertiary processing. For example, the primary processing stage creates genetic sequence data, for example, in one or more BCL and/or FASTQ files, for transmission to system 1. Once in system 1, the sequenced data, including associated metadata, can be advanced to a secondary processing stage 600 to create one or more variant definition files. Therefore, the system may also be configured to take one or more variant definition files together with associated metadata and/or other associated processed data, and perform one or more other operations on one or more tertiary processing stages, for example, to perform on them one or more diagnostic and/or prophylactic and/or therapeutic procedures.

[00777] In particular, data analysis may be initiated, for example, in response to a user request 120, such as made from a remote computing resource 100, and/or in response to data submitted by a third party 121, and/or data automatically retrieved from local storage 200 and/or remote storage 400. Such further processing may include a first tier of processing where the various pipelines executing protocols 700 are configured to perform analytics on specific genetic data, eg, variation, of one or more subjects. For example, first tier tertiary processing devices may include a genomic processing platform that is configured to perform genomic, epigenome, metagenome, genotyping, and/or various variant analyses, and/or other bioinformatics-based analyses. In addition, the second tier of tertiary processing may perform various disease diagnostic, research and/or analytical protocols 800, where the analysis may include one or more of NIPT, NICU, cancer, LDT, biology, AgBio, etc. applications. P.

[00778] System 1 may also be configured to receive and/or transmit various data 900 related to the procedures and processes described herein, such as related to electronic medical records (EMR) data, testing and/or structuring data Federal Drug Administration (USA), annotation-related data, etc. Such data may be useful in providing the user with the ability to use and/or access the generated medical, diagnostic, therapeutic and/or preventive methods developed through the use of the system 1 and/or accessible through it. Accordingly, in various cases, the devices, methods and systems presented herein allow genetic and bioanalytical analysis to be performed securely and the results securely communicated to a forum that can be easily accessed for downstream processing. In addition, in various cases, the devices, methods and systems provided herein allow data to be securely transferred to a system, for example, from one or more health monitoring and/or data storage agencies and/or a government agency such as the FDA or NIH. For example, the system may be configured to securely receive electronic medical record/personal health record (EMR/PHR) data, for example, that may be transmitted from a healthcare facility and/or repository, for use in accordance with the methods described herein, for example, to perform genetic and bioanalytical analysis and securely transmit the results to a forum that can be easily accessed for processing further down the chain.

[00779] In particular, the first tertiary processing tier 700 may include one or more genomic processing platforms, for example, to perform genetic analysis, for example, on mapped and/or aligned data, for example, in a SAM or BAM file format, and/ or to process variant data, for example in VCF format. For example, the first tertiary processing platform may include one or more of a genome pipeline, a metagenome pipeline, a co-genotyping pipeline, and one or more variant analysis pipelines, including: a GATK pipeline, a structural variant pipeline, a somatic variant detection pipeline, and, in some cases, may include an RNA sequencing analysis pipeline. One or more genomic analysis pipelines may also be included.

[00780] More specifically, as shown in FIG 43, in various cases, a multi-tiered and/or multiplexed bioanalytical processing platform includes an additional layer of data generation and/or processing devices. For example, in certain cases, a bioanalytical processing platform comprises one or more processing pipelines in one or more software and/or hardware implementations that are related to the execution of one or more tertiary processing protocols. For example, in specific cases, the tertiary analysis pipeline platform 700 may include one or more of a genome pipeline, an epigenome pipeline, a metagenome pipeline, a co-genotyping pipeline, a variation pipeline such as a GATK pipeline, and/or other pipelines such as an RNA pipeline. In addition, the second layer of the tertiary processing analysis platform may include a number of processing pipelines, such as one or more of a microarray analysis pipeline, a genome analysis pipeline, e.g., a whole genome analysis pipeline, a genotyping analysis pipeline, an exome analysis pipeline, a microbiome analysis pipeline, an analysis pipeline genotyping, including co-genotyping, variant analysis pipelines, including structural variant pipelines, somatic variant pipelines, and GATK and/or MuTect2 pipelines, as well as RNA sequencing pipelines and other genetic analysis pipelines.

[00781] Accordingly, in one embodiment, the stacked bioanalytical processing platform includes a metagenomic pipeline. For example, a metagenomic pipeline may be included, for example, to perform one or more environmental genomics processes. Specifically, in various embodiments, metagenomic analysis may be performed to determine whether a group of organisms has evolved from a common ancestor, such as a species or another clade. More specifically, in various embodiments, an environmental sample may be obtained containing a variety of living and/or dead organisms from which the DNA/RNA present can be isolated, sequenced, and processed using one or more of the processing platforms described herein to identify the present species and/or one or more other genomic factors related to it. Such “environmental” samples may include a variety of human microbiomes (eg, those related to microorganisms found in association with healthy and diseased individuals, including microorganisms found in skin, blood, sputum, stool samples), as well as external factors environment.

[00782] There are many ways to obtain sequenced genetic samples to perform metagenomic processing. The first method involves a protocol for directed cloning of 16S ribosomal RNA and/or gene sequencing. For example, 16S ribosomal RNA varies greatly among species (or even strains of a species). Accordingly, this RNA can be isolated and sequenced to create a genetic profile of biodiversity derived from naturally occurring biological samples that can be used to inform AI or other system databases. However, the problem with such sequencing is that much microbial biodiversity may be lost simply due to the way in which it was cultured.

[00783] Accordingly, the second method includes a shotgun and/or PCR-based protocol that can be used to sample multiple, e.g., all, genes from all biological factors of the communities from which the samples are taken, in a single sequencing can reveal the genetic diversity of microscopic life. In particular, shotgun sequencing can generate an aggregated reference sequence, for example, from many (for example, tens of thousands) reference genomes of different species. However, the combined size of these multiple genomes is gigantic. Therefore, to construct an aggregated reference genome, it is advisable to select one or more distinct subsequences from each reference genome.

[00784] For example, such a subsequence can range in length from a few hundred bases to several thousand bases, with unique sequences not found in other species (or strains) being ideal. These subsequences can then be aggregated to construct reference sequences. Accordingly, isolated, sequenced, mapped and aligned metagenomic sequences can be compared with partial or complete reference genomes for many species and determine biodiversity.

[00785] Metagenomics thus provides a powerful “magnifying glass” for viewing the microbial world that could revolutionize our understanding of the living world. Therefore, in either of these two cases, where there is a significant presence of DNA from organisms in the sample, those species can be identified as living in that environment. Ideally, this will identify species that are unusual among species typically present in a given environment. In particular, when the coverage of all species is normalized for the resulting environmental samples, the genetic diversity of the species present can be determined and compared with the entire coverage, for example, by comparing part of the DNA of a particular organism with the corresponding part of the generated biologically diverse reference genetic sequence.

[00786] The significance of these analyzes can be determined using Bayesian methods, for example, by estimating the probability of observing the sequenced reads of a particular organism given the presence or absence of that organism. Bayesian probability methods aim to describe the probability of an event based on conditions that could be associated with that event. For example, if you want to determine whether a subject has cancer, and the age of the subject is known, and the cancer you want to determine is age-related, then using Bayes' theorem, information about the age of the subject can be used to more accurately estimate the likelihood of cancer.

[00787] Namely, through the interpretation of Bayesian probability, the theorem expresses how the subjective degree of confidence can rationally change given observed evidence. Bayes' theorem is mathematically formulated in the following equation: P =P P /P , where A and B are events and P(B) ≠ 0. P(A) and P(B) are the probabilities of observing A and B regardless of each other. P(A|B) is a conditional probability, which is the probability of observing event A given that B is true. P(B|A) - the probability of observing event B, provided that A is true.

[00788] Accordingly, one or more steps of performing Bayesian probability analyzes in a given context may include one or more of the following: determinations of the presence of clades may be made at various taxonomic levels: kingdom, phylum, class, order, family, genus, species, and /or strain. However, this is complicated by the fact that, in general, the lower the taxonomic levels occupied by organisms, the more similar their DNA is. In addition, often a sample may match a reference genome for multiple species at a higher taxonomic level (or multiple strains of the same species), and therefore in many cases only a more general clade (such as a genus or family) can be defined, rather than specific species or strain. However, to overcome these and other such difficulties, the devices, systems and methods for using them described herein can be used.

[00789] Specifically, in one embodiment, a method is provided for determining the presence of two or more species or clades of organisms in a provided sample. For example, in a first step, genomic sequence data reads can be obtained from the sample, for example where the reads can be in FASTQ or BCL format. Genomic surface mapping can be performed to map reads to multiple genomic reference sequences. In this case, the genomic reference sequences may be a complete genome, or may be a partial genome, to reduce the amount of data required for each species, strain, or clade. However, using a larger genome will increase the sensitivity of detection, and each reference sequence used should be selected to represent a species, strain, or clade that will be distinct from each other.

[00790] All or part of the genomic sequence from the 16S ribosome can be used for this purpose. Thus, two or more genomic reference sequences of species, strains, or clades of organisms suspected of being present in a sample can be constructed to detect members of those groups in the sample. Once the genomic reference sequences have been constructed, an index can also be created for each of them. Indexes can be hash tables or tree indexes, such as a prefix or suffix tree index. Once an index is constructed, the sample's genomic sequence reads can be compared to each of two or more indexes. It can then be determined whether the image's genomic sequence reads map to each of the indices.

[00791] Similarly, genomic sequence reads can also be aligned to the genomic sequences to which they are mapped. This will generate an alignment score according to the methods described herein, which can be used in analyzing the likelihood that a read indicates the presence or absence of a species or clade of an organism in a sample. In particular, mapping and/or alignment can be accomplished using the provided software and/or hardware modules, as described herein. In some implementations, the mapped and aligned data may then be transferred to the 100/300 computing resource for further analysis and processing.

[00792] For example, mapped and/or aligned genomic sequence reads can be analyzed to determine the likelihood that an organism having a given genomic reference sequence is present in the sample. Similarly, a list of species, strains or clades determined to be present in the environmental sample may be reported. In certain implementations, the list may be reported along with a confidence metric (eg, a P-value) to indicate the statistical significance of the estimate. A complete list of species, strains, or clades of organisms analyzed may also be reported, along with an indication of each species, strain, or clade that was present and the confidence metric. It should be noted that the various methods and procedures described herein, although described in relation to microbiome analysis, can be used in the analysis of all other tertiary processing protocols where appropriate.

[00793] For example, in FIG. 43B shows an example implementation of a method for performing environmental analysis of, for example, microbiomes in an environmental sample. For example, in the first case, it is possible to obtain an environmental sample and isolate a variety of genetic material from it. This diverse material can then be processed and sequenced, for example, using a suitably configured SNP.

[00794] As a result, in a first step 1000, after the diverse genetic material has been sequenced, for example, using SNR, it can be transferred to the system 1 described herein. At step 1010, one, two, or more genomic reference sequences of interest, for example, to be detected in the sample, can be constructed. At step 1020, an index may be constructed for one, two, or more genomic reference sequences. Next, at step 1030, the resulting sequenced reads from the genomic sample can be compared to one, two, or more indices, for example, using a suitably configured mapping module. At step 1040, it can be determined whether the sequenced reads of the genomic image map to each of two or more indices.

[00795] At this time, if desired, in step 1050, the mapped reads can be aligned to genomic reference sequences to generate an alignment and/or an alignment estimate. Accordingly, after the resulting genetic materials in the sample have been mapped and/or aligned, at step 1060, the likelihood that a given organism having the reference sequence is present in the sample can be determined. And after processing, the list of species, strains and/or clades that are present in the sample can be identified and/or reported.

[00796] The tertiary processing platform described herein may also be included in an epigenomic pipeline. In particular, epigenetics is the study of genetic influences that are not encoded in the DNA sequence of an organism. The term also refers to the changes themselves: referring to the functional properties of changes in the genome that do not entail changes in the nucleotide sequence. However, epigenetic changes are stably heritable phenotypes resulting from changes in the chromosome that do not change the DNA sequence. These changes may or may not be heritable. Specifically, epigenetic changes modify the activation of certain genes, but not the genetic code of the DNA sequence. It is the DNA microstructure itself (not the code) or associated chromatin proteins that can cause activation or silencing that can be modified.

[00797] The epigenome is involved in the regulation of gene expression and development. tissue differentiation and transposon suppression. Unlike the underlying genome, which is largely static within an individual, the epigenome can be dynamically altered by environmental conditions. This field is similar to genomics and proteomics, which study the genome and proteome of a cell. Additionally, epigenomics involves the study of the complete set of epigenetic modifications on a cell's genetic material, known as the epigenome, which consists of a record of chemical changes in the DNA and histone proteins of an organism. These changes can be passed on to the organism's descendants through transgenerational epigenetic inheritance. Changes in the epigenome can lead to changes in chromatin structure and changes in genome function.

[00798] This epigenetic mechanism allows differentiated cells in a multi-molecular organism to express only the genes necessary for their own activity. As cells divide, epigenetic changes persist. In particular, most epigenetic changes can only occur during the lifespan of an individual organism. However, if gene inactivation occurs in sperm or eggs that lead to fertilization, then some epigenetic changes may be passed on to the next generation. Several types of epigenetic inheritance systems may be involved in what has become known as cellular memory. For example, various covalent modifications of either DNA (eg, cytosine methylation and hydroxymethylation) or histone proteins (eg, lysine acetylation, lysine and arginine methylation, serine and threonine phosphorylation, and lysine ubiquitination and sumoylation) may play central roles in many types of epigenetic inheritance. Because the phenotype of a cell or individual is influenced by which of its genes are transcribed, inherited transcriptional states can lead to epigenetic consequences. Such consequences for cellular and physiological phenotypic traits may result from external or environmental factors that turn genes on and off and influence how cells express genes.

[00799] For example, DNA damage can cause epigenetic changes. DNA gets damaged very often. These lesions are largely repaired, but epigenetic changes may persist at the site of DNA repair. In particular, DNA double-strand breaks can initiate unprogrammed epigenetic gene silencing, both by inducing DNA methylation and by promoting the silencing of types of histone modifications (chromatin remodeling). Other examples of mechanisms that create such changes include DNA methylation and histone modification, each of which alters gene expression without changing the underlying DNA sequence. Nucleosome remodeling has been shown to cause epigenetic silencing of DNA repair. Moreover, DNA-damaging chemicals can also cause significant DNA hypomethylation, for example by activating oxidative stress pathways. In addition, gene expression can be controlled by the action of repressor proteins that attach to DNA silencing regions.

[00800] These epigenetic changes can last through cell divisions throughout the life of the cell, and can also last for many generations, even if they do not involve changes in the underlying DNA sequence of the organism; but non-genetic factors force the body’s genes to change behavior (or “express themselves”). One example of epigenetic change in eukaryotic biology is the process of cell differentiation. During morphogenesis, totipotent stem cells become the various pluripotent cell lineages of the embryo, which in turn become fully differentiated cells. In other words, as one fertilized egg—the zygote—continues to divide, the resulting daughter cells become all of the different types of cells in the body, including neurons, muscle cells, epithelium, blood vessel endothelium, etc., by activation of some genes while suppressing the expression of other genes.

[00801] There are several levels of regulation of gene expression. One pathway for gene regulation is through chromatin remodeling. Chromatin is a complex of DNA and the histone proteins with which it is associated. If the order in which the DNA wraps around the histones changes, gene expression may also change. Firstly, this occurs due to post-translational modification of amino acids, which compensates for the lack of histone proteins. Histone proteins are made up of long chains of amino acids. If the amino acids in the chain are changed, the shape of the histone can be modified. During replication, DNA is not completely unwound. It is therefore possible that modified histones may be introduced into each new copy of DNA. Once there, these histones can act as templates that initiate the environment of new histones to form a new form. By changing the shape of the histones around them, these modified histones will ensure the maintenance of the lineage-specific transcription program after cell division.

[00802] Secondly, it occurs by adding methyl groups to the DNA, mainly at CpG sites, to convert cytosine to 5-methylcytosine. 5-methylcytosine acts much like normal cytosine, pairing with guanine in double-stranded DNA. However, some regions of the genome are more methylated than others, and highly methylated regions tend to have less transcriptional activity through a mechanism that is not fully understood. Cytosine methylation can also pass from the germ line of one parent to the zygote to form a chromosome that is inherited from one parent or the other (genetic imprinting). Although histone modifications occur throughout the entire sequence, the unstructured N-termini of histones (called histone tails) are particularly highly modified. These modifications include acetylation, methylation, ubiquitination, phosphorylation, sumoylation, ribosylation, and citrullination.

[00803] Accordingly, DNA methylation is the presence of a methyl group on some DNA nucleotides, especially "G" bases or "CpG" dinucleotides. Methylation in promoter regions tends to repress gene expression. Methylation analysis is the process of discovering which “C” bases are methylated in the genome of a given sample. Bisulfite sequencing (MethylC-seq) is the most common way to detect methylation using whole genome sequencing, with unmethylated cytosine (“C”) bases chemically converted to uracil (“U”) bases, which become thymine “T” bases after PCR amplification . Methylated "C" bases are resistant to conversion.

[00804] Therefore, in accordance with the devices and methods described herein, it is proposed to detect modifications of DNA molecules, where the modifications do not affect the DNA sequence but affect gene expression, for example, by performing one or more mapping and/or alignment operations on epigenetic genetic material. In such methods, the resulting reads can be mapped and aligned to a reference genome in a manner that allows for alignment of the converted "T" bases to the "C" position of the reference, and the "C" bases in the reference sequence can be replaced with "T" bases before mapping/alignment. . This allows the precise mapping and alignment of reads where the C bases are bisulfite-converted (now T), resulting in non-sulfite-converted (methylated) C bases found in the genomic sequence reads. For reverse-complementary alignments, complementary substitutions can be used, e.g., all " G" can be replaced with "A".

[00805] Likewise, the reference index builder (eg, hash table) and mapper/aligner can be modified to perform these replacements automatically to use MethylC-seq sequencing. In an alternative embodiment, the mapper/aligner may be modified to allow direct alignment of a “T” read to a “C” reference and reverse complementary alignment of an “A” read to a “G” reference. The methods described herein improve accuracy and prevent erroneous direct alignment of a “C” read to a “T” reference or erroneous reverse complementary alignment of a “G” read to an “A” reference.

[00806] Further provided herein are methods for determining the methylation state of cytosine bases in genomic sequence reads. For example, in a first step, genomic sequence reads can be obtained from samples with bisulfite-treated nucleotides. In this regard, in particular, one or more modified protocols can be used to generate a read for secondary processing. Specifically, one or more of: bisulfite sequencing at reduced representation can be used to detect DNA methylation across all parts of the genome at varying levels of resolution down to the base pair level; sequencing by immunoprecipitation of methylated DNA; and sequencing using methyl-sensitive restriction enzymes. In addition, chromatin can be accessed where it is accessible, for example when DNase I hypersensitive site sequencing can be performed, for example when open or accessible regions in the genome can be found using the DNase I enzyme. Additionally, RNA sequencing and expression arrays can be used to identify the expression levels of protein-coding genes. In particular, mmRNA sequencing can be used to determine the expression of small non-coding DNA, primarily miRNA.

[00807] Therefore, after sequencing to generate reads, a genomic reference sequence can be constructed for comparison with the reads. The CpG locations in the genomic reference sequence can then be marked. Next, you can pre-process the genomic reference sequence by replacing all “C”s in it with “Ts”. An index can be constructed for the genomic reference sequence. And once the index is built, the sample's genomic sequence reads can be compared to the index and determine whether the epigenomic sequence reads map to the index.

[00808] Next, the mapped reads can be aligned to a genomic reference sequence to generate an alignment score. In certain embodiments, base swaps can be performed in the sequence of the read, and the read can be compared and realigned to the index again. In some embodiments, alignment orientation constraint may be used during read mapping and/or alignment such that only forward alignments with C to T substitutions in the read and the genomic reference sequence are allowed, and only reverse complementary alignments with G to A substitutions are allowed. in read and reference genomic sequence.

[00809] These mapping and alignment procedures can be accomplished using various software and/or hardware modules described herein. In some implementations, the mapped and aligned data can then be transferred to the CPU/GPU/GPU for further analysis and processing. For example, mapped and aligned reads can be sorted by their reference mapping position. In some embodiments, duplicates can be marked and removed. For each marked CpG location in the reference, overlapping reads from the read pool can be analyzed. In this case, the thiamine (T) that replaced the cytosine (C) indicates unmethylated cytosine and is labeled as such. And cytosine that remains in the read sequence can be labeled as methylated cytosine. It is also possible to label reverse complementary CpG alignments as methylated or unmethylated. For example, a guanine (G) that replaces an adenine (A) is marked as the reverse complement of an unmethylated cytosine (C), whereas a guanine (G) that remains in the read sequence is marked as the reverse complement of a methylated cytosine (C). The probable methylation state of each CpG location on each nucleotide strand can be reported, and an associated confidence metric (eg, p-values) of the methylation determination can be generated. In some embodiments, the methylation state of the marked CpG locations for each chromosome of a diploid chromosome pair may also be indicated.

[00810] With respect to histone modification, this includes various naturally occurring chemical modifications to the histone proteins around which the DNA is wrapped, resulting in the DNA being wrapped more or less tightly. Loosely wrapped DNA, for example, is associated with higher rates of gene expression. Such histone modifications can be determined by chromatin immunoprecipitation sequencing (ChIP-Seq), which can be used to identify genome-wide patterns of histone modifications, for example, using antibodies to the modifications. Additionally, ChIP-seq is a technique that can be used to isolate and sequence DNA that is closely associated with histones (or other selected proteins). Once ChIP-seq is performed, the sample can be prepared, the DNA isolated and sequenced, and the sequenced DNA can then be mapped/aligned to a reference genome as described herein, and the mapped coverage can be used to infer the level of histone binding at various loci in the genome. In addition, this document provides methods for analyzing nucleotide sequences obtained by ChIP, which are similar to the methods described below for the analysis of structural variants.

[00811] Of particular note, epigenetics is useful in cancer research and diagnosis. For example, human tumors are subjected to extensive DNA methylation disruption and histone modification patterns are examined. In fact, the aberrant epigenetic landscape of the cancer cell is characterized by global genomic hypomethylation, hypermethylation of the promoter CpG islands of tumor suppressor genes, altered histone code for critical genes, and global loss of monoacetylated and trimethylated histone H4. Accordingly, the methods described herein can be used for cancer research and/or diagnosis purposes.

[00812] In addition, the methods described herein can be used to generate one or more epigenomic databases and/or reference genomes. For example, the methods described herein, for example, using AI system training protocols, may be useful for generating reference epigenomes for humans, for example, using normal, healthy individuals across a wide range of cell lines, primary cells and/or primary tissues. Once generated, such data can be used to improve the mapping and/or alignment protocols described herein. In addition, once a database of epigenomic differences has been generated, it can be subject to mining, for example using an AI module, to better characterize and identify relevant factors that occur in various disease states such as cancer, Alzheimer's disease and other neurological diseases.

[00813] Accordingly, in various cases, epigenomic analysis can be performed to identify one or more or a complete set of epigenetic modifications that have occurred on the genetic material of a cell. In particular, using the methods described herein, the epigenome of an organism and/or its cells can be determined to catalog and/or record chemical changes in the DNA and histone proteins of the cells of the organism. For example, an example of epigenomic analysis is shown herein in FIG. 43C.

[00814] For example, the first step may be to obtain a genomic sample from an organism, isolate the genetic material from it, and sequence it. As a result, after sequencing at step 1000, the sequenced reads of the sample can be transferred to system 1, where it will be accepted. In this case, reads can be obtained from bisulfite-treated nucleotides of the sample. Similarly, at step 1010, a genomic reference sequence, for example, for an organism, can be constructed, for example, to perform a comparison of epigenomic reads of a sample. At step 1012, any different CpG locations in the genomic reference sequences can be identified.

[00815] Once identified at step 1014, the “C” in the CpG locations in the reference can be replaced with “T” and an index for the modified genomic reference sequence can be generated at step 1020. After generating the index for the modified reference, at step 1030, the sample's genomic sequence reads can be compared to the index, and at step 1040, it can be determined whether the sample's genomic sequence reads are mapped to the index, for example, mapped in accordance with the methods and apparatuses described herein. The mapped reads can then be aligned to a genomic reference sequence and an alignment estimate generated, for example, by performing one or more of the alignment operations discussed herein.

[00816] At this stage, one of a variety of different analyzes can be performed. For example, at step 1051, if more context is needed, base substitutions in the reads that are processed above and/or alignment orientation and/or parameter constraints can be adjusted and comparison steps 1030-1050 can be repeated. This process itself can be repeated as needed until a sufficient level of context is achieved. Accordingly, once a sufficient level of context has been achieved, the mapped and/or aligned reads at step 1080 can be sorted, for example, in the process described herein, by the mapped/aligned reference position. At step 1081, any duplicate reads can be marked and/or removed.

[00817] Next, at step 1082, reads from the pool of reads overlapping each flagged reference CpG location can be analyzed. Where "T" is replaced by "C", the "C" may be noted as unmethylated at step 1083; and where "C" remains in the sequence, at step 1084, "C" can be noted as methylated "C". Finally, at step 1086, the likely methylation state of each CpG location on each nucleotide strand may also be determined/reported, as well as the confidence of the methylation determination.

[00818] Further provided herein are methods for analyzing genomic material, wherein a portion of the genetic material may have a structural variant or be otherwise related thereto. Specifically, structural variation is variation in the chromosome structure of an organism. Structural variations include many types of variations in a species' genome, including microscopic and submicroscopic types such as deletions, duplications, copy number variations, insertions, inversions, and translocations. Many structural variants are associated with genetic diseases. Indeed, about 13% of the human genome is defined as structural variant in the normal population, and there are at least 240 genes that exist as homozygous deletion polymorphisms in human populations. Such structural variations may contain millions of nucleotides of heterogeneity in each genome and are likely to be an important contributor to human disease susceptibility.

[00819] Copy number variation is a large category of structural variation that includes insertions, deletions, and duplications. There are several versions known to be associated with human illness. For example, a recurrent 400kb inversion in the factor VIII gene is a common cause of hemophilia A, and smaller inversions affecting iduronate-2-sulfatase will cause Hunter syndrome. Other examples include Angelman syndrome and Sotos syndrome. The most common types of complex structural variation are non-tandem duplications, where a sequence is duplicated and inserted in an inverted or forward orientation into another part of the genome. Another class of complex structural variants includes deletion-insertion-deletion combinations, duplication-insertion-duplications, and tandem duplications with nested deletions. There are also cryptic translocations and segmental uniparental disomy (UPD).

[00820] However, detection of abnormal DNA structures is problematic and beyond the scope of variant definitions known to date. Structural variants that are difficult to detect include those that have: large insertions and deletions (eg, indels larger than 50-100 base pairs); duplications and other copy number variations (CNVs); insertions and translocations along with aneuploidy (abnormal number of chromosome copies: monosomy, disomy, trisomy, etc.). In certain cases described herein, identified copy number variations can be tested in subjects free of genetic diseases, for example, using quantitative SNP genotyping.

[00821] Detection of structural variation typically begins by performing an alignment mapping operation using the devices and methods described herein. For example, reads from a genomic sample to be analyzed can be mapped and aligned to a reference genome, for example, in a protocol that supports chimeric alignments. Namely, some structural variants (e.g., CNVs and aneuploidy) can be detected by analysis relative to mapped coverage. However, other structural variants (eg, large indels, inversions, translocations) can be detected by analysis of trimmed and chimeric alignments.

[00822] Namely, each structural variant includes one or more “break” positions where the read does not map to the reference genome, for example, when the geometry changes between the sample and the reference. In such a case, the cluster can be configured so that reads within it that slightly overlap structural variant breaks can be trimmed at the break, and reads that significantly overlap structural variant breaks can be chimerically aligned. However, pairs of reads spanning structural variant breaks may be inconsistently aligned when the two paired reads map to completely different reference locations and/or with an anomalous relative orientation of the paired reads. Such obstacles can be overcome using the methods described herein.

[00823] For example, in certain cases, data related to known structural variants can be used to better determine the sequence of a structural variant. For example, a database could be compiled having a list of structural variations in the human genome, eg, focusing on CNVs, and such data could be used in sequencing specific variants, eg, in an appropriately configured weighing protocol. In particular, when a structural variant is known, its "inner" and "outer" coordinates can be used as the minimum and maximum sequence range that can be affected by the structural variation. In addition, known variations in the form of insertion, loss, gain, inversion, PGZ, eversion, translocation and ORD can be classified and entered into the knowledge base of the presented system.

[00824] In various cases, determination of a structural variant can be performed using suitably configured software running on the CPU/GPU/GPU, for example, by using previously determined sequencing data, and in other cases, analysis of structural variants can be performed, for example, in hardware provision described in this document. Accordingly, in specific cases, a method for analyzing genomic sequences for structural variants is provided. For example, in the first step, genomic sequence reads can be obtained from a sample of nucleotides. In certain cases, sequenced reads can be generated from paired-end or conjugate paired read protocols to detect structural variants. An index of the genomic reference sequence may then be constructed, for example where the index may be a hash table or a tree, such as a prefix or suffix tree. Once the index is constructed, the sample's genomic sequence reads can be compared to the index to determine whether the genomic sequence reads map to the index. If so, the sample's genomic sequence reads can then be aligned to the genomic reference sequence to which they are mapped, and an alignment score can be determined.

[00825] As stated above, mapping and/or alignment can be accomplished using a hardware module as described herein. In some implementations, the mapped and aligned data can then be transferred to the associated CPU/GPU/CPU for further analysis and processing. These reads can be sorted by mapped reference positions, and duplicate reads can be marked and removed. Chimeric read alignments and/or unusual relative alignments of two paired reads can be determined, and based on the detected chimeric read alignments and/or unusual relative alignments, possible structural variants (eg, large indel, inversion, or translocation) can be determined. In a similar way, one can calculate the posterior probabilities of each possible structural variant. In some embodiments, haplotypes of structural variants can be determined, for example, using HMM analysis on chimeric read alignments and/or unusual relative alignments. For example, a pairwise HMM can be used for such a determination. Paired HMM can be implemented using a hardware module.

[00826] Accordingly, in various cases, as shown in FIG. 43D, a method for identifying variants in the chromosome structure of an organism is presented. For example, in accordance with the methods described herein, at step 1000, genomic sequence reads may be received. At step 1010, one or more reference sequences may be constructed to perform a comparison between the reads and the reference sequences. Namely, at step 1010, a genomic reference sequence can be constructed to allow comparison of the received reads with the generated reference. More specifically, for these purposes, an index for the genomic reference sequence may be generated at step 1020, for example, a hash table or prefix/suffix tree may be generated at step 1020. As a result, at step 1030, the genomic sequence reads of the sample can be compared with the generated index, for example, in accordance with software and/or hardware implementations described herein.

[00827] If it is determined at step 1040 that the sample's genomic sequence reads are mapped to an index, then at step 1050 the mapped reads can be aligned to a genomic reference sequence and an alignment score generated. At step 1080, the sample reads can be sorted by their mapped reference positions. At this time, at step 1081, duplicate reads can be marked and removed. Further, at step 1090, chimeric read alignments and/or unusual relative alignments may be detected, e.g., two conjugate reads may be detected, and at step 1092, possible structural variants may be determined, e.g., based on the detected chimeric read alignments and/or unusual relative alignments. . In addition, posterior probabilities of each possible structural variant can be calculated and, optionally, at step 1096, haplotypes of the structural variants can be determined, for example, using HMM analysis as described herein, chimeric read alignments, and/or unusual relative alignments.

[00828] In addition, the devices, systems and methods described herein can be used to process RNA sequences. In particular, methods are provided herein for analyzing an RNA sequence, for example, using a splice mapping and alignment protocol (eg, using a suitably configured RNA mapper/aligner). For example, in one embodiment, a transcriptome pipeline may be provided, for example, for ultra-fast analysis of RNA sequence data. In particular, this pipeline may be configured to perform secondary analysis of RNA transcripts, for example, with respect to reference-only alignments, as well as alignments with using annotations).

[00829] Accordingly, in the first method, raw read data, for example in a BCL and/or FASTQ file format, can be generated using a sequencing instrument and entered into a system where mapping, alignment and variant determination can be performed. However, in various cases one or more gene annotation files (GTFs) can be entered into the system, for example to guide spliced alignments, for example splice boundary LUTs can be constructed and used. For example, alignment accuracy and splice boundaries tables can be used. Accordingly, a 2-phase alignment can be performed, for example where new splice boundaries can be used in the first alignment phase, which can then be used to guide the execution of a second mapping/alignment phase. Once the options have been identified, the system will output a standard VCF file ready for tertiary analysis.

[00830] In particular, after the input file is accepted, splice mapping and alignment can be performed, for example, on single-end and paired-end reads. As stated, configurable edge filters can be used to output a single edge. You can sort by position, which may include groupings according to the reference, and then sorting the groups by reference position, and you can flag duplicates, for example, based on the starting position and CIGAR string, to generate a high-quality duplicate report, with with which you can remove all duplicates. Haplotype variant detection can then be performed, for example using the SW and HMM processing engine.

[00831] In addition, the devices, systems and methods described herein can be used to perform somatic variant detection. For example, a somatic variant detection protocol can be used to detect variants occurring in cancer cells. In particular, genomic samples for the determination of somatic variants can be obtained from or biopsies of one or multiple tumors. Optionally, a “normal” (non-tumor) sample may also be obtained, for example for comparison during variant detection, for example where somatic variants will occur in tumor cells but not in cells of the normal sample. DNA/RNA can be isolated from samples and sequenced, for example, using a next-generation sequencer. Sequenced data from, for example, each sample can then be transferred to a secondary processing platform and the reads can be mapped and aligned. Next, reads can be subjected to a variety of variant detection procedures, including processing using one or both of the SW engines and the paired HMM.

[00832] However, the system must be capable of detecting variants with low allele frequency, such as 3% to 10% (or higher). More specifically, a genotyping probability model may be used that is configured to provide arbitrary allele frequencies. One way to provide this capability is to assign allele frequencies for each variant genotype that correspond to the observed allele frequencies in the overlapping reads. For example, if 10% of the overlapping reads exhibit a particular variant, one can test the genotype, states 90% from the reference allele and 10% from the other allele. In the case of dual tumor/normal tissue samples, one can estimate the posterior probability that a variant is present in the tumor sample but absent in the normal sample.

[00833] For example, a somatic variant detector pipeline may be configured to provide information about tumor heterogeneity, such as that a series of different mutation events have occurred, such as when one or more tumor sections of different genotypes (subclone) are identified. Such subclone information can be obtained from determining variant allele frequencies and their distributions, and/or solely by differentially identifying variants among multiple tumor samples.

[00834] Accordingly, methods are provided for detecting sequence variants of cancer cells from a sample. In the first step, genomic sequence reads can be obtained from a sample of nucleotides from cancerous and/or normal cells. Sequence reads can be from paired-end protocols or conjugate paired reads, similar to those used to detect structural variants. An index of a genomic reference sequence may be constructed, for example where the index may be a hash table or a tree, such as a prefix or suffix tree. The genomic sequence reads of a sample, such as a tumor sample and/or a normal sample, can be compared to the index and determine whether the epigenomic sequence reads map to the index.

[00835] The sample's genomic sequence reads can then be aligned to the genomic reference sequence to which they are mapped, and an alignment score can be generated. Mapping and/or alignment can be accomplished using software and/or hardware modules as described herein. In some implementations, the mapped and aligned data can then be transferred to the CPU/GPU/GPU for further analysis and processing. These reads can be sorted by mapped reference positions, and any duplicate reads can be flagged and removed. Variants can be detected using Bayesian analysis, which is modified to expect arbitrary variant allele frequencies and to detect and report possible low allele frequencies (e.g., 3% to 10%).

[00836] In some embodiments, generative variants may be found in both non-cancerous and cancer samples, while somatic variants may only be found in cancer samples. For example, generative and somatic mutations may differ in relative frequency. For each possible cancer variant, posterior probabilities can be calculated, and in some embodiments, haplotypes of structural variants can be determined using HMM analysis of chimeric read alignments and/or unusual relative reads. For example, a pairwise HMM can be used for such a determination. Paired HMM can be implemented using hardware modules as described herein.

[00837] Accordingly, in various embodiments, the somatic variant determination procedure that is explained in FIG. 43E can be performed, for example, to calculate the probability that a variant is a cancer variant. For example, at step 1000, genomic sequence reads of the samples may be generated, for example, by sequencing using SNR, and/or obtained, for example, by transmission via a suitably configured cloud-based network system, for example, from one or both cancer and non-cancerous genetic samples . At step 1010, a genomic reference sequence can be generated, for example, for comparison of reads, at step 1020, an index of the genomic reference sequence can be constructed, and at step 1030, the genomic sequence of the sample can be compared to the index, for example, using software and/or hardware implementations described in herein to map genomic sequence reads to an index at step 1040. Next, at step 1050, the mapped reads may be aligned to a genomic reference sequence to generate an alignment score. At step 1080, the mapped and/or aligned reads can then be sorted by reference position, and optionally, at step 1081, any duplicate reads can be labeled and removed.

[00838] In addition, after the reads are labeled and/or aligned and/or sorted and/or deduplicated, variants can be detected at step 1100, for example, using Bayesian analysis, and at step 1101, variants can be detected generative variants in both non-cancerous and cancerous samples, and optionally somatic variants in these samples. Similarly, at step 1094, the posterior probabilities of each possible cancer variant can be calculated. Next, at step 1096, haplotypes of cancer variants can optionally be determined, for example, by implementing HMM analysis in software and/or hardware, as described herein.

[00839] In addition, the devices, systems and methods described herein may be configured to perform a co-genotyping operation. In particular, the joint genotyping operation can be used to improve the accuracy of variant detection, for example by jointly considering reads from a cohort of multiple subjects. For example, in different cases, genomic variations may be highly correlated in certain populations, such as when certain variants are common among many subjects. In such cases, the sensitivity and specificity of variant detection can be improved by jointly considering supporting evidence for each variant from multiple DNA (or RNA) samples. Namely, sensitivity can be improved in that weak supporting evidence for a variant in one subject can be strengthened by supporting evidence for the same variant in other samples. More specifically, sensitivity can be improved because moderate supporting evidence for a false-positive variant may be weakened by the lack of supporting evidence for the same variant in other samples. Generally speaking, the more samples that participate in co-genotyping, the more accurate the variant identification will be for any given subject.

[00840] Conjoint genotyping involves estimating posterior probabilities for various subsets of all subjects having a given variant using prior probabilities that express observed correlations in genetic variation. In various cases, co-genotyping can be accomplished in a single variant detection run, where aligned reads from multiple samples are examined by a variant detector. This is usually only practical for a small number of samples, since when tens, hundreds, or thousands of samples are involved, the overall data size becomes nearly impossible to quickly access and manipulate.

[00841] In an alternative embodiment, conjoint genotyping can be accomplished by first performing variant determinations separately for each sample, then combining the results using a conjoint genotyping tool that updates variant probabilities for each subject using the combined information. This method uses additional output from each variant determination of a single sample to better modify areas of weak supporting evidence for variants and/or in areas where variants would not have been identified without co-processing. Although the VCF format is typically used to represent specific variants when defining variants of a single sample, the special gVCF format can be used to represent first-stage variant (and non-variant) definitions in preparation for merging. The gVCF format contains records for locations and/or blocks from multiple locations where there is likely to be no variant, so this information can be combined with gVCFs of other determinations or lack of determinations in the same locations to obtain improved joint genotype determinations for each subject.

[00842] Accordingly, a co-genotyping pipeline can be configured to identify variants across multiple samples more quickly and with greater accuracy. In addition, the co-genotyping pipeline may also be configured to support the determination of lineage variants as well as populations from a cohort of samples. For example, the pipeline can be configured to process up to 10, 15, 20, 25, even 50 or more samples simultaneously. In various cases, the configuration for identifying population variants may be configured to simultaneously process samples of many thousands in size. Additionally, the combination of speed and hierarchical clustering of multiple samples provides a computationally efficient solution for co-genotyping analyses. Additionally, sequencing of samples for co-genotyping can be performed in the same flow cell of a next-generation sequencer, allowing the system to simultaneously map/align multiple sample inputs, thereby speeding up the overall co-genotyping process, such as when BCL data can be fed directly into pipeline to generate unique gVCF files for each sample.

[00843] Therefore, this document proposes a method for improving the accuracy of variant detection by jointly reviewing reads from a cohort of multiple subjects. The first step takes genomic sequence reads from two or more samples. A genomic reference sequence is constructed for comparison with reads, and an index of the genomic reference sequence is formed from it. The genomic sequence reads of each sample are then compared to the index and it is determined whether the genomic sequence reads of each sample map to the index.

[00844] The mapped reads can then be aligned to a genomic reference sequence and an alignment score can be generated. These reads can be sorted by mapped reference positions, and duplicate reads can be marked and/or removed. Additionally, overlapping reads from the read pool can then be analyzed to determine whether the majority of the reads are consistent with the reference genomic sequence. Posterior probabilities are calculated for each possible variant, and variant identification data from all samples can be combined to improve the accuracy of variant identification for each individual sample. This improves the accuracy of variant determination (eg, sensitivity and specificity) for each sample and can be done as a processing step after the samples have been subjected to variant determination analysis, or it can be done cumulatively after variant determination for each sample. The likelihood of non-reference alleles in regions where no variants are defined can then be determined and the resulting likelihood of non-reference alleles in regions where variants are not defined can be reported.

[00845] Accordingly, in various embodiments, the somatic variant determination procedure that is explained in FIG. 43F can be performed, for example, to calculate the probability that a variant is a cancer variant. For example, at step 1000, genomic sequence reads of the samples may be generated, for example, by sequencing using SNR, and/or obtained, for example, by transmission via a suitably configured cloud-based network system, for example, from one or both cancer and non-cancerous genetic samples . At step 1010, a genomic reference sequence can be generated, for example, for comparison of reads, at step 1020, an index of the genomic reference sequence can be constructed, and at step 1030, the genomic sequence of the sample can be compared to the index, for example, using software and/or hardware implementations described in herein to map the genomic sequence reads to an index at step 1040. Next, at step 1050, the mapped reads can be aligned to a genomic reference sequence to generate an alignment score. At step 1080, the mapped and/or aligned reads can then be sorted by reference position, and optionally, at step 1081, any duplicate reads can be labeled and removed.

[00846] Similarly, at step 1082, overlapping reads from a pool of reads can be analyzed to determine whether one or more, e.g., a majority, reads match the reference genomic sequences, and at step 1094, posterior probabilities for each possible variant can be calculated. At this point, at step 1096, variant haplotypes can be determined if desired, for example, by performing HMM analysis, and/or at step 1120, variant definition data can optionally be combined, for example, from all samples, to improve the accuracy of variant detection for each separate sample. Next, at step 1122, the likelihood of non-reference alleles can be determined and reported, for example, in regions where no variants have been identified.

[00847] Additionally, as shown in FIG 43, in accordance with one aspect, an online application store is provided to enable users to develop, sell, and use genomic tools that can be embedded in a system and used to analyze genomic data transmitted and entered into the system. In particular, the genomic app store allows customers who wish to develop genetic tests, such as the NICU test, which, once developed, can be uploaded to a system, such as a genetic marketplace, for purchase and use as a platform for the system to all Those who use the newly developed system platform could use the uploaded tests through a web portal. More specifically, the user can go to an application store web portal, find the required test, for example, the NICU test, download it and/or configure the system to implement it, for example, on their downloadable genetic data. Therefore, a “cohort” online marketplace provides a fast and efficient way to deploy new genetic analytical applications that allow identical results to be obtained from any of the platforms that run the downloaded application. More specifically, the Internet marketplace provides a mechanism for anyone working with the system to develop genetic analysis applications that a remote user can download and configure for use according to the workflow models presented.

[00848] In accordance with another aspect, the cohort marketplace described herein allows for secure sharing of data. For example, the transmission and storage of genomic data must be well protected. However, such genetic data is often large and difficult to transmit in a secure and secure manner, for example when the subject's identity is limited. Accordingly, the presented genetic marketplace allows cohort members to share genetic data without the need for subject identification. In such a marketplace, cohort members can share questions and processes to advance their research in a protected and secure environment without compromising the genomic identities of their respective subjects. In addition, the user can enlist the help of other researchers in analyzing their sample sets without revealing the identity of those who own these genomes.

[00849] For example, a user may identify subjects having a particular genotype and/or phenotype, such as stage 3 breast cancer, and/or undergoing therapy with a particular drug. A cohort can be formed to see how these drugs affect the growth of cancer cells at the genetic level. Thus, these characteristics, among others, may form cohort selection criteria that will allow other researchers, for example at a remote location, to perform standard genetic analyzes on genetic data, using uniform analytical procedures, on those subjects available to them who meet the criteria cohorts. Thus, a given researcher does not have to be responsible for the identification and safety of those whose samples are included in the sample set, such as eligible subjects, when justifying their scientific analysis.

[00850] Specifically, researcher A may create a research cohort in the marketplace and determine appropriate selection criteria for subjects, genomic tests to be performed, and parameters for performing the tests. Investigators B and C, located remotely from Investigator A, can then sign up for the cohort, identify and select eligible subjects, and then perform specified tests on their subjects using the uniform procedures described in this manual to help Investigator A quickly achieve your research goals, or to do it more efficiently. This is beneficial because only a portion of the genetic data is shared, the subject's identity is protected, and because the data is analyzed using the same genetic analysis system using the same parameters, the results data will be the same regardless of where or on what machine the tests were performed on. Hence, a cohort marketplace allows users to form and create cohorts by simply publishing selection criteria and execution parameters on a dashboard. It is also possible to publish interest rates and executed payments using a suitably configured program for commercial activities, such as money exchange.

[00851] Anyone who chooses to participate in the cohort can download the criteria and data files and/or use the genetic data they have already generated and/or stored in performing the requested analysis. For example, each cohort member will be required, or have the ability, to create a database of BCL and/or FASTQ files that are stored on their individual servers. These genetic files will be obtained for subjects who meet the selection criteria. Namely, the subject's stored genetic and/or other data may be scanned to determine eligibility for inclusion as part of the cohort selection criteria. Such data may have already been generated for several purposes, but regardless of the reasons for its generation, it can then be selected and subjected to the requested pipeline analyzes and used for inclusion in the cohort.

[00852] Accordingly, in various embodiments, a cohort system may be a forum for connecting researchers to allow them to pool their resources and data, such as genetic sequence data. For example, joining the cohort will allow the first investigator to enter a project requesting analyzes of genetic data that require mining and/or exploration of a number of genomes from different subjects, for example relating to mapping, alignment, variant identification, etc. Therefore, rather than recruiting subjects and selecting sets of samples himself, a cohort initiator can declare a need to perform a given test procedure on sets of samples collected by others, or that will be collected by others, and this collaborative approach to generating sample sets and analyzing them is facilitated by organizing the cohort described in this document. In particular, the cohort initiator may set selections to the cohort, create a configuration file for sharing among potential cohort members, create workflow parameters, for example, in a workflow folder, and can thereby automate the generation and analysis of data, for example, through a workflow management system. The system may also provide the commercial side of the transaction, such as payment processing to compensate cohort participants for their provision of genetic data sets that can be analyzed, for example, for mapping, alignment, variant identification and/or tertiary analysis.

[00853] In various embodiments, cohort structured analysis may be directed to primary processing of, for example, either DNA or RNA, such as image processing and/or base quality score recalibration, methylation analysis, and the like; and/or may be aimed at performing secondary analysis, for example, in relation to mapping, alignment, sorting, identifying variants, etc.; and/or may be directed to tertiary analysis, such as array, genomic, epigenomic, metagenomic, genotyping, variant, and/or other forms of tertiary analysis. Additionally, it should be understood that while many of the pipelines and analyzes performed thereby may involve primary and/or secondary processing, the various analysis platforms described herein may not specifically address primary or secondary processing. For example, in certain cases, an analysis platform may refer solely to performing tertiary analysis, such as on genetic data, or other forms of genomic and/or bioinformatics analyses.

[00854] For example, in specific embodiments, with respect to the particular analytical procedures that will be performed, the analyzes to be performed may include one or more of mapping, alignment, sorting, variant determination, and the like, to generate result data, which may be subject to one or more secondary and/or tertiary analysis procedures depending on the specific pipelines selected for execution. The workflow may be simple or it may be complex, for example it may require the use of a single pipeline module such as mapping or multiple modules such as mapping, alignment, sorting, variant detection and/or others, but the important parameter is that the work flow should be identical for each cohort member. In particular, a unique feature of the system is that the requester creating the cohort specifies control parameters to ensure that the analysis is performed in the same way regardless of where the procedures are performed or on what machines.

[00855] Accordingly, when setting up a cohort, the requester will upload the selection criteria along with the configuration file. Other cohort members will then review the selection criteria to determine whether they have genetic information datasets that fall within the established selection criteria and, if so, perform the requested analysis on those data based on the configuration file settings. Researchers can sign up to be selected as a cohort participant, and if there are large numbers of subscribers, a drawing or competitive selection of participants can be organized. In various cases, an open tender system may be initiated. Outcome data generated by cohort members can be processed on site or in the cloud, and if a configuration file is used, the data processing will be the same. Specifically, the configuration file specifies how the BioIT analytics device is to be configured, and once the device is configured and associated with the system, it will perform the requested genetic analysis in the same manner regardless of where it is located locally or remotely. The results data can then be uploaded to the cohort marketplace and based on the received results data, payment is transferred and accepted.

[00856] For example, the analysis of genetic data can be performed locally and the results uploaded to the cloud, or the genetic data itself can be uploaded and the analysis can be performed on the cloud, for example, on a server or network of servers, for example, on a quantum processing platform associated with the cloud. In various cases, it may be useful to download only the results data to better protect the identities of the subjects. In particular, uploading only the results data is not only secure, it also eliminates the need to transfer large amounts of data, thereby increasing system efficiency.

[00857] More specifically, in various cases a compressed file containing result data from one or more pipelines may be downloaded, and in some cases only the file containing the description of the variations needs to be downloaded. In some cases, you just need to give an answer, such as “yes” or “no.” These responses are preferred because they do not provide identifying information about the subject. However, if the analysis needs to be performed online, such as in the cloud, the selected BCL and/or FASTQ files can be uploaded, the analysis performed, and then the results data can be returned back to the original sender, who can then upload the results data through the cohort interface. The original raw data can then be removed from memory on the network. Due to this and other similar things, the cohort requester will not have access to the identifying data of the subjects.

[00858] Compression, such as using just-in-time (JIT) analysis technology, is particularly useful to improve cohort efficiency. For example, moving data to and from a cohort system using typical procedures is very expensive. Accordingly, while in various configurations the raw and/or uncompressed data uploaded to the system may be stored there, in specific cases the data may be compressed before upload, and the data may then be processed by the system, and the results may then be compressed before transmission from the system, for example, when compression is performed according to the JIT protocol. In this case, storing such data, for example in a compressed form, is cheaper, and therefore the cohort system is very cost-effective.

[00859] In addition, in various cases, multiple cohorts may be represented on a single online marketplace, and with the compression processes described herein, data can be transferred from one cohort to another so that researchers from all sorts of different cohorts can share data between themselves, which would be prohibitively expensive without the compression techniques described herein. In particular, without the speed and efficiency of JIT compression, data once transferred to the cloud will typically remain on the cloud, although still available there for viewing and manipulation. However, JIT allows data to be quickly transferred to and from the cloud for local and/or cloud processing. In addition, as shown in FIG. 41B and 43, in specific cases, the system 1 may be configured to apply further processing to the generated and/or processed data, for example, by a local 100 and/or a remote 300 computing resource, for example, by passing it through one or more pipelines. tertiary processing, such as one or more of a microarray analysis pipeline, a genome analysis pipeline, e.g., a whole genome analysis pipeline, a genotyping analysis pipeline, an exome analysis pipeline, a microbiome analysis pipeline, a genotyping analysis pipeline, including co-genotyping, a variant analysis pipeline, including structural variant pipelines , somatic variant pipelines, and GATK and/or MuTect2 pipelines, as well as RNA sequencing pipelines and/or other tertiary processing pipeline. Data from such processing may then be compressed and/or stored remotely 400 and/or transmitted for storage locally 200.

[00860] In particular, one or more, for example all, of these functions can be performed locally, for example, at the operational location 10, on a local cloud 30, or through controlled access using a hybrid cloud 50. In such a case, a developer environment is created, which allows the user to control the functionality of the system 1 to meet their individual needs and/or to make it available to other users who are looking for the same or similar results. Consequently, the various components, processes, procedures, tools, tiers and hierarchies of the system can be made configurable through a GUI interface that allows the user to select which system components to use, on what data, at what time and in what order according to established user requirements and protocols to form appropriate data and connections between data that can be securely transferred throughout the system, whether locally or remotely. As stated, these components can be configured to seamlessly communicate with each other, e.g., regardless of location and/or type of connection, e.g., through hard-coupled configuration and/or seamless connectivity via the cloud, and/or configurability, for example, through a JIT protocol to perform the same or similar processes in the same or similar way, for example, by using appropriate APIs distributed throughout the system, the use of which allows different users to configure different components to perform different procedures in a similar way.

[00861] For example, an API may be defined in a header file regarding the processes to be performed by each specific component of system 1, where the header describes the functions performed and specifies how to call the functions, such as parameters that are passed, inputs received, and outputs passed. , and the way in which it happens, what comes in and how, what is issued and how, what is returned and how. For example, in various embodiments, one or more components and/or elements thereof, which may form one or more pipelines of one or more tiers of the system, may be configured by instructions input by the user and/or one or more second-tier applications. and/or a third party. These instructions can be passed to the system through appropriate APIs that communicate with one or more different system drivers, indicating to the drivers which parts of the system, such as modules, and/or which processes within them should be activated, when and in what order, taking into account a pre-selected configuration of parameters, which can be defined using an interface that can be selected by the user, for example a GUI,

[00862] In particular, one or more DMA drivers of system 1 may be configured to operate in an appropriate manner, for example, at the kernel level of each component and system 1 as a whole. In such a case, one or more of the provided cores may have its own very low level core API that provides access to the hardware and functions of various components of the system 1 in order to have access to the corresponding registers and modules for configuring and directing processes and how they executed in System 1. Namely, on top of this layer a virtual layer of utility functions can be built to form the building blocks that are used for a variety of functions that send files down to the cores and receive back the results, encode, encrypt and/or transmit the corresponding data and then perform higher-level functions on them. On top of this layer, an additional layer can be built using these utility functions, which can be an API layer that can be interacted with by the user, and this layer can be configured to function primarily to configure the system 1 as a whole or its component parts by downloading files and uploading results, where files and/or results can be transferred throughout the system, either locally or globally. Additional APIs can be configured and enabled, described in more detail above in connection with secure data storage.

[00863] Such configuration of various APIs, memories, and/or system firmware may involve communicating with registers as well as making function calls. For example, as described above herein, one or more function calls necessary and/or useful to perform steps, for example, sequentially, to perform mapping and/or alignment and/or sorting and/or variant determination or other The secondary and/or tertiary processing functions that are described herein may be implemented in accordance with hardware operations and/or associated algorithms to generate the required processes and perform the required steps.

[00864] Namely, because in certain implementations one or more of these operations may be based on one or more structures, it may be necessary to construct various structures necessary to implement these operations. This will require a call to a function that will perform these actions, where the function call will result in the construction of the desired structure to perform the operation, and therefore this call will take the name of the file where the structure parameter files are stored, and will then generate one or more data files that contain and /or configure the desired structure. Another function call may be designed to load a structure that has been generated by the appropriate algorithm and transfer it all the way to on-chip memory and/or system 1 and/or place it in the desired location where the hardware expects it to be. Of course, various other selected functions of System 1 will require various data to be loaded into the chip and/or otherwise transferred to the system generator, and these functions can be performed by a configuration manager, for example by loading whatever is needed to make the pipeline modules in the tiers platforms of the chip and/or the system as a whole performed their functions in memory attached to or otherwise associated with the chip and/or system.

[00865] In addition, the system may be configured to allow various components of the system to communicate with each other, for example, to allow one or more chips of system 1 to communicate with a circuit board of sequencer 121, compute resource 100/300, converter 151 , analyzer 152, interpreter 310, collaborator 320, or other components of the system when included therein, to receive FASTQ and/or other generated and/or processed genetic sequence files directly from the sequencer or other processing components, for example, immediately after they are generated and /or processing, and then pass this information to the configuration manager, which then forwards this information to the appropriate memory banks in the hardware and/or software, which make this information available to the appropriate modules of the hardware, software and/or the system as a whole, so that they can perform their assigned functions on this information for basing, mapping, alignment, sorting, etc. DNA/RNA sample relative to the reference genome and/or performing associated secondary and/or tertiary processing operations on it.

[00866] Accordingly, various implementations may include a client level interface (CLI) that may allow users to directly call one or more of these functions. In various implementations, the CLI may be a software application, for example having a GUI, that is configured to configure the availability and/or use of hardware and/or various other software applications of the system. Therefore, a CLI can be a program that accepts instructions, such as arguments, and makes functionality available by calling an application program. As stated above, the CLI can be command line or GUI (Graphical User Interface) based. The command line level is below the GUI level, with the GUI including a Windows-based file manager with mouse-clickable function blocks that depict which modules, which pipelines, which tiers of which platforms will be used, as well as the parameters for using them. For example, at runtime, if directed, the CLI will find the location of the link, determine whether a hash table and/or index needs to be built, or, if they have already been built, find where they are stored, and direct the unloading of the hash table being built, and /or index, etc. These types of instructions may appear as options on the GUI that the user can select for execution by the associated chips/system 1.

[00867] Additionally, a library may be included that may contain pre-existing editable configuration files, such as files targeting typical user-selectable hardware and/or associated software functionality, such as those related to analysis of a portion or entire genome and/ or proteins, for example, for various analyses, such as the analysis of personal medical histories and pedigrees, or the diagnosis of diseases, or the discovery of new drugs, therapeutics and/or one or more other analytics, etc. Parameters of these types can be preset, for example, to perform such analyses, and can be stored in a library. For example, if the platform described herein is used, for example, for research in the fields of NIPT, NICU, cancer, LDT, AgBio and related research at the collective level, the present parameters may be configured differently than if the platform were aimed at conducting only genomic and /or genealogical research, for example at the individual level.

[00868] More specifically, in the case of a particular diagnosis of an individual, accuracy may be an important factor. Therefore, system parameters can be set to guarantee accuracy, although at the expense of possible speed reduction. However, for other genomics applications, speed may be the main determining factor, and therefore system parameters may be set to maximize speed, but at the expense of accuracy. Accordingly, in various embodiments, frequently used parameter settings for performing various tasks may be preset in the library to facilitate their use. Such parameter settings may also include necessary software applications and/or hardware configurations used in operating system 1. For example, the library may contain code that executes the API and may also include image files, scripts, and any other supporting information needed for the operation of System 1. Therefore, the library can be configured to compile software to execute the API as well as various executable objects.

[00869] Additionally, as shown in FIG. 42C and 43, the system may be configured such that one or more system components can be executed remotely, such as when the system component is configured to perform one or more comparison functions on the data, such as an interpretation function 310 and/or a collaboration function 320 . For example, when an interpretation protocol is applied to data, the interpretation protocol 312 may be configured to analyze and draw conclusions about the data and/or determine various relationships therein, and one or more other analytical protocols may also be performed that include annotating the data 311. performing 313 diagnostics on the data and/or analyzing the data to determine the presence or absence of one or more biomarkers 314. As noted, one or more of these functions may be driven by the WMS and/or performed by the AI module described herein.

[00870] Additionally, when using a collaboration protocol, system 1 may be configured to provide an electronic forum where data can be shared 321, wherein the data sharing protocol may include security 324 and/or privacy settings 322 that can be selected by the user and allow the data to be encrypted and/or password protected so that identifiers and data sources can be hidden from users of the system 1. In specific cases, the system 1 may be configured to allow a 3rd party analyzer 121 to perform virtual simulations on the data. In addition, once generated, data interpreted and/or subjected to one or more collaborative analyzes may be stored either remotely 400 or locally 200 to make it available to remote 300 or local 100 computing resources.

[00871] In accordance with another aspect, as shown in FIG. 44, a method is provided for using a system to generate one or more data files on which one or more secondary and/or tertiary processing protocols can be performed. For example, the method may include providing a genomic infrastructure, for example, for one or more of local, cloud and/or hybrid generation and/or processing and/or analysis in the field of genomics and/or bioinformatics.

[00872] In such a case, the genomic infrastructure may include a bioinformatics processing platform having one or more memories that are configured to store one or more configurable structures for configuring the system to enable one or more analytical processing functions to be performed on data, such as data containing a genomic sequence of interest or processed result data related thereto. The memory may contain a genomic sequence of interest to be processed, for example, after it has been generated and/or obtained, one or more reference genetic sequences and/or may further contain an index of one or more genetic reference sequences and/or a list related thereto splicing boundaries. The system may also include an input device having a platform application program interface (API) for selecting from a list of options one or more configurable processing structures, e.g., by selecting system processing functions to be performed on the data, e.g. or subsequent processing of genomic sequences of interest. It may also be possible to have a graphical user interface (GUI) that can be functionally linked to an API, for example to provide a menu through which the user can select which of the available options to execute on the data.

[00873] Therefore, in these and/or other such cases, the hybrid cloud 50 may be configured to allow data to be transmitted seamlessly and securely to all components of the system, for example, when the hybrid cloud 50 is configured to allow different users of the system to configure its component parts and /or the system itself, for example, using a WMS, to meet the needs of users in the field of research, diagnostic, therapeutic and/or preventative discovery and/or development. In particular, the hybrid cloud 50 and/or the various components of the system 1 can be configured to be operably connected to compatible and/or appropriate APIs that are configured to allow users to remotely configure the various components of the system 1 to deploy the required resources in a desired manner, locally. , remotely or in a combined manner, for example, based on the needs of the system and the characteristics of the analyzes being performed, while ensuring data exchange in a secure environment with the ability to encrypt.

[00874] As described above, the system may be implemented on one or more integrated circuits, which may be formed from one or more sets of configurable, e.g., preconfigured, or hardwired digital logic circuits that may be interconnected with using multiple physical electrical interconnections. In such a case, the integrated circuit may have an input, such as a memory interface, for receiving one or more structure protocols configured, for example, from memory, and may also be configured to implement one or more structures on the integrated circuit in accordance with configurable processing structure protocols. The input memory interface may also be configured to receive genomic sequence data, which may be in the form of multiple genomic data reads. The interface may also be configured to access one or more genetic reference sequences and index(es).

[00875] In various cases, digital logic circuits may be implemented as a set of processing engines, each of which is formed from a subset of digital logic circuits. Digital logic circuits and/or processing engines may be configured to implement one or more preconfigured primary, secondary and/or tertiary processing protocol steps to generate multiple genomic sequence data reads and/or process multiple genomic data reads, for example, in accordance with genetic reference sequences or other information derived from a genetic sequence. The integrated circuit may also have an output to output primary, secondary, and/or tertiary processing data, for example, in accordance with a platform application program interface (API).

[00876] In particular, in various embodiments, digital logic circuits and/or sets of processing engines may form multiple genomic processing pipelines, for example, where each pipeline may have an input that is defined in accordance with the platform application program interface for receiving the resulting bioinformatics processing platform primary and/or secondary processing data and to perform one or more analytical processes on them to create resulting data. In addition, multiple genomic processing pipelines may have a common pipeline API that defines a secondary and/or tertiary processing operation to be performed on the resulting primary and/or secondary processing data, for example, where each of the plurality of genomic processing pipelines is configured to perform a subset of secondary and/or tertiary processing operations and output of the resulting secondary and/or tertiary processing data in accordance with the pipeline API.

[00877] In such cases, a plurality of genomic analysis applications may be stored in memory and/or an associated searchable application repository, such as where each of the plurality of genomic analysis applications may be accessible to a computer via an electronic medium, such as for execution by a computer processor , for the purpose of carrying out the intended analysis of genomic data from pre- or post-processing of the resulting primary, secondary and/or tertiary processing data, for example, by one or more of a plurality of genomic processing pipelines. In specific cases, each of a plurality of genomic analysis applications may be defined by an API and may be configured to receive primary, secondary, and/or tertiary processing resultant data and/or perform intended analysis of preprocessing or postprocessing genomic data and output intended analysis resultant data to one or more genomic databases.

[00878] The method may further include selecting, for example, from a GUI menu, one or more genomic processing pipelines from a plurality of available genomic processing pipelines of the system; selecting one or more genomic analysis applications from a plurality of genomic analysis applications stored in the application repository; and executing, by a computer processor, one or more selected genomic analysis applications to perform the intended analysis of the genomic data from the resulting primary, secondary, and/or tertiary processing data.

[00879] Additionally, in various embodiments, all of mapping, alignment, sorting, and variant determination may occur on-chip, and in various embodiments, local realignment, duplicate flagging, base quality assessment recalibration, and/or one or more protocols and/ or tertiary processing pipelines may also be executed on-chip or in software, and in various cases, various compression protocols such as SAM and/or BAM and/or CRAM may also be executed on-chip. However, after the data from primary, secondary and/or tertiary processing has been created, it may be compressed, for example, before transmission, such as sending across the system, sending to the cloud, for example, to run a variant determination module, a secondary platform, tertiary or other processing, for example, including analysis protocol interpretation and/or collaboration. This can be useful, especially given the fact that identifying variants, including their tertiary processing, can be “shooting at a moving target”, for example there is no standardized, agreed upon algorithm used in the industry.

[00880] Therefore, different algorithms may be used to achieve different types of results, for example by a remote user, and therefore it is useful to have a cloud module to perform this function to provide flexibility in choosing the algorithm that is useful at that particular moment, as well as serial or parallel processing. Accordingly, any of the modules described herein may be implemented either in hardware, such as on a chip, or in software, such as in the cloud, but in certain embodiments, all modules may be configured to perform their functions only on a chip, or all modules may be configured to perform their functions remotely, for example on the cloud, or there may be a mixture of modules where some are located on one or more chips and others are located on the cloud. Additionally, as noted, in various embodiments, the chips themselves may be configured to operate in conjunction with, and in some embodiments, directly interact with, a genetic sequencer such as an SNP and/or an on-chip sequencer.

[00881] More specifically, in various embodiments, a device of the present invention may be a chip, such as a chip that is configured to process genomic data, for example, through the use of a pipeline of data analysis modules. Accordingly, as shown in FIG. 45, a genome pipeline processor chip 100 is proposed along with associated hardware of a genome pipeline processor system 10. The chip 100 has one or more connections 102 to external memory (“DDR3 Memory Controller”) and a connection 104 (e.g., a PCIe or QPI interface) to an external world, such as, for example, host computer 1000. Switch 108 (eg, a switch) provides access to memory interfaces to various requesters. The DMA engines 110 transfer data at high speed between the host and the external memories 102) of the chip processor 100 (via the switch 108) and/or between the host and the central controller 112. The central controller 112 manages the operations of the chip, in particular, coordinating the actions of multiple engines. 13 treatments. Processing engines are formed from a set of hard-wired digital logic circuits that are interconnected by physical electrical connections and organized into 11/114 engine clusters. In some implementations, engines 13 in one cluster 11/114 share one switch port through arbiter 115. Central controller 112 has connections to each engine cluster. Each engine cluster 11/114 has a number of processing engines 13 for processing genomic data, including a mapper 120 (or mapping module), an aligner 122 (or an alignment module), and a sorter 124 (or a sorting module), and one or more engines may be provided processing to perform other functions such as variant determination. Therefore, the engine cluster 11/114 may also contain other engines, such as a variant determiner module.

[00882] In accordance with the data flow model consistent with the implementations described herein, the main CPU 1000 sends commands and data through the DMA engines 110 to the central controller 112, which evenly distributes the data among the processing engines 13. The processing engines return the processed data to the central controller 112, which streams it back to the host device via the DMA engines 110. This data flow model is suitable for mapping, and alignment and identification of variants. As stated, in various cases, communication with the host CPU can be accomplished through relatively loose or tight coupling, such as using a low-latency, wideband interconnect such as QPI, to maintain cache coherence between coupled memory elements of two or more devices.

[00883] For example, in various cases due to various power and/or space constraints, for example, when performing big data analysis such as mapping/alignment/variant identification in a hybrid software/hardware accelerated environment as described herein, where there is a need to move data quickly and seamlessly between system devices, to perform such transfers back and forth across the system between related devices, for example, back and forth to a sequencer, digital signal processor (DSP), CPU and/or GPU, or hybrid CPU/GPU, Accelerated integrated circuit such as FPGA, ASIC (on a network interface card), and other smart network accelerators, fast and while maintaining cache coherence can benefit from a tight coupling interface while maintaining cache coherence. In such cases, a suitable cache coherence-maintaining hard-coupled interconnect may be an interconnect of one or more single-interconnect technologies configured to support processing, for example, between multiple processing platforms using different instruction set architectures (ISAs), while coherently sharing data between different platforms and/or with one or more associated accelerators, such as, for example, a hard-wired FPGA accelerator, to enable efficient heterogeneous computing and thereby significantly improve the computing efficiency of the system, which in various cases can be implemented as a cloud server system. Therefore, in certain cases, a low latency, cache coherency wideband interconnect protocol such as QPI, Coherent Processor Accelerator Interface (CAPI), NVLink/GPU, or any other suitable interconnect protocol may be used to speed up various data transfers between different system components. for example, related to the computational functions of mapping, alignment and/or determination of options, which may include the use of acceleration engines, the operation of which requires accessing, processing and seamless movement of data between various components of the system, regardless of the location of the various data to be processed in system. And, when such data is stored in an associated storage device such as RAM or DRAM, data transfer activities may also include accelerated and coherent search and processing of the in-memory database.

[00884] In particular, in particular embodiments, such heterogeneous computing may include multiple processing and/or acceleration architectures that may be interconnected in a reduced instruction set computing format. In such a case, this interconnect device may be a coherent interconnect-6 (CCVI) device, which is configured in a manner that allows all computing components within the system to apply addressing, reading, and/or writing operations to one or more associated memories in a single consistent and coherent manner. More specifically, a CCVI interconnect can be used to interconnect various system devices, such as CPU and/or GPU or hybrid CPU/GPU, FPGA, and/or associated memories, etc. with each other, for example, with a wide bandwidth, and is designed to increase transmission speeds between different components, as evidenced by a sharp reduction in latency levels. Namely, the CCVI interconnect can be used and configured so that all system components access and process data regardless of where the data resides, and without the need for complex programming environments that would otherwise have to be implemented to ensure data coherence . Other such interconnects that can be used to speed up, e.g., reduce processing time, and improve processing accuracy include QPI, CAPI, NVLink, or other interconnects that can be configured to interconnect different system components and/or operate on top of a related PCI-express peripheral interconnect.

[00885] Therefore, according to an alternative data flow model consistent with the implementations described herein, the main CPU 1000 streams data to external memory 1014 either directly through the DMA engines 110 and switch 108, or through the central controller 112. Main The CPU 1000 sends commands to the central controller 112, which sends commands to the processing engines 13 instructing the processing engines what data to process. Because of the tight coupling, the processing engines 13 input data directly from the external memory 1014 or its associated cache, process it, and write the results back to the external memory 1014, for example, through the tightly coupled interconnect 3, reporting the status to the central controller 112. The central controller 112 either sends the resulting data is streamed back to said host device 1000 from external memory 1014, or notifies the host device to itself retrieve the result data via the DMA engines 110.

[00886] In FIG. 46 shows the genomic processing pipeline processor and system 20 showing a complete set of processing engines 13 within an engine cluster 11/214. Pipeline processor system 20 may comprise one or more engine clusters 11/214. In some implementations, pipelined processor system 20 may contain four or more engine clusters 11/214. The processing engines 13 or types of processing engines may include, but are not limited to, a matcher, an aligner, a sorter, a local re-aligner, a base quality recalibrator, a duplicate flagger, a variant detector, a compressor and/or decompressor. In some implementations, each 11/214 engine cluster has one of each type of processing engine. Accordingly, all processing engines 13 of the same type can access the switch 208 simultaneously through different ports of the switch, so the clusters 11/214 in which each of them is located are different. The formation of a processing engine of each type in each 11/214 cluster is not required. Types of processing engines that require enormous parallel processing or memory bandwidth, such as a matcher (with aligners attached to it) and a sorter, may end up in every engine cluster of a pipelined processor system 20. Other types of engines may appear in only one or a few clusters 214 engines as necessary to meet the requirements for their performance or the performance of the pipeline processor system 20.

[00887] In FIG. 47 illustrates a genome pipeline processor system 30 showing, in addition to the engine clusters 11 described above, one or more embedded central processing units (CPUs) 302. Such examples of embedded CPUs include Snapdragon® cores or standard ARM® cores, or in other cases it may be FPGA. These CPUs execute fully programmable bioinformatics algorithms such as advanced variant detection such as DBG construction or HMM execution. Such processing is accelerated by computational functions in various clusters of 11 engines, which can be called upon by the 302 CPU cores as needed. Additionally, even engine-centric processing such as mapping or alignment can be performed by the CPU cores 302, providing them with increased programmability.

[00888] In FIG. 48 shows a processing flow for a pipelined processor system and a method for a genome. In some preferred implementations, the data is processed in three passes. The first pass includes mapping 402 and alignment 404, with the full set of reads being run through the engines 13. The second pass involves sorting 406, where one large block to be sorted (eg, a substantial portion of all reads previously mapped to one chromosome) is loaded into memory, sorted by the processing engines, and returned to the main device. The third pass includes the next steps in the chain (local realignment 408, duplicate labeling 410, base quality score recalibration (BQSR) 412, SAM inference 414, reduced BAM inference 416, and/or CRAM compression 418). The steps and functions of the third pass can be performed in any combination or subcombination and in any order in a single pass.

[00889] Therefore, in this manner, data flows relatively unimpeded from one or more processing engines to the main CPU, for example, in accordance with one or more of the methods described herein. Thus, a virtual pipeline architecture, such as the one described above, is used to stream reads from the host to circular buffers in memory, through one processing engine after another, sequentially, and back to the host. In some implementations, CRAM decompression may be a separate threading function. In some implementations, SAM output 414, reduced BAM output 416, and/or CRAM compression 418 may be replaced by variant detection, compression, and decompression.

[00890] A hardware implementation of a sequence analysis pipeline is described in various cases. This can be done in many different ways, such as an FPGA, ASIC, or structured ASIC implementation. Functional blocks that are implemented using an FPGA, ASIC, or structured ASIC are shown in FIG. 49. Accordingly, the system includes a number of blocks or modules for performing sequence analysis. The input to the hardware implementation may be a FASTQ file, but this format is not a limitation. In addition to the FASTQ file, the FPGA or ASIC or structured ASIC input data contains auxiliary information, such as Flow Space Information, related to the technology, such as the SNP. The blocks or modules may include the following: error correction, mapping, alignment, sorting, local realignment, duplicate labeling, base quality assessment recalibration, BAM and side information reduction, and/or variant identification.

[00891] These blocks or modules may be present internally, or may be implemented in hardware, but to achieve the purpose of implementing a sequence analysis pipeline, some of these blocks may be omitted and other blocks may be added. Blocks 2 and 3 describe two alternative platform options for a sequence analysis pipeline. The sequence analysis pipeline platform contains an FPGA, ASIC, or structured ASIC, and software supported by a host device (e.g., PC, server, cluster, or cloud computing facilities) using cloud and/or cluster storage. Blocks 4-7 describe the various interfaces that a sequence analysis pipeline may have. In blocks 4 and 6, the interface may be a PCIe and/or QPI/CAPI/CCVI/NVLink interface, but is not limited to PCIe, QPI or other interface. In blocks 5 and 7, the hardware (FPGA, or ASIC, or structured ASIC) can be directly integrated into the sequencer. Blocks 8 and 9 describe the integration of a hardware sequence analysis pipeline built into a host system such as a PC, server cluster, or sequencer. Surrounding the FPGA, ASIC, or structured ASIC hardware are multiple DDR3 memory elements and a PCIe/QPI/CAPI/CCVI/NVLink interface. The FPGA/ASIC/sASIC board is connected to a host computer consisting of a main CPU and/or GPU, which may be a low-power CPU such as an ARM®, Snapdragon®, or any other processor. Block 10 illustrates a sequence analysis hardware pipeline API that can be accessed by third party applications to perform tertiary analysis.

[00892] In FIG. 50A and 50B illustrate an expansion board 104 having a chip 100, such as an FPGA, in accordance with the present invention, as well as one or more corresponding elements 105 for coupling the FPGA 100 to a host CPU/GPU, such as for transferring data, such as data to be processing, and the resulting data, back and forth from the CPU/GPU to the FPGA 100. In FIG. 50B shows the expansion board shown in FIG. 50A, which has a plurality of, for example 3, slots containing a plurality of, for example, 3, processing chips according to the present invention.

[00893] Specifically, as shown in FIG. 50A and 50B, in various embodiments, a device of the present invention may comprise a computing architecture, such as embedded in a silicon field programmable gate array (FPGA) or application specific integrated circuit (ASIC) 100. The FPGA 100 may be integrated on a printed circuit board (PCB) 104 , such as a Peripheral Component Interface Express (PCIe) card that can be inserted into a computing platform. In various cases, as shown in FIG. 50A, PCIe card 104 may include a single FPGA 100, which may be surrounded by local memories 105, but in different implementations, as shown in FIG. 50B, PCIe card 104 may include a plurality of FPGAs 100A, 100B, and 100C. In various cases, a PCI card may also contain a PCIe bus. The 104 PCIe card can be added to a computing platform to run algorithms on extremely large data sets. In an alternative embodiment, which is mentioned above with reference to FIG. 34, in various embodiments, the FPGA may be configured to be directly connected to the CPU/GPU, for example via a jumper, and hard coupled to it, for example via a QPI, CAPI, CCVI interface. Accordingly, in various cases, the entire genome sequencing workflow involving the FPGA may include the following: sample preparation, alignment (including mapping and alignment), variant analysis, biological interpretation, and/or custom applications.

[00894] Therefore, in various embodiments, the device of the present invention may comprise a computing architecture that achieves highly efficient execution of algorithms, such as mapping and alignment algorithms, that operate on extremely large data sets, for example, when the data sets exhibit poor locality of reference (LOR) ). These algorithms are designed to reconstruct the entire genome from millions of short read sequences from modern so-called next-generation sequencers, requiring multi-gigabyte random access data structures. Once reconstruction is achieved, as described above in this document, further algorithms with similar characteristics are used to compare one genome with libraries of others, perform functional analysis of genes, etc.

[00895] There are two other typical architectures that typically can be built to perform one or more of the operations described in detail herein, including, for example, general purpose multi-core CPUs and general purpose graphics processing units (GPPUs). In this case, each CPU/GPU in a multi-core system can have a classic cache-based architecture, where instructions and data are fetched from the Level 1 cache (L1 cache), which is small but extremely fast to access. Multiple L1 caches can be connected to a larger but slower L2 cache. The L2 cache can be coupled to a larger but slower DRAM (Dynamic Random Access Memory) memory system, or it can be coupled to an even larger but slower L3 cache, which can then be coupled to DRAM. The benefit of this arrangement may be that applications in which programs and data exhibit reference locality behave almost as if they were running on a computer with a single storage device as large as DRAM but as fast as an L1 cache. Since fully custom, highly optimized CPUs operate at very high clock speeds, such as 2 to 4 GHz, this architecture can be essential to achieving good performance. Additionally, as discussed in detail with reference to FIG. 33, in various embodiments, the CPU may be hard coupled to an FPGA, e.g., an FPGA configured to perform one or more functions related to the various operations described herein, e.g., through QPI, CCVI, CAPI, to further improve performance characteristics, and the speed and coherence of data transferred throughout the system. In such a case, cache coherence can be maintained between the two devices, as noted above.

[00896] In addition, this architecture can be extended using GPUON, for example, by implementing a very large number of small CPUs, each with its own L1 cache, with each CPU executing the same instructions on different subsets of data. This is the so-called OKMD architecture (one command stream, many data streams). Savings can be achieved by sharing instruction fetching and decoding logic across a large number of CPUs. Each cache has access to many large external DRAMs through a network of interconnects. Assuming that the computation must be highly parallelizable, GPUs have a significant advantage over general-purpose CPUs due to the large amount of computational resources they have. However, they still have a cache-based architecture and their performance is degraded by applications that do not have a high enough degree of reference locality. This results in high cache miss rates and processor idle while waiting for data to arrive from external DRAM.

[00897] For example, dynamic RAM may be used as system memory in various cases because it is more cost effective than static random access memory (SRAM). Approximately, we can assume that at the same cost, the volume of DRAM is 4 times greater than the volume of RAM. However, with demand for SRAM falling in favor of DRAM, the gap between the two has widened due to the significant space savings offered by DRAM, which is in high demand. Regardless of cost, DRAM is 4 times denser than SRAM placed on the same silicon area, as it requires only one transistor and one capacitance per bit compared to 4 transistors per bit to implement a flip-flop in SRAM. DRAM represents one bit of information as the presence or absence of charge in the capacitance.

[00898] The problem with this structure is that the charge weakens over time, so it must be renewed periodically. This need has led to architectures that organize memory into independent blocks and access mechanisms that produce multiple words of memory for each request. This compensates for the time when a given block is unavailable during an update. The idea is to move a huge amount of data while a given block is available. This is in contrast to RAM, in which any memory location is accessible in one access for a constant amount of time. This characteristic allows you to focus on one word, rather than a block, when accessing memory. DRAMs work well in a cached architecture because each cache miss results in a block of memory being read from the DRAM. The theory of locality of references is that immediately after a reference to the word N, there will likely be a reference to the words N + 1, N + 2, N + 3, etc.

[00899] In FIG. 51 illustrates an example implementation of a system 500 of the present invention comprising one or more expansion cards depicted in FIG. 50, for example, for bioinformatics processing 10. The system includes a bioinformatics processing chip 100 configured to perform one or more functions in the processing pipeline, such as base detection, error correction, mapping, alignment, sorting, assembly, variant detection, etc. .p., as described in this document.

[00900] System 500 also includes a configuration manager that is configured to be configured on board by one or more processors 100. Specifically, in various embodiments, the configuration manager is configured to communicate instructions to an internal FPGA controller, such as firmware, such as through appropriate an image configured driver over a loosely or tightly coupled interconnect to configure one or more processing functions of the system 500. For example, the configuration manager may be configured to configure internal clusters 11 and/or associated processing engines 13 to perform one or more desired operations, such as mapping, alignment, sorting, identifying options, etc. in accordance with accepted instructions. Thus, only the clusters 11 containing processing engines 13 for performing the requested processing operations on the data provided from the host system 1000 to the chip 100 can be engaged to process that data in accordance with the received instructions.

[00901] Additionally, in various embodiments, the configuration manager may also be configured to be customized by a third party user, for example, through an API connection as described in more detail herein, for example, through a user interface ( GUI) represented by an application of the system 500. In addition, the configuration manager may be connected to one or more external memories, such as a memory that forms or otherwise contains a database, for example, a database containing one or more references, or separately sequenced genomes, and/or an index thereof, and/or one or more previously mapped, aligned and/or sorted genomes or parts thereof. In various cases, the database may also contain one or more genetic profiles characterizing a disease state, for example, to perform one or more tertiary processing protocols, for example, on the newly mapped, aligned genetic sequences or associated VCF file.

[00902] System 500 may include web access to enable remote data exchange, such as over the Internet, to form a data exchange platform via the cloud or at least a hybrid cloud 504. Similarly processed information generated by a bioinformation processor, e.g. the results data may be encrypted and stored in an electronic health record, for example, in an external database, such as a remote database. In various cases, an EMR database may be configured to search, for example, genetic information stored therein, to perform one or more statistical analyzes on the data, for example, to identify disease states or trends, or to analyze the effectiveness of one or more preventive or therapeutic measures. Such information, along with the EMR data, can then be further processed and/or stored in yet another database 508 in a manner that ensures the confidentiality of the source of the genetic information.

[00903] More specifically, in FIG. 51 shows a system 500 for running a sequence analysis pipeline on genetic sequence data. System 500 includes a configuration manager 502, which includes a computing system. The configuration manager computing system 502 may comprise a personal computer or other computer workstation, or may be implemented by a collection of networked computers. Configuration manager 502 may also include one or more third-party applications connected to the computing system through one or more APIs that, together with one or more proprietary applications, generate a configuration for processing genome data from a sequencer or other genome data source. The configuration manager 502 also includes drivers that load the configuration into the genome pipeline processor system 10. Pipeline genome processing system 10 may output the resulting data to or be accessible through the Internet 504 or other network to store the resulting data in an electronic health record 506 or other knowledge database 508 .

[00904] As noted several times earlier herein, a genome pipeline processor chip may be coupled or integrated with a sequencer. The chip may also be coupled or integrated, for example, directly via a jumper, or indirectly, for example, on an expansion card such as a PCIe card, and the expansion card may be coupled or integrated with the sequencer. In other implementations, the chip may be coupled or integrated with a server computer that is coupled to the sequencer to transmit genomic reads from the sequencer to the server. In still other implementations, the chip may be coupled or integrated with a server in a cloud computing cluster of computers and servers. The system may include one or more sequencers connected (eg, via Ethernet) to a server containing the chip, with genome reads generated by the plurality of sequencers, transmitted to the server, and then mapped and aligned on the chip.

[00905] For example, typically in next-generation DNA sequencer data pipelines, the processing of the primary analysis step is typically specific to a given sequencing technology. This primary analysis step is performed to translate the physical signals detected within the sequencer into nucleotide sequence “reads” along with associated quality scores, such as FASTQ files or other formats containing sequence and typically quality information. Primary analysis, as mentioned above, is often quite specific in nature to the sequencing technology used. In various sequencers, nucleotides are detected by measuring changes in fluorescence and/or electrical charges, electrical currents, or emitted light. Some primary analysis pipelines often include: signal processing to amplify, filter, separate, and measure the sensor output; reducing the volume of data, for example, by dividing into subgroups, thinning, averaging, transformation, etc.; image processing or digital processing to identify and amplify significant signals and associate them with specific reads and nucleotides (eg, image bias calculation, cluster identification); algorithmic processing and heuristic procedure to compensate for sequencing technology artifacts (eg, phasing estimation, crosstalk matrices); Bayesian probability calculations; hidden Markov models; base determination (selection of the most likely nucleotide at each position in the sequence); assessment of the quality (reliability) of determining the grounds, etc. As described herein above, one or more of these steps would benefit from having one or more of the required processing functions implemented in hardware, eg, implemented using an integrated circuit such as an FPGA. In addition, once such a format is obtained, it proceeds to secondary analysis, as described herein, to determine the contents of the sequenced DNA (or RNA, etc.) sample, for example, by mapping and alignment of reads to a reference genome, sorting, labeling of duplicates , recalibration of ground quality assessment, local realignment and variant identification. Tertiary analysis may then follow to extract medical and research conclusions from specific DNA contents.

[00906] Accordingly, given the sequential nature of the processing functions described above, it would be desirable to combine primary, secondary and/or tertiary processing acceleration in a single integrated circuit and multiple integrated circuits located on a single expansion board. This can be beneficial because sequencers produce data that typically requires both primary and secondary analysis to be useful and can be further used in various tertiary processing protocols, and combining them into a single device is most efficient in terms of in terms of cost, space, power and resource sharing. Therefore, in accordance with one specific aspect, the invention relates to a system, for example a system, for running a sequence analysis pipeline on genetic sequence data. In various cases, the system may include an electronic data source, such as a data source that provides digital signals, for example, digital signals representing a plurality of genomic data reads, wherein each of the plurality of genomic data reads contains a nucleotide sequence. The system may include one or more memories, such as a memory storing one or more genetic reference sequences and/or an index of one or more genetic reference sequences; and/or the system may include an integrated circuit such as an ASIC, FPGA, or sASIC.

[00907] One or more aspects or features of the subject matter described herein may be implemented in a digital electronic circuit, integrated circuit, application-specific integrated circuit (ASIC), field programmable gate array (FPGA), or structured ASIC, computer hardware software, firmware, software and/or combinations thereof.

[00908] These various aspects or features may include implementation in one or more computer programs that are configured to be executed or interpreted on a programmable system including at least a programmable processor, which may be special purpose or general purpose, and associated with a storage system for receiving data and instructions therefrom and transmitting thereto data and instructions, at least one input device and at least one output device. A programmable system or computing system may include clients or servers. The client and server are usually remote from each other and typically communicate via a communications network. The client-server relationship occurs due to computer programs running on the respective computers and having a client-server relationship with each other.

[00909] These computer programs, which may also be called programs, software, software applications, applications, components, or code, contain machine instructions for a programmable processor and may be implemented in a procedural or object-oriented high-level programming language and/or assembly language /machine language. As used herein, the term “computer-readable storage medium” refers to any computer program product, instrument and/or device, such as, for example, magnetic disks, optical disks, memory, and programmable logic units (PLUs) used to provide machine instructions. and/or data into a programmable processor, including a computer-readable storage medium, that receives machine instructions as a computer-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor. A computer-readable storage medium may store such machine instructions non-transiently, such as a non-transitory solid-state memory or hard disk drive or any other equivalent storage media. A computer-readable storage medium may alternatively or additionally store such machine instructions transiently, such as a processor cache or other random access memory associated with one or more physical processor cores.

[00910] In addition, with the enormous growth in data creation and acquisition in the 21st century, there is a need for increased processing power that is able to handle the ever-increasing computational intensity analyzes on which modern development is based. Supercomputers appeared, which helped the progressive development of technology on a number of platforms. However, while supercomputing is useful, it has proven insufficient for some of the very complex computing problems tech companies face today. In particular, since the sequencing of the human genome, there has been an exponential development of technology in biological fields. However, as the complexity of the raw data generated daily grows at a rapid pace, a bottleneck has developed in the processing and analysis of the generated data. Therefore, quantum computers were developed to eliminate this bottleneck. Quantum computing represents the cutting edge of computing, providing an entirely new approach to addressing the world's most complex computing needs.

[00911] Quantum computing has been known since 1982. For example, Richard Feynman in the International Journal of Theoretical Physics theoretically described a system for performing quantum computing. Namely, Richard Feynman proposed a quantum system that could be configured for use in simulating other quantum systems so that traditional computer processing functions could be performed more quickly and efficiently. See Feynman, 1982, International Journal of Theoretical Physics 21, pp. 467-488, which is incorporated herein by reference in its entirety. In particular, a quantum computer system can be designed to provide exponential time savings on complex calculations. Such controllable quantum systems are commonly known as quantum computers and have been successfully developed into general-purpose processing computers that can be used not only to simulate quantum systems but also be made capable of executing specialized quantum algorithms. More specifically, complex problems can be modeled as an equation, such as a Hamiltonian function, which can be represented in a quantum system such that the behavior of the system provides information about the solution to the equation. See Deutsch, 1985, Proceedings of the Royal Society of London A 400, pp. 97-117, which is incorporated herein by reference in its entirety. In such cases, the solution of the model due to the behavior of the quantum system can be performed in such a way that it includes the solution of a differential equation related to the quantum mechanical description of the particle, for example, the Hamiltonian of the quantum system.

[00912] Essentially, quantum computing is a computing system that uses quantum mechanical phenomena, such as superposition and/or entanglement, to perform various calculations on large amounts of data at extreme speed. Therefore, quantum computers are a huge improvement over traditional digital logic computers. In particular, traditional digital logic circuits operate by using binary digital logic gates, which are formed by rigidly mounting an electronic circuit onto a conductive substrate. In a digital logic circuit, the on/off state of a transistor serves as a basic unit of information, such as a bit. In particular, a conventional digital computer processor uses binary digits, such as bits, in an on or off state, such as 0 or 1, to encode data. On the other hand, quantum computing uses an information device that uses superpositions of entangled states called quantum bits or qubits.

[00913] The basis for performing such quantum computing is the information device that generates the quantum bit. A qubit is analogous to the digital “bit” in traditional digital computers, except that the computing power of a qubit is much greater than that of a digital bit. In particular, as described in more detail herein, instead of encoding only two discrete states, such as "0" or "1", as in the case of a digital bit, a qubit can also be brought into a superposition state of "0" or "1". Namely, a qubit can exist in both states, “0” or “1”, simultaneously. Therefore, the qubit allows quantum computing to be performed on both states simultaneously. In general, N qubits can be in a superposition of 2 ^N states. Therefore, quantum algorithms can use this property of superposition to speed up certain calculations.

[00914] Therefore, a qubit is analogous to a bit in a traditional digital computer, and is a type of information device that exhibits coherence. In particular, a quantum computing device is constructed from many building blocks, such as qubits, information devices. For example, the computing power of a quantum computer increases as the information devices that form its building blocks are linked, such as entangled, together in a controlled manner. In such a case, the quantum state of one information device affects the quantum state of each of the other information devices with which it is connected.

[00915] Accordingly, like a bit in classical digital computing, a qubit in quantum computing serves as a basic unit for encoding information, such as quantum information. Like a bit, a qubit encodes data in a two-state system, which in this case is a quantum mechanical system. In particular, for a qubit, two quantum states imply entanglement, for example involving the polarization of a single photon. Therefore, while in a classical system a bit must be in one state or the other, in a quantum computing platform a qubit can be a superposition of both states at the same time, and this property is fundamental to quantum processing. Therefore, the distinguishing feature between a qubit and a classical bit is that many qubits exhibit quantum entanglement. This entanglement is a nonlocal property that allows a set of qubits to express higher correlation than is possible in a classical system.

[00916] In order for such information devices, such as quantum bits, to work, they must satisfy several requirements. Firstly, the information device must be designed with the ability to reduce it to a quantum two-level system. This means that the information device must have two distinguishable quantum states that can be used to perform calculations. Second, information devices must be able to create quantum effects like entanglement and superposition. In addition, in certain cases, the information device may be configured to store information, such as quantum information, for example in a coherent form. In such cases, the coherent device may have a quantum state that persists without significant degradation over a long period of time, such as on the order of microseconds or more.

[00917] Specifically, quantum entanglement is a physical phenomenon that occurs when a pair or group of particles is formed or otherwise configured to interact in such a way that the quantum state of one particle cannot be described independently of the others, despite the space separating them. Therefore, instead of describing the state of one particle separately from the others, the quantum state must be described for the system as a whole. In such cases, measurements of different physical properties such as position, momentum, spin and/or polarization made on entangled particles are correlated. For example, if a pair of particles is formed such that their total spin is known to be zero, and one particle is found to have a clockwise spin on a particular axis, then the spin of the other particle measured on that same axis will be found to have a counterclockwise spin. , as would be expected given their intricacy.

[00918] Consequently, one particle of an entangled pair simply "knows" what measurement was made on the other and with what result, even if the means by which such information was to be transmitted between particles that at the time of the measurement may have been removed are not known from each other at arbitrarily large distances. Because of this interconnection, unlike classical bits that can only have one value at a time, entanglement allows multiple states to operate simultaneously. It is these unique relationships and quantum states that were used in the development of quantum computing.

[00919] Accordingly, there are various types of physical operations using pure qubit states that can be performed. For example, a quantum logic gate may be generated and configured to operate on a qubit basis, where the qubit undergoes a unitary transformation, such as where the unitary transformations correspond to rotations or other quantum phenomena of the qubit. Indeed, any two-level system can be used as a qubit, for example, photons, electrons, nuclear spins, coherent radiation states, optical lattices, Josephson junctions, quantum dots, etc. In particular, a quantum gate is the basis for a quantum circuit that operates on a small number of qubits. For example, a quantum circuit contains quantum gates that operate on a fixed number of qubits, such as two, three, or more. Therefore, qubits are the building blocks of quantum circuits, similar to classical logic gates for traditional digital circuits. Namely, a quantum circuit is a model for quantum computation, where the computation is a sequence of quantum gates that are reversible transformations on the quantum mechanical analogue of an n-bit register. Such similar structures are called n-qubit registers. However, unlike classical logic gates, quantum logic gates are always reversible.

[00920] In particular, as described above, a digital logic gate is a physical, wiring device that can be implemented using one or more diodes or transistors that act as electronic switches to perform logical operations, such as Boolean functions, on one or more binary input signals to create one binary output signal. With the help of amplification, logic gates can be cascaded in exactly the same way as Boolean functions, which makes it possible to build a physical model of all Boolean logic and, therefore, all algorithms and mathematical relationships that can be described using Boolean logic can be performed using digital logic gates. valves In a similar way, a cascade of quantum logic gates can be formed to perform Boolean logic operations.

[00921] Quantum gates are typically represented as matrices. In various implementations, the quantum gate operates on k qubits, which can be represented by a 2k × 2k unitary matrix. In such cases, the number of qubits at the input and output of the gate must be the same, and the action of the gate on a particular quantum state is found by multiplying the vector representing the state by the matrix representing the gate. Therefore, with this configuration, quantum computing operations can be performed on a very small number of quantum bits. For example, there are quantum algorithms that are designed to perform significantly more complex calculations faster than any possible probabilistic classical algorithm. Specifically, a quantum algorithm is an algorithm that runs on a model of a quantum computation circuit.

[00922] While a classical algorithm is a finite sequence of step-by-step instructions or procedures that can be executed by the digital logic circuits of a classical computer, a quantum algorithm is a step-by-step procedure where each step can be performed on a quantum computer. However, despite the existence of quantum algorithms, such as Shor's, Grover's, and Simon's algorithms, all classical algorithms can also be executed on a quantum computer with the proper configurations. Quantum algorithms are typically used as algorithms that are quantum in nature, such as those that use superposition or quantum entanglement. Quantum algorithms can be formulated in various models of quantum computing, such as the Hamiltonian oracle model.

[00923] Accordingly, while a classical computer has a memory constructed of bits, where each bit is represented by either a "1" or a "0", a quantum computer maintains a sequence of qubits, where one qubit can represent a one, a zero, or a quantum superposition of these two states of the qubit. Accordingly, a pair of qubits can be any quantum superposition of 4 states, and three qubits can be any superposition of 8 states. In general, a quantum computer with n qubits can be in an arbitrary superposition of ²ⁿ different states at the same time, compared to a conventional computer which can only be in one of ²ⁿ states at a time. Consequently, qubits can contain exponentially more information than their classical counterparts. During operation, a quantum computer performs operations by setting qubits to a drift state, which solves the problem by manipulating these qubits using a fixed sequence of quantum logic gates. In this sequence of quantum logic gates, which forms the operations of quantum algorithms. The calculation ends with a measurement that collapses the system of qubits into one of ²ⁿ pure states, where each qubit is a "0" or a "1", thereby decomposing into classical states. Therefore, traditional algorithms can also be executed on a quantum computing platform, where the result will typically be n-bit classical information.

[00924] In standard representation, the basic states of a qubit are called “0” and “1” states. However, during quantum computing, the state of a qubit can typically be a superposition of ground or basis states such that the qubit has a non-zero probability of being in the basis state "0" and simultaneously a non-zero probability of being in the basis state "1". Accordingly, the quantum nature of a qubit is largely a derivative of its ability to exist in a coherent superposition of basis states and the fact that the qubit state must have a phase. The qubit will retain its ability to exist in a coherent superposition of basis states as long as it is sufficiently isolated from sources of decoherence.

[00925] Therefore, to complete a calculation using a qubit, the state of the qubit is measured. As stated above, when performing a qubit measurement, the quantum character of the qubit may be temporarily lost and the superposition of basis states may collapse into either a "0" basis state or a "1" basis state. Therefore, in this way, the qubit again becomes similar to a traditional digital “bit”. However, the actual state of the qubit after collapse will depend on the various probabilistic states that occurred immediately before the measurement operation. Thus, qubits can be used to form quantum circuits, which themselves can be configured to form a quantum computer.

[00926] There are several general approaches to designing and operating a quantum computer. One approach that has been formulated is the circuit model of quantum computing. The circuit model of quantum computing requires long quantum coherence, so the type of information device used in quantum computers that support this approach would be a qubit, which by definition has a long coherence time. Accordingly, the circuit model of quantum computing is based on the premise that qubits can be formed and acted upon by logic gates, much like bits, and can be programmed using quantum logic to perform calculations such as Boolean calculations. Research has been conducted to develop qubits that can be programmed to perform quantum logic functions in this manner. For example, see Shor, 2001, arXiv.org:quant-ph/0005003, which is incorporated herein by reference in its entirety. Likewise, a computer processor can take the form of a quantum processor in the form of a superconducting quantum processor.

[00927] A superconducting quantum processor may comprise a number of qubits and associated biasing devices, such as two, three, or more superconducting qubits. Accordingly, although in various embodiments the computer processor may be configured as a non-conventional superconducting processor, in other embodiments the computer processor may be configured as a superconducting processor. For example, in some embodiments, the unconventional superconducting processor may be designed such that it does not focus on quantum effects such as superposition, entanglement, and/or quantum tunneling, but rather may operate by emphasizing various principles, such as principles encompassing operation of classical computer processors. In other embodiments, the computer processor may be a traditional superconducting processor, such as one that can adapt to the process through various quantum effects such as superposition, entanglement, and/or quantum tunneling.

[00928] Accordingly, the implementation of such superconducting processors may have certain advantages in various cases. In particular, due to their natural physical properties, superconducting processors can generally be designed to allow higher switching speeds and shorter computation times than non-superconducting processors, and therefore it may be more practical to solve certain problems on superconducting processors. Further details and implementations of examples of quantum processors that can be used in conjunction with the presented devices, systems and methods of using them are described in USSN: 11/317,838; 12/013.192; 12/575.345; 12/266.378; 13/678.266 and 14/255.561; as well as in various divisional applications, continuing applications and/or continuation-in-part applications thereof, including US Patent Nos. 7,533,068; 7,969,805; 9,026,574; 9,355,365; 9,405,876; and all their foreign equivalents, which are incorporated herein by reference in their entirety.

[00929] In addition, in addition to the above, quantum devices, systems and methods for their application to solve complex computing problems are also proposed. For example, the quantum devices and systems described herein may be used to control the quantum state of one or more information devices and/or systems in a coherent manner to perform one or more stages of a bioinformatics and/or genomic processing pipeline, e.g., to perform one or more operations in image processing, base detection, mapping, alignment, sorting, variant detection and/or other genomic and/or bioinformatics pipelines. In specific implementations, one or more operations may be a Burrows-Wheeler, Smith-Waterman, and/or HMM operation.

[00930] In particular, solving complex genomic and/or bioinformatics computing problems with a quantum computing device may include forming one or more qubits and using them to form quantum logic circuits representing the computational problem, encoding the representation of the logic circuit as a discrete optimization problem and solving a discrete optimization problem using a quantum processor. The representation may be an arithmetic and/or geometric problem to be solved using addition, subtraction, multiplication and/or division circuits. A discrete optimization problem may comprise a set of miniature optimization problems, where each miniature optimization problem encodes a corresponding logic gate from a logic circuit representation. For example, a mathematical diagram can use binary representations of factors, and these binary representations can be decomposed to reduce the total number of variables required to represent the mathematical diagram. Accordingly, in accordance with the teachings set forth herein, a computer processor may take the form of a digital and/or analog processor, for example, a quantum processor such as a superconducting quantum processor. A superconducting quantum processor may contain a number of qubits and associated local bias devices, for example, two or more superconducting qubits, which can be configured into one or more quantum logic circuit representations.

[00931] More specifically, a superconducting integrated circuit may be provided in various embodiments. Namely, in particular embodiments, such a superconducting integrated circuit may comprise a first superconducting current path that is located in a metal, such as a first metal layer. A dielectric layer, for example a first dielectric layer, may also be included, for example where at least a portion of the dielectric layer is associated with and/or provided on the first metal layer. The second superconducting current path may also be included and located in the second metal layer, for example, in a metal layer formed on or otherwise associated with the first dielectric layer. In such an embodiment, at least a portion of the second superconducting current path may overlap at least a portion of the first superconducting current path. Likewise, a second dielectric layer may also be included, for example, where at least a portion of the second dielectric layer is associated with and/or provided on the second metal layer. In addition, the third superconducting current path may be included and located in a third metal layer, which may be associated with or provided on the second dielectric layer, for example, where at least a portion of the third superconducting current path may overlap at least a portion of one or both from the first and second superconducting current paths. One or more additional layers, dielectric layers and/or current paths may also be included and configured accordingly.

[00932] In addition, a first superconducting connection may be located between the first superconducting current path and the third superconducting current path, for example, where the first superconducting connection extends through both the first and second dielectric layers. A second superconducting connection may be located between the first superconducting current path and the third superconducting current path, for example, where the second superconducting connection may extend through both the first and second dielectric layers. In addition, at least a portion of the second superconducting current path may be surrounded by an outer superconducting current path, which may be formed by at least a portion of one or more of the first superconducting current path, at least a portion of the second superconducting current path, and/or the first and second superconducting current paths. connections. Accordingly, in such cases, the second superconducting current path may be configured to couple, for example inductively coupling, the signal to the outer superconducting current path.

[00933] In some embodiments, the mutual inductance between the second superconducting current path and the outer superconducting current path may be approximately linearly proportional to the thickness of the first dielectric layer and the thickness of the second dielectric layer. Each of the second and first superconducting connections may also include at least one superconducting through-connection. Additionally, in various embodiments, the second superconducting current path may be part of an input signal line, and one or both of the first and third superconducting current paths may be coupled to the programmable superconducting device. In other embodiments, the second superconducting current path may be part of a programmable superconducting device, and the first and third superconducting current paths may both be coupled to an output signal line. In specific embodiments, the programmable superconducting device may be a superconducting qubit, which may also be coupled, for example, in a quantum manner, to one or more other qubits to form a quantum circuit, such as a quantum processing device.

[00934] Accordingly, devices, systems and methods are provided herein for solving computational problems, particularly problems associated with resolving bottlenecks in genomics and/or bioinformatics, as described above herein. In various embodiments, these devices, systems and methods introduce a method whereby a logic representation of a computational problem can be solved directly and/or can be encoded as a discrete optimization problem, and the discrete optimization problem can then be solved using a computer processor such as quantum processor. For example, in specific embodiments, solving such discrete optimization problems may include executing a logic circuit to solve the original computational problem.

[00935] Consequently, the devices, systems and methods described herein can be implemented using any form of computer processor, including, for example, traditional logic circuits and/or logic circuit representations, for example, configured to be used as quantum processor and/or superconducting processor. In particular, the various steps of performing image processing, base detection, mapping, alignment, and/or variant determination of a bioinformatics pipeline can be coded as discrete optimization problems, and therefore may be particularly suitable for solution by the quantum processors described herein. In other cases, such calculations can be resolved in a more conventional way by using a computer processor that adapts quantum effects to achieve such calculations; and/or in other cases, such calculations may be performed using a dedicated integrated circuit such as an FPGA, ASIC, or structured ASIC, as described in detail herein. In some implementations, the discrete optimization problem is treated as a problem of configuring logic circuits, qubits, and/or taps in a quantum processor. In some implementations, the quantum processor may be specifically configured to solve such discrete optimization problems.

[00936] Throughout this specification and the accompanying claims, reference is often made to a “logic circuit representation,” such as a computational problem. Depending on the context, a logic circuit may contain a set of logical inputs, a set of logical outputs, and a set of logic gates (for example, NAND gates, XOR gates, etc.) that convert logical inputs into logical outputs. through a set of intermediate logical inputs and intermediate logical outputs. A complete logic diagram may contain a representation of the inputs to a computational problem, a representation of the outputs from the computational problem, and a representation of the sequence of intermediate steps between the inputs and outputs.

[00937] Thus, when used for various purposes by the presented devices, systems and methods, a computational problem may be defined by its inputs, its outputs, and intermediate steps that transform the inputs into outputs, and a “logic diagram representation” may include all of these elements . Those skilled in the art will appreciate that encoding a "logic circuit representation" of a computational problem as a discrete optimization problem and then mapping the discrete optimization problem to a quantum processor can result in any number of participating layers and any number of qubits per layer. In addition, such a mapping may implement some kind of inter-qubit coupling scheme to provide some kind of inter-layer coupling scheme (for example, coupling between qubits of different layers) and intra-layer coupling (for example, coupling between qubits within a particular layer).

[00938] Accordingly, as noted, in some embodiments, the logic circuit structure may be divided into layers. For example, logical inputs may be represented by a first layer, each subsequent logical (or arithmetic) operation may represent a corresponding additional layer, and logical outputs may represent another layer. As described earlier, a logic operation can be performed by a single logic gate or a combination of logic gates depending on the specific logic operation being executed. Thus, a "layer" in a logic circuit may include a single logic gate or a combination of logic gates depending on the particular logic circuit being implemented.

[00939] Therefore, in various implementations, such as where the logic circuit structure is divided into layers (for example, where the logic inputs represent a first layer, each subsequent logic operation represents a corresponding additional layer, and the logic outputs represent another layer), each layer may be implemented using an appropriate set of qubits in a quantum and/or superconducting processor. For example, in one embodiment of a quantum processor, one or more, for example, each row, qubits may be programmed to represent a corresponding layer of a quantum logic circuit. That is, specific qubits can be programmed to represent inputs to a logic circuit, other qubits can be programmed to represent a first logical operation (executed by either one or multiple logic gates), yet other qubits can be programmed to represent a second logical operation (in a similar manner executed by either one or many logic gates), and the following qubits can be programmed to represent the outputs of the logic circuit.

[00940] Additionally, when different sets of qubits represent different layers of the problem, it may be desirable to provide independent dynamic control of each respective set. Additionally, in various embodiments, various sequential logic circuits may be mapped to a quantum processor and the corresponding qubits mapped to facilitate functional interactions for quantum processing in a manner suitable for allowing them to be independently controlled. Given the above, those skilled in the art will appreciate how a similar objective function can be defined for any logic gate. Thus, in some embodiments, a problem representing a logic circuit may essentially comprise a plurality of miniature optimization problems, where each gate in the logic circuit corresponds to a particular miniature optimization problem.

[00941] Therefore, using systems and methods that are known in the art, example logic circuit representations can be generated. In one example, a logic representation of a computational problem, such as a genomics and/or bioinformatics problem, may be generated and/or encoded using a classical digital computer processor and/or a quantum and/or superconducting processor as described herein. Accordingly, the logic representation of a computing problem may be stored on at least one computer- or processor-readable storage medium, such as a non-transitory computer-readable storage medium or memory (eg, volatile or non-volatile). Therefore, as noted above, a logic representation of a computational problem may be encoded as a discrete optimization problem or a set of optimization objectives, and in various implementations, such as where a classical digital computer processing model is configured to solve the problem, the system may be implemented such that the bit the terms that satisfy the logic circuit had zero energy, and all other bit strings had energy greater than zero, and then the discrete optimization problem can be solved in a way that solves the original computational problem.

[00942] Additionally, in other embodiments, the discrete optimization problem may be solved using a computer processor, such as a quantum processor. In such a case, solving the discrete optimization problem may involve, for example, deploying the quantum processor to a configuration that minimizes the system's energy to establish a bit string that satisfies the optimization goals. Accordingly, in some embodiments, the act of solving a discrete optimization problem may involve three actions. First, the discrete optimization problem can be mapped to a computer processor. In some embodiments, the computer processor may be a quantum and/or superconducting processor, and mapping the discrete optimization problem to the computer processor may include programming elements (e.g., qubits and couplers) of the quantum and/or superconducting processor. Mapping the discrete optimization problem to a computer processor may include the discrete optimization problem on at least one computer- or processor-readable storage medium, such as non-transitory computer-readable storage media or memory (eg, volatile or non-volatile).

[00943] Accordingly, in view of the foregoing, in various cases, an apparatus, system and method are provided for executing a sequence analysis pipeline, for example, on genomic material. For example, the genomic material may include multiple reads of genomic data, such as in a BCL, FASTQ, or the like image file. In various embodiments, the apparatus and/or system may be used to perform sequence analysis on genomic data, such as genomic data reads, e.g., using an index of one or more genetic reference sequences, e.g., stored in memory, e.g., where each genomic read data and each reference sequence represents nucleotide sequences.

[00944] In particular, in various embodiments, the device may be a quantum computing device, for example, formed from a collection of quantum logic circuits, for example, hard-wired quantum logic circuits, such as logic circuits connected to each other. In various cases, quantum logic circuits can be interconnected with each other using one or more superconducting connections. In addition, one or more superconducting connections may include a memory interface, for example for accessing memory. Together, logic circuits and interconnects can be configured to process information representing a quantum state that is itself represented as a collection of one or more qubits. More specifically, a set of hard-wired quantum logic circuits may be implemented as a set of processing engines, for example, where each processing engine may be formed by a subset of hard-wired quantum logic circuits and may be configured to implement one or more stages in the sequence-on-read analysis pipeline genomic data.

[00945] For example, a set of processing engines may be configured to include an image processing, base detection, mapping, alignment, sorting, variant detection, and/or other genomics and/or bioinformatics module. For example, in various embodiments, a mapping module may be included, for example, in a first hard-wired configuration. Additionally, in other embodiments, an alignment module may be included, for example, in a second hard-wired configuration. Further, a sorting module may be included, for example, in a third hard-wired configuration. And in additional embodiments, a variant determination module may be included, for example, in a fourth hard-wired configuration. Moreover, in various embodiments, an image processing and/or variant determination module may be included in additional hard-wired configurations, for example, where one or more of these hard-wired configurations may contain hard-wired quantum logic circuits, which module may be organized into in the form of a set of processing engines.

[00946] Namely, in specific cases, a quantum computing device and/or system may comprise a mapping module, where the mapping module includes a plurality of quantum logic circuits that are organized as a set of processing engines, one or more of which are configured to implement one or more more stages of the mapping procedure. For example, one or more quantum processing engines may be configured to receive a read of genomic data, for example, through one or more of a plurality of superconducting connections. Further, one or more quantum processing engines may be configured to extract a portion of the read to form a seed, for example where the seed may be a subset of the nucleotide sequence represented by the read. In addition, one or more quantum processing engines may be configured to calculate the first address in the index based on the seed and access the address in the index to memory to accept entries from the address, for example, when the entry represents position information in a genetic reference sequence. In addition, one or more quantum processing engines may be configured to determine, for example, based on a record, one or more matching read positions with a genetic reference sequence and output at least one of the matching positions to memory via a memory interface.

[00947] Moreover, the mapping module may include a set of quantum logic circuits that are organized as a set of processing engines configured to calculate a second address in the index, for example, based on an entry and a second subset of the nucleotide sequence that is not contained in the first subset of the nucleotide sequence. The processing engines may then access the second address in the index into memory to accept a second entry from the second address, for example, when the second entry, or subsequent entry, contains information about a position in a genetic reference sequence. The processing engine may also be configured to determine, based on the position information, one or more matching read positions to the genetic reference sequence.

[00948] Additionally, in various cases, a quantum computing device and/or system may include an alignment module, where the alignment module includes a plurality of quantum logic circuits that are organized as a set of processing engines, one or more of which are configured to implement one or more more steps in the alignment procedure. For example, one or more quantum processing engines may be configured to receive a plurality of mapped positions from memory and access the memory to retrieve a genetic reference sequence segment corresponding to each mapped position. One or more processing engines configured as an alignment module may also be configured to calculate a read alignment for each extracted genetic reference sequence segment to generate a score for each alignment. Next, after one or more scores have been generated, at least one read with the best alignment score can be selected. In specific cases, a quantum computing device may comprise a set of quantum logic circuits organized as a set of processing engines that are configured to perform gapped or gapless alignment, such as Smith-Waterman alignment.

[00949] Also, in certain cases, a quantum computing device and/or system may include a variant determination module, where the variant determination module includes a plurality of quantum logic circuits that are organized as a set of processing engines, one or more of which are configured to implement one or more steps in the options identification procedure. For example, a variant detection quantum computing module may comprise a set of quantum logic circuits that are configured to perform analysis on multiple genomic data reads, e.g., using one or more candidate haplotypes, e.g., stored in memory, where each genomic data read and each a candidate haplotype is a sequence of nucleotides.

[00950] In particular, a set of quantum logic circuits may be configured as one or more quantum processing engines that are configured to receive one or more reads of genomic data and generate and/or receive one or more candidate haplotypes, for example, from memory, for example, through one or more of a variety of superconducting compounds. In addition, one or more of the quantum processing engines may be configured to receive one or more genomic data reads and one or more candidate haplotypes from memory, and compare nucleotides in each of the one or more reads to nucleotides of one or more haplotypes. candidates to determine the likelihood of each candidate haplotype representing the correct variant definition. In addition, one or more quantum processing engines may be configured to generate output based on a certain probability.

[00951] Additionally, in various cases, a set of quantum logic circuits may be configured as one or more quantum processing engines that are configured to determine the probability of observing each read of a plurality of reads based on at least one candidate haplotype being a valid nucleotide sequence , for example, an organism that is the source of many reads. In special cases regarding probability determination, one or more quantum processing engines can be implemented to execute the hidden Markov model. More specifically, in additional embodiments, one or more quantum processing engines may be configured to combine multiple reads into one or more contiguous nucleotide sequences and/or generate one or more candidate haplotypes from one or more contiguous nucleotide sequences. For example, in various embodiments, combining multiple reads involves one or more quantum processing engines constructing a de Bruijn graph.

[00952] Accordingly, in view of the above, a system is proposed for performing various calculations in solving problems associated with genomic and/or bioinformatics processing. For example, the system may include one or more automated on-site sequencers, such as an SNP, and/or a processing server, one or both of which may comprise one or more CPU/GPU and/or other integrated circuit, in particular such as an FPGA, an ASIC, and/or a structured ASIC that is configured to implement one or more steps in a sequence analysis pipeline as described herein. In particular, a next generation sequencer may be configured to sequence a plurality of nucleic acid sequences to generate one or more BCL and/or FASTQ image files representing the sequenced nucleic acid sequences, wherein the nucleic acid sequences may be DNA and/or RNA sequences. These sequence files may be processed by the sequencer itself or an associated server device, for example, where the sequencer and/or associated server contains embedded circuitry, such as an FPGA or ASIC, configured as described herein, capable of implementing one or more stages in a secondary pipeline. sequence analysis.

[00953] However, in various cases, such as when the automated sequencer and/or associated server is not configured to perform secondary sequence analysis on the data generated by the sequencer, the generated data may be transmitted to a remote server that is configured to perform secondary and/or tertiary analysis on data, for example through a cloud interface. In such a case, the cloud accessible server may be configured to receive generated sequence data, such as an image in the form of a BCL and/or FASTQ file, and may also be configured to perform analysis of the primary, such as image processing, and/or secondary, and /or tertiary processing, such as a sequence analysis pipeline, on the received data. For example, the cloud-accessible server may be one or more servers comprising a CPU and/or GPU, of which one or both may be coupled to an integrated circuit such as an FPGA or ASIC, as described herein. In particular, in certain cases, the server accessible from the cloud may be a quantum computing server, as described herein.

[00954] Namely, the cloud-accessible server may be configured to perform primary, secondary and/or tertiary genomic and/or bioinformatics analysis on the received data, which analysis may include performing one or more steps in one or more of the protocols image processing, base determination, mapping, alignment, sorting and/or variant determination. In certain cases, some steps may be performed by one processing platform, such as a CPU or GPU, and others may be performed by another processing platform, such as an associated, e.g., hard-coupled, integrated circuit such as an FPGA or ASIC that is specifically configured to implementing various steps in the sequence analysis pipeline. In such cases, where data and analysis results are to be transferred from one platform to another, the system and its components may be configured to compress the data before transmission and decompress the data after transmission, and therefore such system components may be configured to generate one or more SAM, BAM or CRAM files, for example for transfer. Additionally, in various embodiments, the cloud-accessible server may be a quantum computing platform that, as described herein, is configured to perform one or more steps in a sequence analysis pipeline as described herein, and may include performing one or more secondary and/or tertiary processing steps in accordance with one or more methods described herein.

[00955] In addition, with respect to quantum computing, details and implementations of examples of quantum processors and methods of application thereof that can be used in conjunction with the presented devices, systems and methods are described in US Pat. Nos. 7,135,701; 7,533,068; 7,969,805; 8,560,282; 8,700,689; 8,738,105; 9,026,574; 9,355,365; 9,405,876; as well as various analogues thereof, which are incorporated herein by reference in their entirety.

[00956] In addition, with respect to the artificial intelligence module described above, in accordance with one aspect, there is provided a cloud-accessible artificial intelligence module that is operably connected in communication with one or more other components of the BioIT pipeline described in this document. For example, the AI module may work closely with the WMS to effectively guide and/or control the various processes of the system described in this manual. Accordingly, in various embodiments, an AI module is provided that is configured to act as an interface between the genomic and clinical domains.

[00957] For example, in various cases, the BioIT system may be configured to receive clinical data. In such a case, the workflow manager system may be configured to analyze clinical data and other such data and implement one or more deterministic rule systems to produce outcome data in accordance with its analysis of the clinical data. For example, in certain embodiments, the various databases of the system may be configured to have a relational architecture.

[00958] These constructs may be represented by one or more table structures. The WMS can then use a series of tables, for example, for correlation in an iterative manner. For example, in various usage models, the first correlation may be made against a subject's name with a medical condition. Another table can then be used to correlate the subject's medical condition with his conservative treatment. Similarly, another table can be used to correlate the progress of conservative treatment with the improvement of symptoms and/or the disease itself. To correlate tables, a key can be used, access to which is provided in response to a given question or command. The key may be some general identifier such as a name, number such as a social security number, taxpayer identification number, employee code, telephone number, etc., by which one or more tables can be accessed, correlated, and/or or get an answer to a question. Accordingly, without a key it will be much more difficult to build correlations between information in one table with information in another table.

[00959] However, in other cases, the AI module may be configured to provide more comprehensive analysis on the generated and/or provided data. For example, the AI module may be configured to implement one or more machine learning protocols on system data that are designed to train the AI module to create correlations between genomic data, e.g., generated by the system, and accumulated clinical data of one or more subjects, e.g. with EMR and other clinically relevant data entered into the system.

[00960] In particular, the AI module may include programming to quickly train the system, such as instantly, to recognize how output data was obtained based on the type or characteristics of input data received. Therefore, the system is configured to learn from the inputs it receives and the results of its outputs in order to be able to more quickly and accurately establish correlations based on the inputs it receives. Typically, input data can be of two general types. In the first case, the data may be of a type in which the output, such as the response, is known. This type of data can be entered into the system and used for training purposes. The second type of data may be data where the answer is not known, and therefore must be determined; these data will likely be genomic data on which analysis needs to be performed, or clinical data on which clinically relevant outcomes need to be determined. In particular, these methods can be used to improve the ability of AI modules to learn from inputs of the first type in order to better predict the outcome of inputs of the second type. Namely, based on past confirmatory data, the AI module may be configured to learn to predict outcomes based on previously observed data.

[00961] More specifically, this document presents a clinical genomic platform that is capable of correlating clinical disease outcomes with genomic data. Subjects' clinical profiles can then be entered into the system and assessed along with their defined genomic profile. In particular, the AI module is configured to determine, based on the combination of these two data sets, various interdependencies between them. Accordingly, at the first stage, you can build a graph database or knowledge graph. For example, in this case, the knowledge graph may contain three types of elements, which include subject, predicate and object, and these elements form nodes, and the relationship between the nodes needs to be defined. Any specific data point can be selected as a node, and the nodes can change based on the queries being executed. There are several different types of relationships that can be defined. For example, relationships can be defined based on their consequences, such as their influence; or they can be determined by inference, such as relationships that are unknown but determinable.

[00962] Accordingly, with regard to the construction of a knowledge graph, any particular data point can form a node. For example, on one side of the graph the node could be a disease state, and on the other side of the graph the node could be a genotype, such as a sequence of variations. Between these two nodes there may be a third node, for example, a series of nodes such as one or more symptoms, one or more medications, one or more allergies, one or more other conditions or phenotypic traits, such as blood pressure, cholesterol, etc. d. In addition, there are relationships between these nodes that can be defined.

[00963] In particular, when constructing a knowledge graph, clinical data is entered into the system, for example, from an institution maintaining medical records, such as electronic medical records, family history of medical conditions, etc., which can be encrypted and securely transmitted using electronic means. Likewise, a subject's genomic data may be sequenced and generated according to the secondary processing steps described herein. In addition, after these two nodes have been created, one or more nodes of a third type can be introduced into the system, from which the relationship(s) between the two original nodes can be determined.

[00964] For example, in one example, the first node may be represented by medical records of a person or population of people, and the second node may be represented by a characteristic of a disease. In this case, one or more third nodes can be introduced into the system and formed on the graph, where, for example, the third node can be a drug; physical, biological, mental state and/or characteristic; allergies; geographical area; diet, food product and/or ingredient; environmental conditions; geographical condition; power lines, cell towers, etc. A series of relationships can then be identified by analyzing the various connection points between these three elements. Specifically, in a given case, one node may represent a patient suffering from a disease condition, a second node may be the patient's genomic data, and third nodes may include the patient's genomic variations, such as the subject's chromosome-by-chromosome mutations, his medications, physiological conditions and so on. Likewise, this process can be repeated for multiple subjects having the same diagnosis or condition. Therefore, the correlation between clinical and genomic domains of activity can be determined in a similar manner.

[00965] Accordingly, a step in the construction of a clinical genomic graph is the definition of anchor nodes, which are two boundary elements between which all the different entities are defined and studied. Therefore, the next step is to identify all possible known matches between two anchor nodes that can be represented in the graph as a third node. These known correspondences can be constructed by detailing the consequences caused by one or another node and/or their characteristics. These may be known and/or observable relationships between nodes. Based on these known relationships, the second type of relationship can be explored and/or what relationships can be inferred. Further, to better determine causal and/or predictive outcomes, weights can be assigned to various different relationships, for example, based on degree of certainty, number of commonalities, number of instances sharing a node, number of common relationships, and the like.

[00966] Thus, the construction and implementation of a dynamic knowledge graph is at the core of a clinical genomic processing platform. As stated, the various processing platforms of a global system can be linked together, for example, to seamlessly transfer data between its various components. For example, as noted, the mapping, alignment, and/or variant determination pipelines may be configured to pass data, such as result data, to an artificial intelligence module. In particular, the AI module may be configured to receive input data from one or more components of the secondary processing platform and/or one or more other components of the system. More specifically, the AI module is configured to receive mapping, alignment, and/or variant determination data from the matcher, aligner, and/or variant determination processing engines and take that data and use it to form one or more nodes in the knowledge graph. In addition, the AI module may be configured to receive input data from one or more sources, for example, from a medical facility, from a healthcare provider, from a research laboratory, a record storage facility, and the like, for example, when the records contain data, relating to the physical, mental and/or emotional well-being of one or more subjects, and taking this data into circulation and using it to form one or more nodes in the knowledge graph.

[00967] Additionally, once the knowledge graph architecture is built, it can be continuously updated and grown by adding more and more relevant data to the knowledge structure, building more and more potential nodes and/or relationships. In such a case, the boundary nodes can be any combination of nodes, for example, in certain cases can be selected by the user. For example, in various embodiments, the system may be configured to be accessible by a third party. In such a case, the user can access the AI module, for example through a suitably configured user interface, upload relevant information to the system and/or define relevant nodes to limit the query to, for example, by clicking or dragging them, and can formulate a question to receive a response from the AI module. Accordingly, the user can view and/or select boundary nodes and then allow the system to generate a corresponding knowledge map using the selected nodes and determine the relationships between those nodes, and based on these relationships, questions can be asked and answers obtained, at least speculatively, e.g. from the AI system.

[00968] For example, in one use model, the user may be a physician who wants to know how a certain dosage of a drug affects a patient for a given disease. Accordingly, the doctor can upload the patient's EMR, disease state and drug dosage, and with the help of this data, the AI module can generate a suitable knowledge graph (and/or add to an existing knowledge graph), and based on this knowledge graph, boundary nodes can be selected and identify relationships. In addition, in various cases, the user can upload the patient's genetic data, which can be subjected to secondary processing, and the results, such as mapped, aligned and/or variant-identified resulting data, can be uploaded to the AI module. In such a case, disease and/or EMR and/or family history data can be correlated with genomic data, from which various relationships can be determined, inferences assessed, and predictions made.

[00969] In particular, a subject's VCF file can be entered into the system, for example, all identified chromosomal properties can be downloaded, for example, as a group of nodes, and these nodes can be used to define various relationships relevant to the subject, for example, by entering request to the system and allow it to form appropriate connections from which responses can be logically derived. More precisely, one or more phenotypic characteristics of a subject, for example, a human phenotype ontology, can be uploaded into the system to form another group of nodes. For example, when the genomic and/or medical histories of two people are entered into the system, any relationships between them can be determined by the AI module, for example in relation to common genotypes, phenotypes, conditions, environment, geographical locations, allergies, ethnic-cultural background, medications and so on.

[00970] In addition, relationships between two or more characteristics of a subject can be determined. For example, the system may determine the relationship between a subject's systolic and diastolic blood pressure. In particular, a series of historical systolic and diastolic pressure readings may be input into the system, from which the machine learning platform of the system may analyze those readings and/or determine one or more relationships between the two pressures, such that when a given systolic pressure is entered into the system, a expected diastolic pressure, taking into account predictable relationships between them. It should be noted that although the previous example is given for the blood pressure of a single subject, the same is true for any given nodes that are mathematically related to each other, for example, for multiple subjects and/or multiple conditions.

[00971] Additionally, while in some cases the relationships may be configured in a linear array, such as forming a neural network of information, in various other cases the relationships may be formed in multiple stages, such as in a deep learning protocol. For example, an AI system may be designed to process information in layers or stages, such as for deep learning purposes. Accordingly, the system may be configured to evaluate data at stages. In particular, the AI module may be designed such that as it examines various data, such as running a learning protocol, stage by stage, each relationship between data is assigned a weight by the system, for example, based on past supporting data and/or characteristics of the relationships.

[00972] The more stages of learning initiated in the system, the better the weighting of connections and the deeper the learning. In addition, uploading data at stages improves data convergence in the system. In particular, you can also use different feature extraction paradigms to better organize, weight, and analyze the most salient features of the data that will be dumped. Additionally, to better correlate data, one or more users may input and/or modulate basic weighting functions, although the system itself may employ more advanced weighting functions based on active learning protocols.

[00973] To provide user interaction, one or more aspects or features of the subject matter described herein may be implemented on a computer having a display device, such as, for example, a cathode ray tube (CRT), liquid crystal display (LCD), or a light-emitting diode (LED) monitor for displaying information to the user, and a keyboard or pointing device, such as a mouse or trackball, with which the user can make inputs into the computer. Other types of devices can also be used to provide user interaction. For example, the user feedback may be any form of sensory feedback, such as, for example, visual feedback, audio feedback, or haptic feedback; and user input may be received in any form, including, but not limited to, audio, speech, or tactile input. Other possible input devices include, but are not limited to, touch screens or other touch devices such as single or multi-point resistive or capacitive touch panels, voice recognition hardware or software, optical scanners, optical pointers, digital image capture devices with associated software provision for interpretation, etc.

[00974] The subject matter described herein may be implemented in systems, devices, methods and/or products depending on the desired configuration. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. On the contrary, these are just some examples corresponding to aspects related to the described subject matter. Although several options are described in detail above, other modifications and additions are possible. In particular, the features and/or options described herein may be supplemented by other features and/or options. For example, the implementations described above may relate to various combinations and subcombinations of the described features and/or to combinations and subcombinations of several additional features described above. In addition, the logical flows depicted in the accompanying figures and/or described herein do not necessarily require adherence to the particular order shown or sequential order to achieve the desired results. Other implementations are possible within the scope of the following claims.

Claims (54)

1. A method for improving the accuracy of variant detection by jointly evaluating reads that map two or more regions of a reference sequence that are homologous, the method comprising: accessing, by one or more computers, a shared collection of multiple sequence reads, wherein the shared collection comprises a first read collection aligned to a first region of a reference sequence, and at least a second collection of reads aligned to a second region of a reference sequence, wherein the first region and the second region homologous to each other; determining, by one or more computers, a set of candidate variants from the joint population, each candidate variant in the set of candidate variants corresponding to a candidate variant between the determination of the base of one of said plurality of sequence reads and the determination of the base of a corresponding reference sequence location; establishing, by one or more computers, an order for processing the candidate variants; evaluating, by one or more computers, each candidate of the set of candidates based on the established processing order; And generating, by one or more computers and based on the evaluation of the candidate variants, a variant definition file identifying the one or more candidate variants. 2. The method according to claim 1, also including obtaining a plurality of homologous reference sequence regions from one or more storage devices. 3. The method according to claim 1, according to which determining a set of candidate variants using a joint cluster includes using the de Bruijn graph to identify candidate variants from a joint population. 4. The method according to claim 3, according to which the nodes in the specified graph represent a list of candidates, wherein the use of the de Bruijn graph involves the formation of a de Bruijn graph using each region of the specified reference sequence as a skeleton and aligning each position of the candidate variant to universal coordinates. 5. The method of claim 1, wherein establishing, by one or more computers, an order in which candidate variants are processed includes establishing, by one or more computers, the order in which candidate variants are processed as a function of read length or insertion size. 6. The method according to claim 5, according to which establishing the order of processing candidate variants as a function of the read length or insertion size includes formation of a connectivity matrix that establishes the order of processing candidate variants as a function of the read length and insertion size. 7. The method of claim 1, wherein the evaluation by one or more computers of each candidate from the set of candidates based on a specified processing order includes: for each candidate option from a set of candidate options: formation of joint candidate diplotypes, calculation of the posterior probability of each joint diplotype, computing the genotype matrix, pruning joint candidate diplotypes, and including the next active position as supporting data for the current position. 8. A system for improving the accuracy of variant detection by jointly evaluating reads that map two or more regions of a reference sequence that are homologous, wherein the system contains one or more computers and one or more storage devices storing instructions configured to cause, when executed by one or more computers, the performance of one or more computers operations, including: accessing, by one or more computers, a shared collection of multiple reads of a sequence, wherein the shared collection comprises a first collection of reads aligned to a first region of a reference sequence, and at least a second collection of reads aligned to a second region of a reference sequence, wherein the first region and the second region homologous to each other; determining, by one or more computers, a set of candidate variants from the joint population, each candidate variant in the set of candidate variants corresponding to a candidate variant between the determination of the base of one of said plurality of sequence reads and the determination of the base of a corresponding reference sequence location; establishing, by one or more computers, an order in which the candidate variants are processed; evaluating, by one or more computers, each candidate of the set of candidates based on the established processing order; And generating, by one or more computers and based on the evaluation of the candidate variants, a variant definition file identifying the one or more candidate variants. 9. The system according to claim 8, in which the operations also include obtaining a plurality of homologous reference sequence regions from one or more storage devices. 10. The system of claim 8, wherein determining a set of candidate variants using the joint aggregation includes using the de Bruijn graph to identify candidate variants from a joint population. 11. The system of claim 10, wherein the nodes in said graph represent a list of candidates, wherein the use of the de Bruijn graph involves forming a de Bruijn graph using each region of the specified reference sequence as a skeleton and aligning each position of the candidate candidate to universal coordinates. 12. The system of claim 8, wherein establishing, by one or more computers, an order in which candidate variants are processed includes establishing, by one or more computers, the order in which candidate variants are processed as a function of read length or insertion size. 13. The system of claim 12, wherein establishing the processing order of candidate variants as a function of read length or insertion size includes formation of a connectivity matrix that establishes the order of processing candidate variants as a function of the read length and insertion size. 14. The system of claim 8, wherein the evaluation by one or more computers of each candidate of the set of candidates based on the established processing order includes: for each candidate option from a set of candidate options: formation of joint candidate diplotypes, calculation of the posterior probability of each joint diplotype, computing the genotype matrix, pruning joint candidate diplotypes, and including the next active position as supporting data for the current position. 15. A machine-readable storage device storing instructions that, when executed by a processing device, cause the processing device to perform operations to improve the accuracy of variant detection by jointly evaluating reads that map two or more regions of a reference sequence that are homologous, wherein the operations include: accessing, by one or more computers, a shared collection of multiple sequence reads, wherein the shared collection comprises a first read collection aligned to a first region of a reference sequence, and at least a second collection of reads aligned to a second region of a reference sequence, wherein the first region and the second region homologous to each other; determining, by one or more computers, a set of candidate variants from the joint population, each candidate variant in the set of candidate variants corresponding to a candidate variant between the determination of the base of one of said plurality of sequence reads and the determination of the base of a corresponding reference sequence location; establishing, by one or more computers, an order in which the candidate variants are processed; evaluating, by one or more computers, each candidate of the set of candidates based on the established processing order; And generating, by one or more computers and based on the evaluation of the candidate variants, a variant definition file identifying the one or more candidate variants. 16. The machine-readable storage device of claim 15, wherein the operations further include obtaining a plurality of homologous reference sequence regions from one or more storage devices. 17. The computer-readable storage device of claim 15, wherein determining a set of candidate candidates using the co-aggregation includes using the de Bruijn graph to identify candidate variants from a joint population. 18. The machine-readable storage device of claim 17, wherein the nodes in said graph represent a list of candidates, wherein using the de Bruijn graph includes generating a de Bruijn graph using each region of said reference sequence as a backbone and aligning each position of the candidate candidate to universal coordinates . 19. The computer-readable storage device of claim 15, wherein establishing, by the one or more computers, an order of processing the candidate variants includes formation of a connectivity matrix that establishes the order of processing candidate variants as a function of the read length and insertion size. 20. The computer-readable storage device of claim 15, wherein the evaluation by the one or more computers of each candidate of the set of candidates based on the established processing order comprises: for each candidate option from a set of candidate options: formation of joint candidate diplotypes, calculation of the posterior probability of each joint diplotype, computing the genotype matrix, pruning joint candidate diplotypes, and including the next active position as supporting data for the current position.

Images