WO2024259679A1 - 基因序列的子序列搜索方法、比对方法、系统及设备 - Google Patents

基因序列的子序列搜索方法、比对方法、系统及设备 Download PDF

Info

Publication number
WO2024259679A1
WO2024259679A1 PCT/CN2023/101916 CN2023101916W WO2024259679A1 WO 2024259679 A1 WO2024259679 A1 WO 2024259679A1 CN 2023101916 W CN2023101916 W CN 2023101916W WO 2024259679 A1 WO2024259679 A1 WO 2024259679A1
Authority
WO
WIPO (PCT)
Prior art keywords
search
gene sequence
data
sequencing data
subsequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/101916
Other languages
English (en)
French (fr)
Inventor
袁东
龚淳
李胜康
张勇
黎宇翔
陈奥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bgi Hubei
BGI Shenzhen Co Ltd
Original Assignee
Bgi Hubei
BGI Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bgi Hubei, BGI Shenzhen Co Ltd filed Critical Bgi Hubei
Priority to PCT/CN2023/101916 priority Critical patent/WO2024259679A1/zh
Priority to CN202380094923.5A priority patent/CN120826741A/zh
Publication of WO2024259679A1 publication Critical patent/WO2024259679A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16B—BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
    • G16B30/00—ICT specially adapted for sequence analysis involving nucleotides or amino acids
    • G16B30/10—Sequence alignment; Homology search

Definitions

  • the present invention relates to the field of nucleic acid information, and in particular to a subsequence search method, comparison method, system and equipment for gene sequences.
  • a gene sequence is a long string of four bases, ACGT.
  • One way to compare gene sequences is to compare a short gene sequence with a reference gene sequence to find out the position of the short gene sequence in the reference gene sequence and the difference between the two.
  • Existing comparison software has problems such as high latency, low search efficiency, and poor performance when processing gene sequences, and cannot meet the processing requirements of gene sequences.
  • the technical problem to be solved by the present invention is to overcome the defect of low search efficiency of gene search algorithms in the prior art and to provide a subsequence search method, comparison method, system and device for gene sequences.
  • the present invention provides a subsequence search method for a gene sequence, the subsequence search method comprising:
  • the search interval for the next search is determined according to the next base to be searched for each sequencing data and the search interval of the previous search.
  • the number of sequencing data of the gene sequence to be searched is based on the base number of the search interval.
  • the pre-fetching duration and calculation duration of the base data are determined.
  • the subsequence search method further comprises:
  • the step of repeatedly obtaining base data of the search interval for the next search includes:
  • the pre-fetching of base data in the search interval is restarted in the search phase with the end position of the previous search as the starting position.
  • the step of repeatedly acquiring base data of the search interval for the next search further includes:
  • the step of obtaining a preset amount of sequencing data of the gene sequence to be searched includes:
  • the sequencing data is received from an IO (input and output) thread using an empty queue of a buffer pool; the empty queue is composed of buffer areas that do not store data, and the buffer areas enter the full queue of the buffer pool after storing data;
  • the full queue is used to transmit the sequencing data to the comparison thread for subsequence search; after the comparison thread completes the subsequence search, the data stored in the buffer area is released.
  • the present invention also provides a gene sequence comparison method, the comparison method comprising:
  • the gene sequence to be compared is obtained by using the gene sequence subsequence search method as described above.
  • the target subsequence is expanded to obtain the alignment result of the gene sequence to be aligned.
  • the present invention also provides a subsequence search system for a gene sequence, the subsequence search system comprising:
  • a sequencing data acquisition module used to acquire a preset amount of sequencing data of the gene sequence to be searched
  • a search data acquisition module used to sequentially acquire base data of a search interval corresponding to each sequencing data on a reference gene sequence, and perform a search based on the acquisition result; the search interval corresponding to each sequencing data is determined based on a base sequence to be searched for each sequencing data;
  • a subsequence determination module is used to repeatedly call the search data acquisition module to obtain base data of a search interval for the next search and to search based on the acquisition result to obtain a target subsequence; the search interval for the next search is determined according to the next base to be searched for each sequencing data and the search interval of the previous search.
  • the amount of sequencing data of the gene sequence to be searched is determined based on the pre-fetching duration and calculation duration of the base data in the search interval.
  • the subsequence search system further comprises:
  • a search phase configuration module configured to configure multiple search phases for each of the sequencing data according to the search direction and the starting position corresponding to each of the sequencing data;
  • the subsequence determination module is specifically used to call the search data acquisition module to restart the pre-fetching of base data of the search interval in the search phase with the end position of the previous search as the starting position when there is no corresponding search interval in the next search.
  • the subsequence determination module is specifically used to call the search data acquisition module to restart the pre-fetching of base data in the search interval in the next search phase when the base data in the search interval corresponding to the base sequence to be searched in the current search phase are all pre-fetched; and/or,
  • the subsequence determination module is specifically used to call the sequencing data acquisition module to acquire the next sequencing data of the gene sequence to be searched when the base data of the search interval corresponding to the forward base sequence to be searched of the current sequencing data in the current search phase are all pre-fetched; and/or,
  • the subsequence determination module is specifically used to call the sequencing data acquisition module to acquire the next sequencing data of the gene sequence to be searched when the base data of the search intervals in all search phases are pre-fetched.
  • the sequencing data acquisition module is specifically used to receive the sequencing data from the IO thread using an empty queue of the buffer pool;
  • the empty queue is composed of buffer areas that do not store data, and the buffer areas enter the full queue of the buffer pool after storing data;
  • the sequencing data acquisition module is specifically used to use the full queue to transmit the sequencing data to the comparison thread for subsequence search; after the comparison thread completes the subsequence search, it releases the data stored in the buffer area.
  • the present invention also provides a gene sequence comparison system, the comparison system comprising:
  • a gene sequence acquisition module is used to acquire the gene sequence to be compared
  • a target sequence determination module used to obtain a target subsequence of the gene sequence to be compared using the gene sequence subsequence search system as described above;
  • the target sequence expansion module is used to expand the target subsequence to obtain the comparison result of the gene sequence to be compared.
  • the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the subsequence search method of the gene sequence as described above or the comparison method of the gene sequence as described above is implemented.
  • the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the subsequence search method of a gene sequence as described above or the gene sequence comparison method as described above is implemented.
  • the gene sequence subsequence search method provided by the present invention obtains the sequencing data of the gene sequence to be searched in batches, and pre-extracts the base data of the search interval corresponding to each sequencing data on the reference gene sequence, and processes the sequencing data in batches so that the CPU can process the calculation of other sequencing data while waiting for the pre-fetched results, that is, the batch processing makes up for the CPU's calculation time of waiting for the pre-fetched results. It saves time and improves the efficiency of gene search.
  • FIG1 is a flow chart of a subsequence search method for a gene sequence in Example 1 of the present invention.
  • FIG. 2 is an example diagram of sequencing data in FASTQ format in Example 1 of the present invention.
  • FIG. 3 is an example diagram of determining the amount of sequencing data of a gene sequence to be searched in Embodiment 1 of the present invention.
  • FIG. 4 is an example diagram of parsed sequencing data in Example 1 of the present invention.
  • FIG. 5 is a schematic diagram of the setting search phase in Embodiment 1 of the present invention.
  • FIG. 6 is a flow chart of the gene sequence comparison method in Example 2 of the present invention.
  • FIG. 7 is a schematic diagram of the structure of a subsequence search system for a gene sequence in Example 3 of the present invention.
  • FIG8 is a schematic diagram of the structure of the gene sequence comparison system in Example 4 of the present invention.
  • FIG9 is a schematic diagram of the structure of an electronic device in Embodiment 5 of the present invention.
  • system means for distinguishing different components, elements, parts, portions or assemblies at different levels.
  • device means for distinguishing different components, elements, parts, portions or assemblies at different levels.
  • unit means for distinguishing different components, elements, parts, portions or assemblies at different levels.
  • the words can be replaced by other expressions.
  • a or B at least one of A and/or B, or “one or more of A and/or B” include any and all combinations of the words listed therewith.
  • “A or B”, “at least one of A and B”, or “at least one of A or B” means (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.
  • the core algorithm of the alignment software is mainly divided into two parts: Seed search and expansion based on Seed; Seed refers to a subsequence of a sequenced base sequence.
  • the search efficiency of Seed has a great impact on the alignment efficiency, so general alignment software uses suffix array (SA, suffix) in the Seed stage.
  • Array suffix array
  • HashTable hash table
  • other index structures improve the efficiency of Seed search.
  • this embodiment proposes a subsequence search method for a gene sequence.
  • Figure 1 is a flow chart of the subsequence search method of the gene sequence in this embodiment. Specifically, as shown in Figure 1, the subsequence search method includes:
  • S101 obtaining a preset amount of sequencing data of the gene sequence to be searched; specifically, as shown in FIG2 , taking FASTQ (a file format for sequencing data) as an example, the file has a record (name, sequence, annotation, quality value) for every 4 lines, and the record is called reads.
  • FASTQ a file format for sequencing data
  • the base data of the search interval can be obtained by pre-fetching technology; pre-fetching is a technology used by the CPU (Central Processing Unit, central processing unit) to extract instructions or data from the slower memory to the faster local Cache (cache memory) before they are actually needed, thereby improving execution performance; if there is no pre-fetching mechanism, due to the dependency between data, most of the time is wasted waiting for the relevant data in the memory, and the computing unit is idle most of the time.
  • pre-fetching is a technology used by the CPU (Central Processing Unit, central processing unit) to extract instructions or data from the slower memory to the faster local Cache (cache memory) before they are actually needed, thereby improving execution performance; if there is no pre-fetching mechanism, due to the dependency between data, most of the time is wasted waiting for the relevant data in the memory, and the computing unit is idle most of the time.
  • the pre-fetching mechanism allows the computing unit and the related access unit to maintain efficient operation and maximize the use of related hardware resources; in this embodiment, by batch processing sequencing data, the CPU can process the calculation of other reads while waiting for the pre-fetch results, that is, by batch processing, the CPU calculation time fills the waiting time for pre-fetching, which is equivalent to hiding the CPU waiting time for pre-fetching, thereby improving the efficiency of gene search.
  • software pre-fetching is adopted.
  • the amount of sequencing data of the gene sequence to be searched is determined based on the pre-fetching time and calculation time of the base data in the search interval.
  • the CPU just receives the pre-fetched result of the sequencing data R1, and can continue the search calculation based on the pre-fetched result of the sequencing data R1. According to actual tests, this method can achieve the best performance when the number of batch read sequencing data is 8 to 16.
  • step S101 includes:
  • S1011 using an empty queue of the buffer pool to receive sequencing data from the IO thread;
  • the empty queue is composed of buffer areas that do not store data, and the buffer areas enter the full queue of the buffer pool after storing data;
  • the sequencing data can be analyzed by the following steps:
  • the comparison phase is multi-threaded. Multiple threads compete for pipeline reading permissions. Threads that fail in the competition need to wait.
  • the thread that successfully competes reads a certain amount of reads information from the pipeline, copies it to the thread local cache, and then releases the read permission to wake up the thread that failed in this round of competition;
  • comparison thread completes the comparison task, it will be added to the list of threads competing for pipeline reading permissions in the next round;
  • the Reads data is copied multiple times, from the decompression cache to the pipeline, and from the pipeline to the thread-local cache pool; only one thread competes successfully each time, and the rest of the threads all wait, resulting in low thread execution efficiency.
  • this embodiment analyzes the sequencing data through the following steps:
  • the IO thread is responsible for parsing the file, obtaining a buffer from the empty queue of the buffer pool, writing a certain number of reads records into the buffer, and then joining the full queue;
  • the comparison thread obtains the buffer area from the full queue, and adds the buffer area to the empty queue after completing the MMP search and MMP expansion;
  • the comparison thread consumes the buffer faster than the IO thread generates the buffer, the comparison thread will wait for the "full" buffer, and the performance bottleneck is determined by the IO parsing thread; if the comparison thread consumes the buffer slower than the IO thread generates the buffer, the IO thread will wait for the empty buffer, and the performance bottleneck is determined by the efficiency of the alignment algorithm.
  • the IO thread consumes the buffer slower than the IO thread generates the buffer, the IO thread will wait for the empty buffer, and the performance bottleneck is determined by the efficiency of the alignment algorithm.
  • only one reads information copy is required, which reduces the number of data copies, and multiple threads are executed in parallel, which improves the thread execution efficiency.
  • the size of the buffer can be adjusted. Actual tests show that a buffer can accommodate 120k (thousand) reads.
  • the subsequence search method further includes:
  • multiple search phases are configured for each sequencing data
  • a buffer is obtained from the full queue, and the starting position of each parameter of each read in the buffer is recorded according to the read format. All reads in the buffer are preprocessed, such as converting the bases in the sequence to 0, 1, 2, 3, and initializing the parameters of each read. A fixed number of reads are processed each time, and only one base is searched for each read, and the corresponding search task is updated.
  • Each read has multiple MMP search subprocesses, each search has a different starting position, and also involves complex logic, such as searching in two directions and selecting the starting position.
  • this embodiment designs a stage (search stage) structure: a stage represents a search transaction for one read. According to the search direction and starting position, a read can be configured with multiple stages. Each stage-(off, dir) identifies the starting position and direction of the MMP search.
  • a task structure is designed in this implementation; task-(sa1, sa2, off, len, dir, flg) identifies the starting position (off), search direction (dir), match length (len), The range of MMP is in the search interval SA[sa1,sa2]; the task uses FMIndex (an index structure used for sequence-based search, characterized by searching one character in reverse at a time) to search one base at a time. If the search is successful, (sa1,sa2,len) is updated; if the search fails, the task ends and an MMP is recorded.
  • FMIndex an index structure used for sequence-based search, characterized by searching one character in reverse at a time
  • step S103 may include:
  • step S103 further includes:
  • the next sequencing data of the gene sequence to be searched is obtained.
  • a read is reloaded from the buffer queue until all reads in the buffer are processed; as shown in Figure 5, the MMP search of a read involves the transfer between multiple stages, such as after stage-0 is completed, it switches to stage-1, and so on, until all stages of the read are completed, the MMP search of the current read is completed, that is, the next sequencing data of the gene sequence to be searched is obtained.
  • the next sequencing data of the gene sequence to be searched is obtained; specifically, not all stages will be executed, and a stage may be skipped. For example, if the MMP matching length in the search stage (such as stage-0) corresponding to the forward base sequence to be searched is equal to reads length, the search stage (such as stage-4) corresponding to the reverse base sequence to be searched can be skipped, that is, the next sequencing data of the gene sequence to be searched can be obtained.
  • the gene sequence subsequence search method provided in this embodiment obtains the sequencing data of the gene sequence to be searched in batches, and adopts a pre-fetch mechanism to obtain the base data of the search interval corresponding to each sequencing data on the reference gene sequence.
  • the CPU can process the calculation of other sequencing data while waiting for the pre-fetch result. That is, the CPU calculation time fills the waiting time for pre-fetching through batch processing, thereby improving the efficiency of gene search.
  • Figure 6 is a flow chart of the gene sequence comparison method in this embodiment. Specifically, as shown in Figure 6, the comparison method includes:
  • the gene sequence comparison method provided in this embodiment utilizes the above-mentioned subsequence search method, so that the CPU can process the calculation of other sequencing data while waiting for the pre-fetch results, that is, through batch processing, the CPU calculation time fills the waiting time for pre-fetching, thereby improving the efficiency of gene search.
  • Figure 7 is a schematic diagram of the structure of the subsequence search system of the gene sequence in this embodiment.
  • the subsequence search system includes:
  • the sequencing data acquisition module 1 is used to obtain a preset amount of sequencing data of the gene sequence to be searched; specifically, as shown in FIG2 , taking FASTQ as an example, every 4 lines of the file are a record (name, sequence, annotation, quality value), and the record is called reads.
  • the search data acquisition module 2 is used to sequentially acquire the base data of the search interval corresponding to each sequencing data on the reference gene sequence, and perform a search based on the acquired results; the search interval corresponding to each sequencing data is determined based on the base sequence to be searched of each sequencing data; specifically, in an optional implementation
  • the search data acquisition module 2 can obtain the base data of the search interval through the pre-fetch technology; pre-fetching is a technology used by the CPU, which extracts the data from the slower memory to the faster local cache (cache memory) before the instruction or data is actually needed, thereby improving the execution performance; if there is no software pre-fetching mechanism, due to the dependency between the data, most of the time is wasted waiting for the relevant data in the memory, and the computing unit is idle most of the time.
  • the pre-fetching mechanism allows the computing unit and the related access unit to maintain efficient operation and maximize the use of related hardware resources; in this embodiment, by batch processing sequencing data, the CPU can process the calculation of other reads while waiting for the pre-fetch results, that is, by batch processing, the CPU's calculation time fills the waiting time for pre-fetching, which is equivalent to hiding the CPU's waiting time for pre-fetching, thereby improving the efficiency of gene search.
  • the search data acquisition module 2 can adopt the software pre-fetching method.
  • the number of sequencing data of the gene sequence to be searched is determined based on the pre-fetching duration and calculation duration of the base data in the search interval. As shown in FIG3 , when the calculation of sequencing data R4 is completed, the CPU just receives the pre-fetching result of sequencing data R1, and can continue the search calculation based on the pre-fetching result of sequencing data R1. According to actual tests, this method can achieve the best performance when the number of batch read sequencing data is 8 to 16.
  • the subsequence determination module 3 is used to repeatedly call the search data acquisition module to obtain the base data of the search interval for the next search and search based on the acquisition result to obtain the target subsequence; the search interval for the next search is determined according to the next base to be searched in each sequencing data and the search interval of the previous search.
  • the sequencing data acquisition module 1 is specifically used to receive sequencing data from the IO thread using an empty queue of the buffer pool; the empty queue is composed of a buffer area that does not store data, and the buffer area enters the full queue of the buffer pool after storing data;
  • the sequencing data acquisition module 1 is specifically used to transmit the sequencing data to the comparison thread using the full queue for subsequence search; after the comparison thread completes the subsequence search, the data stored in the buffer area is released.
  • sequencing data can be analyzed by the following methods:
  • the comparison phase is multi-threaded. Multiple threads compete for pipeline reading permissions. Threads that fail in the competition need to wait.
  • the thread that successfully competes reads a certain amount of reads information from the pipeline, copies it to the thread local cache, and then releases the read permission to wake up the thread that failed in this round of competition;
  • comparison thread completes the comparison task, it will be added to the list of threads competing for pipeline reading permissions in the next round;
  • the Reads data is copied multiple times, from the decompression cache to the pipeline, and from the pipeline to the thread-local cache pool; only one thread competes successfully each time, and the rest of the threads all wait, resulting in low thread execution efficiency.
  • the sequencing data acquisition module 1 of this embodiment specifically analyzes the sequencing data through the following steps:
  • the IO thread is responsible for parsing the file, obtaining a buffer from the empty queue of the buffer pool, writing a certain number of reads records into the buffer, and then joining the full queue;
  • the comparison thread obtains the buffer area from the full queue, and adds the buffer area to the empty queue after completing the MMP search and MMP expansion;
  • the comparison thread consumes the buffer faster than the IO thread generates the buffer, the comparison thread will wait for the "full" buffer, and the performance bottleneck is determined by the IO parsing thread; if the comparison thread consumes the buffer slower than the IO thread generates the buffer, the IO thread will wait for the empty buffer, and the performance bottleneck is determined by the efficiency of the alignment algorithm.
  • the efficiency of the alignment algorithm In this embodiment, only one reads information copy is required, which reduces the number of data copies, and multiple threads are executed in parallel, which improves the thread execution efficiency.
  • the size of the buffer can be adjusted. Actual tests show that a buffer can accommodate 120k reads.
  • the subsequence search system further includes:
  • a search phase configuration module 4 configured to configure multiple search phases for each sequencing data according to the search direction and starting position corresponding to each sequencing data
  • a buffer is obtained from the full queue, and the starting position of each parameter of each read in the buffer is recorded according to the read format. All reads in the buffer are preprocessed, such as converting the bases in the sequence to 0, 1, 2, 3, and initializing the parameters of each read. A fixed number of reads are processed each time, and only one base is searched for each read, and the corresponding search task is updated.
  • Each read has multiple MMP search subprocesses, each search has a different starting position, and also involves complex logic, such as searching in two directions and selecting the starting position.
  • this embodiment designs a stage (search stage) structure: a stage represents a search transaction for one read. According to the search direction and starting position, a read can be configured with multiple stages. Each stage-(off, dir) identifies the starting position and direction of the MMP search.
  • a task For each stage, there will be multiple MMP searches.
  • a task (search task) structure is designed in this implementation; task-(sa1, sa2, off, len, dir, flg) identifies the starting position (off), search direction (dir), match length (len) of the current MMP search, and the range of MMP is in the search interval SA[sa1, sa2]; task uses FMIndex (an index structure used for sequence-based search, characterized by searching one character in reverse each time) to search one base at a time. If the search is successful, (sa1, sa2, len) is updated. If the search fails, the task ends and an MMP is recorded.
  • FMIndex an index structure used for sequence-based search, characterized by searching one character in reverse each time
  • the subsequence determination module 3 is specifically used to call the search data acquisition module 2 to restart the pre-fetching of base data in the search interval in the search phase with the end position of the previous search as the starting position when there is no corresponding search interval in the next search. Specifically, if a task is completed and the remaining matching bases are sufficient, a new task is started at the non-matching position; each newly created task will be initialized by the pre-index structure, the purpose of which is to shorten the search interval and improve the search efficiency of FMIndex.
  • the subsequence determination module 3 is specifically used to call the search data acquisition module 2 in the next search phase when the base data of the search interval corresponding to the base sequence to be searched in the current search phase are all pre-fetched. Restart pre-fetching of base data in the search interval within the stage; specifically, when a task structure ends, it is necessary to start a new task at the unmatched position, or start a new task after the stage transfer; if there are too few remaining matching bases after a task ends, switch directly to the next stage.
  • the subsequence determination module 3 is specifically used to call the sequencing data acquisition module 1 to obtain the next sequencing data of the gene sequence to be searched when the base data of the search interval corresponding to the forward base sequence to be searched of the current sequencing data in the current search stage are all pre-fetched; specifically, not all stages will be executed, and a stage may be skipped. For example, if the MMP matching length in the search stage (such as stage-0) corresponding to the forward base sequence to be searched is equal to the length of reads, the search stage (such as stage-4) corresponding to the reverse base sequence to be searched can be skipped, that is, the next sequencing data of the gene sequence to be searched can be obtained.
  • the subsequence determination module 3 is specifically used to call the sequencing data acquisition module 1 to obtain the next sequencing data of the gene sequence to be searched when the base data of the search interval of all search stages are pre-fetched. Specifically, when all stages of a read are executed, a read is reloaded from the buffer queue until all reads in the buffer are processed; as shown in Figure 5, the MMP search of a read involves the transfer between multiple stages, such as after stage-0 is completed, it switches to stage-1, and so on, until all stages of the read are completed, the MMP search of the current read is completed, that is, the next sequencing data of the gene sequence to be searched is obtained.
  • the gene sequence subsequence search system obtaineds the sequencing data of the gene sequence to be searched in batches, and adopts a pre-fetch mechanism to obtain the base data of the search interval corresponding to each sequencing data on the reference gene sequence.
  • the CPU can process the calculation of other sequencing data while waiting for the pre-fetch result. That is, the CPU calculation time makes up for the waiting time for pre-fetching through batch processing, thereby improving the efficiency of gene search.
  • Figure 8 is a schematic diagram of the structure of the gene sequence comparison system in this embodiment. Specifically, as shown in Figure 8, the comparison system includes:
  • Gene sequence acquisition module 5 used to acquire the gene sequence to be compared
  • a target sequence determination module 6 is used to obtain a target subsequence of the gene sequence to be compared using the gene sequence subsequence search system of Example 3;
  • the target sequence expansion module 7 is used to expand the target subsequence to obtain the comparison result of the gene sequence to be compared.
  • the gene sequence comparison system utilizes the above-mentioned subsequence search system so that the CPU can process the calculation of other sequencing data while waiting for pre-fetch results, that is, through batch processing, the CPU calculation time fills the waiting time for pre-fetching, thereby improving the efficiency of gene search.
  • FIG9 is a schematic diagram of the structure of an electronic device provided in Example 5 of the present invention.
  • the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the processor executes the program, the subsequence search method of the gene sequence in Example 1 or the gene sequence comparison method in Example 2 is implemented.
  • the electronic device 30 shown in FIG9 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
  • the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device.
  • the components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).
  • the bus 33 includes a data bus, an address bus, and a control bus.
  • the memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and/or a cache memory 322 , and may further include a read-only memory (ROM) 323 .
  • RAM random access memory
  • ROM read-only memory
  • the memory 32 may also include a program/utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
  • program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
  • the processor 31 executes various functions by running the computer program stored in the memory 32. It can be applied and used for data processing, such as the subsequence search method of the gene sequence of Example 1 of the present invention or the gene sequence comparison method of Example 2.
  • the electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input/output (I/O) interface 35.
  • the model-generated device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) via a network adapter 36.
  • the network adapter 36 communicates with other modules of the model-generated device 30 via a bus 33.
  • model-generated device 30 may be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.
  • This embodiment provides a computer-readable storage medium on which a computer program is stored.
  • the program is executed by a processor, the subsequence search method of the gene sequence in embodiment 1 or the gene sequence comparison method in embodiment 2 is implemented.
  • the readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.
  • the present invention can also be implemented in the form of a program product, which includes a program code.
  • the program product When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the subsequence search method of the gene sequence of Example 1 or the gene sequence comparison method of Example 2.
  • the program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Proteomics, Peptides & Aminoacids (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biotechnology (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Chemical & Material Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)

Abstract

本发明公开了一种基因序列的子序列搜索方法、比对方法、系统及设备，所述子序列搜索方法包括：获取待搜索基因序列的预设数量的测序数据；依次获取参考基因序列上每一所述测序数据对应的搜索区间的碱基数据，并基于获取结果进行搜索；每一所述测序数据对应的搜索区间基于每一所述测序数据的待搜索碱基序列确定；重复获取下一次搜索的搜索区间的碱基数据并基于获取结果进行搜索以得到目标子序列；所述下一次搜索的搜索区间根据每一所述测序数据下一个待搜索碱和上一次搜索的搜索区间确定。本发明的搜索方法，预先获取搜索区间的碱基数据，通过批量处理测序数据，在等待预取结果时可以处理其他测序数据的计算，提高了基因搜索的效率。

Description

基因序列的子序列搜索方法、比对方法、系统及设备 技术领域
本发明涉及核酸信息领域，特别涉及一种基因序列的子序列搜索方法、比对方法、系统及设备。
背景技术
基因序列是包含ACGT四种碱基的长字符串，基因序列比对的一种方式是将短基因序列与参考基因序列对比，找出短基因序列在参考基因序列中的位置及两者间差异。现有的比对软件在处理基因序列时存在延迟高，搜索效率低，性能不够好的问题，不能满足基因序列的处理需求。
发明内容
本发明要解决的技术问题是为了克服现有技术中基因搜索算法的搜索效率低的缺陷，提供一种基因序列的子序列搜索方法、比对方法、系统及设备。
本发明是通过下述技术方案来解决上述技术问题：
本发明提供一种基因序列的子序列搜索方法，所述子序列搜索方法包括：
获取待搜索基因序列的预设数量的测序数据；
依次获取参考基因序列上每一所述测序数据对应的搜索区间的碱基数据，并基于获取结果进行搜索；每一所述测序数据对应的搜索区间基于每一所述测序数据的待搜索碱基序列确定；
重复获取下一次搜索的搜索区间的碱基数据并基于获取结果进行搜索以得到目标子序列；所述下一次搜索的搜索区间根据每一所述测序数据下一个待搜索碱基和上一次搜索的搜索区间确定。
较佳地，所述待搜索基因序列的测序数据的数量基于所述搜索区间的碱 基数据的预取时长和计算时长确定。
较佳地，在依次获取参考基因序列上每一所述测序数据对应的搜索区间的碱基数据的步骤之前，所述子序列搜索方法还包括：
根据每一所述测序数据对应的搜索方向和起始位置，为每一所述测序数据配置多个搜索阶段；
重复获取下一次搜索的搜索区间的碱基数据的步骤包括：
当下一次搜索不存在对应的搜索区间时，以上一次搜索的结束位置为起始位置在所述搜索阶段内重新开始搜索区间的碱基数据的预取。
较佳地，在以上一次搜索的结束位置为起始位置在所述搜索阶段内重新开始搜索区间的碱基数据的预取的步骤之后，所述重复获取下一次搜索的搜索区间的碱基数据的步骤还包括：
当当前搜索阶段的待搜索碱基序列对应的搜索区间的碱基数据均完成预取，在下一搜索阶段内重新开始搜索区间的碱基数据的预取；和/或，
当在当前搜索阶段中当前测序数据的正向待搜索碱基序列对应的搜索区间的碱基数据均完成预取，获取待搜索基因序列的下一个测序数据；和/或，
当所有搜索阶段的搜索区间的碱基数据均完成预取，获取待搜索基因序列的下一个测序数据。
较佳地，获取待搜索基因序列的预设数量的测序数据的步骤包括：
利用缓存池的空队列从IO(输入输出)线程中接收所述测序数据；所述空队列由未存储数据的缓存区排列而成，缓存区存储数据后进入缓存池的满队列；
利用所述满队列将所述测序数据传输到比对线程以进行子序列搜索；所述比对线程完成子序列搜索后释放缓存区存储的数据。
本发明还提供一种基因序列的比对方法，所述比对方法包括：
获取待比对基因序列；
利用如上所述的基因序列的子序列搜索方法得到所述待比对基因序列 的目标子序列；
扩展所述目标子序列以得到所述待比对基因序列的比对结果。
本发明还提供一种基因序列的子序列搜索系统，所述子序列搜索系统包括：
测序数据获取模块，用于获取待搜索基因序列的预设数量的测序数据；
搜索数据获取模块，用于依次获取参考基因序列上每一所述测序数据对应的搜索区间的碱基数据，并基于获取结果进行搜索；每一所述测序数据对应的搜索区间基于每一所述测序数据的待搜索碱基序列确定；
子序列确定模块，用于重复调用所述搜索数据获取模块以获取下一次搜索的搜索区间的碱基数据并基于获取结果进行搜索以得到目标子序列；所述下一次搜索的搜索区间根据每一所述测序数据下一个待搜索碱基和上一次搜索的搜索区间确定。
较佳地，所述待搜索基因序列的测序数据的数量基于所述搜索区间的碱基数据的预取时长和计算时长确定。
较佳地，所述子序列搜索系统还包括：
搜索阶段配置模块，用于根据每一所述测序数据对应的搜索方向和起始位置，为每一所述测序数据配置多个搜索阶段；
所述子序列确定模块具体用于当下一次搜索不存在对应的搜索区间时，调用所述搜索数据获取模块以上一次搜索的结束位置为起始位置在所述搜索阶段内重新开始搜索区间的碱基数据的预取。
较佳地，所述子序列确定模块具体用于当当前搜索阶段的待搜索碱基序列对应的搜索区间的碱基数据均完成预取，调用所述搜索数据获取模块在下一搜索阶段内重新开始搜索区间的碱基数据的预取；和/或，
所述子序列确定模块具体用于当在当前搜索阶段中当前测序数据的正向待搜索碱基序列对应的搜索区间的碱基数据均完成预取，调用所述测序数据获取模块获取待搜索基因序列的下一个测序数据；和/或，
所述子序列确定模块具体用于当所有搜索阶段的搜索区间的碱基数据均完成预取，调用所述测序数据获取模块获取待搜索基因序列的下一个测序数据。
较佳地，所述测序数据获取模块具体用于利用缓存池的空队列从IO线程中接收所述测序数据；所述空队列由未存储数据的缓存区排列而成，缓存区存储数据后进入缓存池的满队列；
所述测序数据获取模块具体用于利用所述满队列将所述测序数据传输到比对线程以进行子序列搜索；所述比对线程完成子序列搜索后释放缓存区存储的数据。
本发明还提供一种基因序列的比对系统，所述比对系统包括：
基因序列获取模块，用于获取待比对的基因序列；
目标序列确定模块，用于利用如上所述的基因序列的子序列搜索系统得到所述待比对的基因序列的目标子序列；
目标序列扩展模块，用于扩展所述目标子序列以得到所述待比对基因序列的比对结果。
本发明还提供一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如上所述的基因序列的子序列搜索方法或如上所述的基因序列的比对方法。
本发明还提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述的基因序列的子序列搜索方法或如上所述的基因序列的比对方法。
本发明的积极进步效果在于：
本发明提供的基因序列的子序列搜索方法，通过批量获取待搜索基因序列的测序数据，并预先提取参考基因序列上每一测序数据对应的搜索区间的碱基数据，通过批量处理测序数据，使得CPU在等待预取结果时可以处理其他测序数据的计算，即通过批量处理使得CPU的计算时间填补了等待预 取的时间，提高了基因搜索的效率。
附图说明
为了更清楚地说明本说明书实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单的介绍。显而易见地，下面描述中的附图仅仅是本说明书的一些示例或实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图将本说明书应用于其它类似情景。
图1为本发明实施例1中的基因序列的子序列搜索方法的流程图。
图2为本发明实施例1中的FASTQ格式的测序数据的示例图。
图3为本发明实施例1中的确定待搜索基因序列的测序数据的数量的示例图。
图4为本发明实施例1中的解析测序数据的示例图。
图5为本发明实施例1中的设置搜索阶段的示意图。
图6为本发明实施例2中的基因序列的比对方法的流程图。
图7为本发明实施例3中的基因序列的子序列搜索系统的结构示意图。
图8为本发明实施例4中的基因序列的比对系统的结构示意图。
图9为本发明实施例5中的电子设备的结构示意图。
具体实施方式
下面通过实施例的方式进一步说明本发明，但并不因此将本发明限制在所述的实施例范围之中。
在本文提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在文中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
应当理解，本文使用的“系统”、“装置”、“单元”和/或“模组”是用于区分不同级别的不同组件、元件、部件、部分或装配的一种方法。然而，如果其他词语可实现相同的目的，则可通过其他表达来替换所述词语。
如本文中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其它的步骤或元素。
本文中的包括的定义，如这里所使用的术语“具有”、“可以具有”、“包括”或“可以包括”指示本文的相应功能、操作、元件等的存在，并且不限制其它的一个或多个功能、操作、元件等的存在。此外应当理解到，如这里所使用的术语“包括”或“具有”是指示在说明书中所描述的特点、数字、步骤、操作、元件、部件或其组合的存在，而不排除一个或多个其它特点、数字、步骤、操作、元件、部件或其组合的存在或增加。
本文中和/或的定义，如这里所使用的术语“A或B”、“A和/或B的至少之一”或“A和/或B的一个或多个”包括与其一起列举的单词的任意和所有组合。例如，“A或B”、“A和B的至少之一”或“A或B的至少之一”意味着(1)包括至少一个A，(2)包括至少一个B，或(3)包括至少一个A和至少一个B两者。
本文中使用了流程图用来说明根据本文的实施例的系统所执行的操作。应当理解的是，前面或后面操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各个步骤。同时，也可以将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。
实施例1
比对软件的核心算法主要分为两个部分：Seed搜索和基于Seed进行扩展；其中Seed指的是一条测序碱基序列的子序列。Seed的搜索效率对比对效率影响很大，所以一般的比对软件在Seed阶段采用后缀数组(SA，suffix  array)、哈希表(HashTable)等索引结构提升Seed搜索效率。
以基于SA的二分搜索算法为例，每次搜索需要访问参考序列的不同位置，内存访问空间局部性差，内存延迟高，且搜索MMP(Maximal Mappable Prefix，最大可映射前缀)结束后需要再次在SA数组中确认匹配区间，导致搜索效率低，性能不够好。针对比对软件的核心算法存访问空间局部性差，内存延迟高的缺陷，本实施例提出一种基因序列的子序列搜索方法。
请参考图1，其为本实施例中的基因序列的子序列搜索方法的流程图。具体的，如图1所示，所述子序列搜索方法包括：
S101、获取待搜索基因序列的预设数量的测序数据；具体地，如图2所示，以FASTQ(一种测序数据的文件格式)为例，该文件每4行为一个记录(名称、序列、注释、质量值)，该记录称为reads。
S102、依次获取参考基因序列上每一测序数据对应的搜索区间的碱基数据，并基于获取结果进行搜索；每一测序数据对应的搜索区间基于每一测序数据的待搜索碱基序列确定；具体地，在一种可选的实施方式中，可以通过预取技术获取搜索区间的碱基数据；预取是CPU(Central Processing Unit，中央处理器)使用的一种技术，在指令或数据实际需要之前，将从其在较慢内存中提取到较快的本地Cache(高速缓冲存储器)中，从而提高执行性能；如果没有预取机制，由于数据之间存在依赖性，大部分时间都浪费在等待内存的相关数据，计算单元大部分时间处于空闲状态，采用预取机制使得计算单元和相关存取单元可以保持高效地运作，最大化地利用相关的硬件资源；本实施例中，通过批量处理测序数据，使得CPU在等待预取结果时可以处理其他reads的计算，即通过批量处理使得CPU的计算时间填补了等待预取的时间，相当于隐藏了CPU等待预取的时间，提高了基因搜索的效率。优选地，采用软件预取的方式。
在一种可选的实施方式中，待搜索基因序列的测序数据的数量基于搜索区间的碱基数据的预取时长和计算时长确定。如图3所示，当测序数据R4 的计算结束时，CPU正好接收测序数据R1的预取结果，可以继续基于测序数据R1的预取结果的搜索计算。根据实际测试，当批量读取测序数据的数量为8～16时本方法可以获得最佳性能。
S103、重复获取下一次搜索的搜索区间的碱基数据并基于获取结果进行搜索以得到目标子序列；下一次搜索的搜索区间根据每一测序数据下一个待搜索碱基和上一次搜索的搜索区间确定。
以下通过举例对上述方法的步骤进行具体说明。在一种可选的实施方式中，步骤S101包括：
S1011、利用缓存池的空队列从IO线程中接收测序数据；空队列由未存储数据的缓存区排列而成，缓存区存储数据后进入缓存池的满队列；
S1012、利用满队列将测序数据传输到比对线程以进行子序列搜索；比对线程完成子序列搜索后释放缓存区存储的数据。
在一个方案中可以通过以下步骤对测序数据的解析：
1、解压FASTQ文件，将reads数据写入管道；
2、比对阶段是多线程，多个线程竞争管道读取权限，竞争失败的线程需要等待；
3、竞争成功的线程从管道读取一定数量的reads信息，将其拷贝至线程本地缓存，然后释放读取权限，唤醒本轮竞争失败的线程；
4、本轮竞争失败的线程将再次竞争管道，竞争失败的线程将继续等待；
5、若比对线程完成比对任务，则将加入下一轮竞争管道读取权限的线程列表；
6、当FASTQ文件处理完毕，比对线程结束。
在这个对测序数据的解析方法中，Reads数据存在多次拷贝，从解压缓存拷贝至管道，从管道拷贝至线程本地缓存池；每次只有一个线程竞争成功，其余线程全部等待，线程执行效率低。
如图4所示，本实施方式通过以下步骤对测序数据的解析：
1、IO线程负责解析文件，从缓存池的空队列中获取一个缓存区，将一定数量的reads记录写入该缓存区，然后加入满队列；
2、比对线程从满队列中获取缓存区，完成MMP搜索、MMP扩展后将缓存区加入空队列；
可见，如果比对线程消耗缓存区的速度比IO线程产生缓存区的速度快，会导致比对线程等待“满”缓存区，性能瓶颈由IO解析线程决定；如果比对线程消耗缓存区的速度低于IO线程产生缓存区的速度，则会导致IO线程等待空缓存区，这时，性能瓶颈是由对齐算法的效率决定的。在本实施方式中只需要一次reads信息拷贝，减少了数据拷贝次数，多个线程并行执行，提高了线程执行效率。缓冲区的大小可以调整，实际测试表明，一个缓冲区可以容纳120k(千)条reads。
在一种可选的实施方式中，在步骤S102之前，子序列搜索方法还包括：
根据每一测序数据对应的搜索方向和起始位置，为每一测序数据配置多个搜索阶段；
具体地，从满队列中获取一个缓存区，根据reads格式解析，记录缓存区中每次reads的各参数的起始位置。对缓冲区中的所有reads进行预处理，例如将序列中的碱基转换为0、1、2、3，并初始化每个reads的参数。每次处理固定数量的reads，每次reads只搜索一个碱基，并更新对应的搜索任务。
每条reads有多个MMP搜索子过程，每次搜索的起始位置不同，并且还涉及复杂的逻辑，例如两个方向的搜索和起始位置的选择。为了简化处理流程，本实施例设计了一个stage(搜索阶段)结构：一个stage代表一个reads的搜索事务。根据搜索方向和起始位置，一条reads可以配置多个阶段。每个stage-(off,dir)标识MMP搜索的起始位置和方向。
对于每个阶段，都会有多个MMP搜索。为了识别每个MMP搜索状态，本实施方式中设计了一个task(搜索任务)结构；task-(sa1,sa2,off,len,dir,flg)标识当前MMP搜索的起始位置(off)、搜索方向(dir)、匹配长度(len)、 MMP的范围在搜索区间SA[sa1,sa2]上；task使用FMIndex(一种索引结构，应用于基于序列搜索，特点是每次反向搜索一个字符)一次搜索一个碱基。如果搜索成功，则更新(sa1,sa2,len)，如果搜索失败，则任务结束并记录一个MMP。
在本实施方式中，步骤S103可以包括：
S1031、当下一次搜索不存在对应的搜索区间时，以上一次搜索的结束位置为起始位置在搜索阶段内重新开始搜索区间的碱基数据的预取。具体地，如果一个task结束，剩下的匹配碱基足够了，在不匹配的位置开始一个新的task；每一个新创建的task会被pre-index结构进行初始化，目的在于缩减搜索区间提升FMIndex的搜索效率。
此外，在步骤S1031之后，步骤S103还包括：
S1032、当当前搜索阶段的待搜索碱基序列对应的搜索区间的碱基数据均完成预取，在下一搜索阶段内重新开始搜索区间的碱基数据的预取；具体地，当一个task结构结束时，需要在不匹配的位置开始一个新的task，或者在stage转移后开始一个新的task；如果一个task结束后剩余的匹配碱基太少，直接切换到下一个stage。
S1033、当所有搜索阶段的搜索区间的碱基数据均完成预取，获取待搜索基因序列的下一个测序数据。具体地，当一个reads的所有stage都执行完毕后，从缓冲区队列中重新加载一条reads，直到缓冲区中的所有reads处理完；如图5所示，一条reads的MMP搜索涉及到多个stage之间的转移，如stage-0完成后，切换到stage-1，以此类推，直到该reads的所有stage完成，当前reads的MMP搜索完成，即获取待搜索基因序列的下一个测序数据。
S1034、当在当前搜索阶段中当前测序数据的正向待搜索碱基序列对应的搜索区间的碱基数据均完成预取，获取待搜索基因序列的下一个测序数据；具体地，并非所有阶段都会执行，可能会跳过一个阶段。例如，如果正向待搜索碱基序列对应的搜索阶段(如stage-0)中的MMP匹配长度等于reads 的长度，则可以跳过反向待搜索碱基序列对应的搜索阶段(如stage-4)，即获取待搜索基因序列的下一个测序数据。
本实施例提供的基因序列的子序列搜索方法，通过批量获取待搜索基因序列的测序数据，并采用预取机制获取参考基因序列上每一测序数据对应的搜索区间的碱基数据，通过批量处理测序数据，使得CPU在等待预取结果时可以处理其他测序数据的计算，即通过批量处理使得CPU的计算时间填补了等待预取的时间，提高了基因搜索的效率。
实施例2
请参考图6，其为本实施例中的基因序列的比对方法的流程图。具体的，如图6所示，所述比对方法包括：
S201、获取待比对基因序列；
S202、利用实施例1的基因序列的子序列搜索方法得到待比对基因序列的目标子序列；
S203、扩展目标子序列以得到待比对基因序列的比对结果。
本实施例提供的基因序列的比对方法，通过利用上述子序列搜索方法，使得CPU在等待预取结果时可以处理其他测序数据的计算，即通过批量处理使得CPU的计算时间填补了等待预取的时间，提高了基因搜索的效率。
实施例3
请参考图7，其为本实施例中的基因序列的子序列搜索系统的结构示意图。具体的，如图7所示，所述子序列搜索系统包括：
测序数据获取模块1，用于获取待搜索基因序列的预设数量的测序数据；具体地，如图2所示，以FASTQ为例，该文件每4行为一个记录(名称、序列、注释、质量值)，该记录称为reads。
搜索数据获取模块2，用于依次获取参考基因序列上每一测序数据对应的搜索区间的碱基数据，并基于获取结果进行搜索；每一测序数据对应的搜索区间基于每一测序数据的待搜索碱基序列确定；具体地，在一种可选的实 施方式中，搜索数据获取模块2可以通过预取技术获取搜索区间的碱基数据；预取是CPU使用的一种技术，在指令或数据实际需要之前，将从其在较慢内存中提取到较快的本地Cache(高速缓冲存储器)中，从而提高执行性能；如果没有软件预取机制，由于数据之间存在依赖性，大部分时间都浪费在等待内存的相关数据，计算单元大部分时间处于空闲状态，采用预取机制使得计算单元和相关存取单元可以保持高效地运作，最大化地利用相关的硬件资源；本实施例中，通过批量处理测序数据，使得CPU在等待预取结果时可以处理其他reads的计算，即通过批量处理使得CPU的计算时间填补了等待预取的时间，相当于隐藏了CPU等待预取的时间，提高了基因搜索的效率。优选地，搜索数据获取模块2可以采用软件预取的方式。
在一种可选的实施方式中，待搜索基因序列的测序数据的数量基于搜索区间的碱基数据的预取时长和计算时长确定。如图3所示，当测序数据R4的计算结束时，CPU正好接收测序数据R1的预取结果，可以继续基于测序数据R1的预取结果的搜索计算。根据实际测试，当批量读取测序数据的数量为8～16时本方法可以获得最佳性能。
子序列确定模块3，用于重复调用搜索数据获取模块以获取下一次搜索的搜索区间的碱基数据并基于获取结果进行搜索以得到目标子序列；下一次搜索的搜索区间根据每一测序数据下一个待搜索碱基和上一次搜索的搜索区间确定。
以下通过举例对上述方法的步骤进行具体说明。在一种可选的实施方式中，测序数据获取模块1具体用于利用缓存池的空队列从IO线程中接收测序数据；空队列由未存储数据的缓存区排列而成，缓存区存储数据后进入缓存池的满队列；
测序数据获取模块1具体用于利用满队列将测序数据传输到比对线程以进行子序列搜索；比对线程完成子序列搜索后释放缓存区存储的数据。
在一个方案中可以通过以下方法对测序数据的解析：
1、解压FASTQ文件，将reads数据写入管道；
2、比对阶段是多线程，多个线程竞争管道读取权限，竞争失败的线程需要等待；
3、竞争成功的线程从管道读取一定数量的reads信息，将其拷贝至线程本地缓存，然后释放读取权限，唤醒本轮竞争失败的线程；
4、本轮竞争失败的线程将再次竞争管道，竞争失败的线程将继续等待；
5、若比对线程完成比对任务，则将加入下一轮竞争管道读取权限的线程列表；
6、当FASTQ文件处理完毕，比对线程结束。
在这个对测序数据的解析方法中，Reads数据存在多次拷贝，从解压缓存拷贝至管道，从管道拷贝至线程本地缓存池；每次只有一个线程竞争成功，其余线程全部等待，线程执行效率低。
如图4所示，本实施方式的测序数据获取模块1具体通过以下步骤对测序数据的解析：
1、IO线程负责解析文件，从缓存池的空队列中获取一个缓存区，将一定数量的reads记录写入该缓存区，然后加入满队列；
2、比对线程从满队列中获取缓存区，完成MMP搜索、MMP扩展后将缓存区加入空队列；
可见，如果比对线程消耗缓存区的速度比IO线程产生缓存区的速度快，会导致比对线程等待“满”缓存区，性能瓶颈由IO解析线程决定；如果比对线程消耗缓存区的速度低于IO线程产生缓存区的速度，则会导致IO线程等待空缓存区，这时，性能瓶颈是由对齐算法的效率决定的。在本实施方式中只需要一次reads信息拷贝，减少了数据拷贝次数，多个线程并行执行，提高了线程执行效率。缓冲区的大小可以调整，实际测试表明，一个缓冲区可以容纳120k条reads。
在一种可选的实施方式中，子序列搜索系统还包括：
搜索阶段配置模块4，用于根据每一测序数据对应的搜索方向和起始位置，为每一测序数据配置多个搜索阶段；
具体地，从满队列中获取一个缓存区，根据reads格式解析，记录缓存区中每次reads的各参数的起始位置。对缓冲区中的所有reads进行预处理，例如将序列中的碱基转换为0、1、2、3，并初始化每个reads的参数。每次处理固定数量的reads，每次reads只搜索一个碱基，并更新对应的搜索任务。
每条reads有多个MMP搜索子过程，每次搜索的起始位置不同，并且还涉及复杂的逻辑，例如两个方向的搜索和起始位置的选择。为了简化处理流程，本实施例设计了一个stage(搜索阶段)结构：一个stage代表一个reads的搜索事务。根据搜索方向和起始位置，一条reads可以配置多个阶段。每个stage-(off,dir)标识MMP搜索的起始位置和方向。
对于每个阶段，都会有多个MMP搜索。为了识别每个MMP搜索状态，本实施方式中设计了一个task(搜索任务)结构；task-(sa1,sa2,off,len,dir,flg)标识当前MMP搜索的起始位置(off)、搜索方向(dir)、匹配长度(len)、MMP的范围在搜索区间SA[sa1,sa2]上；task使用FMIndex(一种索引结构，应用于基于序列搜索，特点是每次反向搜索一个字符)一次搜索一个碱基。如果搜索成功，则更新(sa1,sa2,len)，如果搜索失败，则任务结束并记录一个MMP。
在本实施方式中，子序列确定模块3具体用于当下一次搜索不存在对应的搜索区间时，调用搜索数据获取模块2以上一次搜索的结束位置为起始位置在搜索阶段内重新开始搜索区间的碱基数据的预取。具体地，如果一个task结束，剩下的匹配碱基足够了，在不匹配的位置开始一个新的task；每一个新创建的task会被pre-index结构进行初始化，目的在于缩减搜索区间提升FMIndex的搜索效率。
此外，子序列确定模块3具体用于当当前搜索阶段的待搜索碱基序列对应的搜索区间的碱基数据均完成预取，调用搜索数据获取模块2在下一搜索 阶段内重新开始搜索区间的碱基数据的预取；具体地，当一个task结构结束时，需要在不匹配的位置开始一个新的task，或者在stage转移后开始一个新的task；如果一个task结束后剩余的匹配碱基太少，直接切换到下一个stage。
子序列确定模块3具体用于当在当前搜索阶段中当前测序数据的正向待搜索碱基序列对应的搜索区间的碱基数据均完成预取，调用测序数据获取模块1获取待搜索基因序列的下一个测序数据；具体地，并非所有阶段都会执行，可能会跳过一个阶段。例如，如果正向待搜索碱基序列对应的搜索阶段(如stage-0)中的MMP匹配长度等于reads的长度，则可以跳过反向待搜索碱基序列对应的搜索阶段(如stage-4)，即获取待搜索基因序列的下一个测序数据。
子序列确定模块3具体用于当所有搜索阶段的搜索区间的碱基数据均完成预取，调用测序数据获取模块1获取待搜索基因序列的下一个测序数据。具体地，当一个reads的所有stage都执行完毕后，从缓冲区队列中重新加载一条reads，直到缓冲区中的所有reads处理完；如图5所示，一条reads的MMP搜索涉及到多个stage之间的转移，如stage-0完成后，切换到stage-1，以此类推，直到该reads的所有stage完成，当前reads的MMP搜索完成，即获取待搜索基因序列的下一个测序数据。
本实施例提供的基因序列的子序列搜索系统，通过批量获取待搜索基因序列的测序数据，并采用预取机制获取参考基因序列上每一测序数据对应的搜索区间的碱基数据，通过批量处理测序数据，使得CPU在等待预取结果时可以处理其他测序数据的计算，即通过批量处理使得CPU的计算时间填补了等待预取的时间，提高了基因搜索的效率。
实施例4
请参考图8，其为本实施例中的基因序列的比对系统的结构示意图。具体的，如图8所示，所述比对系统包括：
基因序列获取模块5，用于获取待比对的基因序列；
目标序列确定模块6，用于利用实施例3的基因序列的子序列搜索系统得到待比对的基因序列的目标子序列；
目标序列扩展模块7，用于扩展目标子序列以得到待比对基因序列的比对结果。
本实施例提供的基因序列的比对系统，通过利用上述子序列搜索系统，使得CPU在等待预取结果时可以处理其他测序数据的计算，即通过批量处理使得CPU的计算时间填补了等待预取的时间，提高了基因搜索的效率。
实施例5
图9为本发明实施例5提供的一种电子设备的结构示意图。所述电子设备包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现实施例1的基因序列的子序列搜索方法或实施例2的基因序列的比对方法。图9显示的电子设备30仅仅是一个示例，不应对本发明实施例的功能和使用范围带来任何限制。
如图9所示，电子设备30可以以通用计算设备的形式表现，例如其可以为服务器设备。电子设备30的组件可以包括但不限于：上述至少一个处理器31、上述至少一个存储器32、连接不同系统组件(包括存储器32和处理器31)的总线33。
总线33包括数据总线、地址总线和控制总线。
存储器32可以包括易失性存储器，例如随机存取存储器(RAM)321和/或高速缓存存储器322，还可以进一步包括只读存储器(ROM)323。
存储器32还可以包括具有一组(至少一个)程序模块324的程序/实用工具325，这样的程序模块324包括但不限于：操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。
处理器31通过运行存储在存储器32中的计算机程序，从而执行各种功 能应用以及数据处理，例如本发明实施例1的基因序列的子序列搜索方法或实施例2的基因序列的比对方法。
电子设备30也可以与一个或多个外部设备34(例如键盘、指向设备等)通信。这种通信可以通过输入/输出(I/O)接口35进行。并且，模型生成的设备30还可以通过网络适配器36与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器36通过总线33与模型生成的设备30的其它模块通信。应当明白，尽管图中未示出，可以结合模型生成的设备30使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理器、外部磁盘驱动阵列、RAID(磁盘阵列)系统、磁带驱动器以及数据备份存储系统等。
应当注意，尽管在上文详细描述中提及了电子设备的若干单元/模块或子单元/模块，但是这种划分仅仅是示例性的并非强制性的。实际上，根据本发明的实施方式，上文描述的两个或更多单元/模块的特征和功能可以在一个单元/模块中具体化。反之，上文描述的一个单元/模块的特征和功能可以进一步划分为由多个单元/模块来具体化。
实施例4
本实施例提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现实施例1的基因序列的子序列搜索方法或实施例2的基因序列的比对方法。
其中，可读存储介质可以采用的更具体可以包括但不限于：便携式盘、硬盘、随机存取存储器、只读存储器、可擦拭可编程只读存储器、光存储器件、磁存储器件或上述的任意合适的组合。
在可能的实施方式中，本发明还可以实现为一种程序产品的形式，其包括程序代码，当所述程序产品在终端设备上运行时，所述程序代码用于使所述终端设备执行实现实施例1的基因序列的子序列搜索方法或实施例2的基因序列的比对方法。
其中，可以以一种或多种程序设计语言的任意组合来编写用于执行本发明的程序代码，所述程序代码可以完全地在用户设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户设备上部分在远程设备上执行或完全在远程设备上执行。
虽然以上描述了本发明的具体实施方式，但是本领域的技术人员应当理解，这仅是举例说明，本发明的保护范围是由所附权利要求书限定的。本领域的技术人员在不背离本发明的原理和实质的前提下，可以对这些实施方式做出多种变更或修改，但这些变更和修改均落入本发明的保护范围。

Claims (10)

  1. 一种基因序列的子序列搜索方法，其特征在于，所述子序列搜索方法包括：
    获取待搜索基因序列的预设数量的测序数据；
    依次获取参考基因序列上每一所述测序数据对应的搜索区间的碱基数据，并基于获取结果进行搜索；每一所述测序数据对应的搜索区间基于每一所述测序数据的待搜索碱基序列确定；
    重复获取下一次搜索的搜索区间的碱基数据并基于获取结果进行搜索以得到目标子序列；所述下一次搜索的搜索区间根据每一所述测序数据下一个待搜索碱基和上一次搜索的搜索区间确定。
  2. 如权利要求1所述的子序列搜索方法，其特征在于，所述待搜索基因序列的测序数据的数量基于所述搜索区间的碱基数据的预取时长和计算时长确定。
  3. 如权利要求1所述的子序列搜索方法，其特征在于，在依次获取参考基因序列上每一所述测序数据对应的搜索区间的碱基数据的步骤之前，所述子序列搜索方法还包括：
    根据每一所述测序数据对应的搜索方向和起始位置，为每一所述测序数据配置多个搜索阶段；
    重复获取下一次搜索的搜索区间的碱基数据的步骤包括：
    当下一次搜索不存在对应的搜索区间时，以上一次搜索的结束位置为起始位置在所述搜索阶段内重新开始搜索区间的碱基数据的预取。
  4. 如权利要求3所述的子序列搜索方法，其特征在于，在以上一次搜索的结束位置为起始位置在所述搜索阶段内重新开始搜索区间的碱基数据的预取的步骤之后，所述重复获取下一次搜索的搜索区间的碱基数据的步骤还包括：
    当当前搜索阶段的待搜索碱基序列对应的搜索区间的碱基数据均完成预取，在下一搜索阶段内重新开始搜索区间的碱基数据的预取；和/或，
    当在当前搜索阶段中当前测序数据的正向待搜索碱基序列对应的搜索区间的碱基数据均完成预取，获取待搜索基因序列的下一个测序数据；和/或，
    当所有搜索阶段的搜索区间的碱基数据均完成预取，获取待搜索基因序列的下一个测序数据。
  5. 如权利要求1所述的子序列搜索方法，其特征在于，获取待搜索基因序列的预设数量的测序数据的步骤包括：
    利用缓存池的空队列从IO线程中接收所述测序数据；所述空队列由未存储数据的缓存区排列而成，缓存区存储数据后进入缓存池的满队列；
    利用所述满队列将所述测序数据传输到比对线程以进行子序列搜索；所述比对线程完成子序列搜索后释放缓存区存储的数据。
  6. 一种基因序列的比对方法，其特征在于，所述比对方法包括：
    获取待比对基因序列；
    利用如权利要求1-5任一项所述的基因序列的子序列搜索方法得到所述待比对基因序列的目标子序列；
    扩展所述目标子序列以得到所述待比对基因序列的比对结果。
  7. 一种基因序列的子序列搜索系统，其特征在于，所述子序列搜索系统包括：
    测序数据获取模块，用于获取待搜索基因序列的预设数量的测序数据；
    搜索数据获取模块，用于依次获取参考基因序列上每一所述测序数据对应的搜索区间的碱基数据，并基于获取结果进行搜索；每一所述测序数据对应的搜索区间基于每一所述测序数据的待搜索碱基序列确定；
    子序列确定模块，用于重复调用所述搜索数据预取模块以获取下一次搜索的搜索区间的碱基数据并基于获取结果进行搜索以得到目标子序列；所述下一次搜索的搜索区间根据每一所述测序数据下一个待搜索碱基和上一次 搜索的搜索区间确定。
  8. 一种基因序列的比对系统，其特征在于，所述比对系统包括：
    基因序列获取模块，用于获取待比对的基因序列；
    目标序列确定模块，用于利用如权利要求7所述的基因序列的子序列搜索系统得到所述待比对的基因序列的目标子序列；
    目标序列扩展模块，用于扩展所述目标子序列以得到所述待比对基因序列的比对结果。
  9. 一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至5中任一项所述的基因序列的子序列搜索方法或权利要求6所述的基因序列的比对方法。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至5中任一项所述的基因序列的子序列搜索方法或权利要求6所述的基因序列的比对方法。
PCT/CN2023/101916 2023-06-21 2023-06-21 基因序列的子序列搜索方法、比对方法、系统及设备 Ceased WO2024259679A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2023/101916 WO2024259679A1 (zh) 2023-06-21 2023-06-21 基因序列的子序列搜索方法、比对方法、系统及设备
CN202380094923.5A CN120826741A (zh) 2023-06-21 2023-06-21 基因序列的子序列搜索方法、比对方法、系统及设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2023/101916 WO2024259679A1 (zh) 2023-06-21 2023-06-21 基因序列的子序列搜索方法、比对方法、系统及设备

Publications (1)

Publication Number Publication Date
WO2024259679A1 true WO2024259679A1 (zh) 2024-12-26

Family

ID=93934733

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/101916 Ceased WO2024259679A1 (zh) 2023-06-21 2023-06-21 基因序列的子序列搜索方法、比对方法、系统及设备

Country Status (2)

Country Link
CN (1) CN120826741A (zh)
WO (1) WO2024259679A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103793626A (zh) * 2012-10-29 2014-05-14 三星Sds株式会社 碱基序列比对系统及方法
US20170237445A1 (en) * 2014-08-05 2017-08-17 Illumina Cambridge Limited Methods and systems for data analysis and compression
CN107844684A (zh) * 2016-09-18 2018-03-27 深圳华大基因研究院 基因序列比对方法和装置
CN114121153A (zh) * 2021-11-23 2022-03-01 广州金域医学检验中心有限公司 基因突变位点检测方法、装置、电子设备及存储介质
WO2022041881A1 (zh) * 2020-08-24 2022-03-03 华为技术有限公司 序列查找方法、装置、设备及介质
CN115527612A (zh) * 2022-10-28 2022-12-27 天津大学四川创新研究院 基于数值特征表达的基因组二四代融合组装方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103793626A (zh) * 2012-10-29 2014-05-14 三星Sds株式会社 碱基序列比对系统及方法
US20170237445A1 (en) * 2014-08-05 2017-08-17 Illumina Cambridge Limited Methods and systems for data analysis and compression
CN107844684A (zh) * 2016-09-18 2018-03-27 深圳华大基因研究院 基因序列比对方法和装置
WO2022041881A1 (zh) * 2020-08-24 2022-03-03 华为技术有限公司 序列查找方法、装置、设备及介质
CN114121153A (zh) * 2021-11-23 2022-03-01 广州金域医学检验中心有限公司 基因突变位点检测方法、装置、电子设备及存储介质
CN115527612A (zh) * 2022-10-28 2022-12-27 天津大学四川创新研究院 基于数值特征表达的基因组二四代融合组装方法及系统

Also Published As

Publication number Publication date
CN120826741A (zh) 2025-10-21

Similar Documents

Publication Publication Date Title
CN102216899B (zh) 管理高速缓存数据和元数据
CN102150131B (zh) 管理高速缓存数据和元数据
US9189248B2 (en) Specialized boot path for speeding up resume from sleep state
US8639886B2 (en) Store-to-load forwarding mechanism for processor runahead mode operation
US9037554B2 (en) Bloom bounders for improved computer system performance
JP3731843B2 (ja) 命令シーケンスを実行する方法及び装置
CN120335869B (zh) 向量配置指令的实现方法、系统及存储介质
JPH05282266A (ja) 自己編成式並列コンピュータ・システムおよびその方法
CN112579156B (zh) 一种业务事件的处理系统以及处理方法、装置和设备
WO2024259679A1 (zh) 基因序列的子序列搜索方法、比对方法、系统及设备
CN109298884B (zh) 一种通用字符操作加速处理硬件装置及控制方法
JP5613252B2 (ja) 順次アクセス・バックアップ・アーカイブからデータを効率的に復元する装置及び方法
JP2000194572A (ja) Ｕｐｓ連携による自動ジョブ運用方式
CN116149803A (zh) 数据库的事务处理方法、装置、电子设备及存储介质
CN116628143B (zh) 一种语言模型的处理方法及装置
CN101814019B (zh) 一种前瞻失效的线程识别方法
TW200931443A (en) Apparatus for predicting memory access and method thereof
WO2021147360A1 (zh) 一种异步io数据处理方法、装置、设备及介质
JP3796281B2 (ja) Ｉ／ｏ処理方法
US8601241B2 (en) General purpose register cloning
US20080181254A1 (en) Data transmission method
CN104239080B (zh) 异构系统程序启动方法和装置
WO2025092509A9 (zh) 断点调试方法、断点调试装置及可读介质
CN120631451A (zh) 向量处理器的寄存器重命名方法、系统及存储介质
CN121541903A (zh) 程序写入方法、装置及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23941958

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202380094923.5

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 202380094923.5

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE